any tips for exercising with bad knees and feet?

Options
I have just started going back to the gym after a few years (40lbs gained in those years ). I have been slowly working up to re-joining and have myself in a good routine of going five days a week and exercising 30-40 minutes. The problem is that because of my tiny feet and weak ankles and large body, my knees hurt, and the soles of my feet do as well. I realize that if I lose the weight this problem will get better, but in the mean time it feels worse right now and I am afraid that I will do permanent damage. It's sort of a catch 22- I need to lose the weight to help the problem, and the exercise is what is going to help me do that. I have been alternating walking on the treadmill with the elliptical machine and the stationary bike. Is alternating the way to go? I know the treadmill is tough on the knees so I added the elliptical and bike to give my knees a bit of a break, but then my feet go numb. Is that normal? Am I doing the right thing here? Any advice?

It took a lot for me to walk back into that gym again and it takes a lot for me to go back every day, but I have myself in a nice little routine that I don't want to break.

Thanks!