Best exercise to start with?

Walking.

Also, the first two winters I was losing weight (I lost 90 pounds) I went to deep water running classes at our local rec centre. Once I got past that initial walk from the change room to the water in a bathing suit, I absolutely loved it and usually went three times a week. You can make the class as easy or as difficult as you like so as you get fitter, you just work harder. Its so kind to your joints and you wear a foam belt so no worries about sinking. If that sort of a class is an option for you, I'd definitely recommend it.