lowered libido w/ diet and exercise?

well for a while (3 months) I was bulking up (eating about 2800 calories per day and almost no cardio---just walking the dog 2 times a day)

I did my last hurrah around Christmas time (I was planning a short break from the gym of 4 days and then try to maintain my muscle size gains and get my stomach back down),

we were on a weekend vacation (4 days) and I ate a bit of junk. All was good the first 3 days, but then day 4 and it wasn't happening :-( I chalked that up to all the junk I ate and the fact that I hadn't been in the gym and whatever....

then we returned and I started back running and lowered my calories to 2400...and it has been hit or miss since then :-(

Is it just that my body needs to adjust back to my new routine and nutrition? I am 40, could that be part of it?

I went to the doctor and he said it could be winter blues, or rapid weight gain loss can lower libido as well (but since I started back running and stuff I only lost 3 lbs about 1 lb a week, (i started on 2 Jan))

ideas? help?