lost while not working out/ calorie counting

I stopped calorie counting some months ago and try to do it once a while when I think am going way over my regular intake.
something un understandable happened. this peak holiday season when I was out, eating at parties n regularly consuming desserts on a frequent basis and consuming more than i do on a normal day, and not exercising, I lost pounds. am much closer to my goal weight now than when i was watching n counting calories n working out.
why did this happen? also, does it take a long time to reflect all that I ate n binged with a weight increase in the future?