That after work drink

Options
Advice on how to not alienate coworkers, but also to try to go out every so often? I work in the food business, so there's always someone asking me to go someplace afterwards. Night eating and after work eating is my worst habit, so this doesn't help. I do want to go sometimes, but then I end up ordering food. Sometimes I'm good and at least box up half of it,, sometimes someone will split with me. I also am in a medical program right now, so i'm always asked to go to lunch, but normally i split with someone or get low fat options. The after hours dining just doesn't have the same low fat variety~!

My old weight loss doctor told me most people won't understand, but you have to be firm. What are some nice ways to let them down and also some things to watch out for if I do go out with them?