Saying 'No' to unplanned food and how to deal

So, I just wanted to talk to ya'll about each one of you deal with saying 'No, Thank you,' To the every polite and genuine ' Would you like some?'

I feel like at work, or when I come home its a constant question. Though I am always more than happy to share my meals with others, there are times I have to say no with such ferocity it almost gets me down. It is totally INFURIATING. For example, I came home, my brother was relaxing and asked the dreaded questions. " Would you like some of this? I can't eat anymore." I dont even bother to look at it. My brother, a bean pole at 6 foot, is addicted to sweet and sugary things. " No dude, thanks though. I appreciate it. I do my thing, shedding all of the **** I seem to accumulate from work to home.
" Are you SURE you dont want this? Its awesome!"
" No thanks, really. I already ate."
" C'mon you gotta finish it for me! It is just so good!"
" Seriously! Stop asking! No thanks!"

I feel like this attitude of " You have to have this!" Or " You know, if you are losing weight, you really shouldn't eat bananas." Follows me around EVERYWHERE. Its like I am embarrassed to bring my food to work because someone is going to make a comment about how healthy I am eating, and how they could never eat like that.
/rant

Anyway, how do you folks deal with these kinds of questions?
Or how do you deal with the the attitudes that come with your lifestyle change?