Estimating Calories Without Labels – Is Photo Recognition a Feasible Aid?

1193068639xcy

One ongoing challenge in nutrition tracking is estimating calorie content when no nutrition label is available — for example, when eating at restaurants or trying homemade or specialty foods.

Studies have shown that calorie underestimation is common in these scenarios. For example, Urban et al. (American Journal of Public Health, 2013) found that people underestimated restaurant meal calories by an average of 20%–40%. This can have a meaningful impact over time for individuals managing weight or specific dietary goals.

I’ve been wondering about the role that technology could play here. In theory, image recognition combined with existing nutrition databases (e.g., USDA FoodData Central) could give a quick ballpark calorie estimate from a simple photo. While it’s unlikely to be perfect, it might help guide decisions in the moment.

From a nutrition science perspective:

• What do you think are the main limitations of using photo-based calorie estimation?
• Are there studies on the accuracy of visual estimation methods?
• Could this approach be useful as a supplementary tool alongside traditional tracking methods?

I’d be interested in hearing any peer-reviewed research, personal experience, or professional opinion on whether photo-based calorie estimation could realistically support better dietary decision-making.

AnnPT77

MFP Premium has a "Scan a Meal" option when adding food, though I think it's only available in the phone/tablet app, not in web browser MFP. It may not be obvious, because the user has to scroll the graphic bar near the top to see it. (The complete set of options I'm seeing there is "Voice Log", "Scan a Barcode", "Scan a Meal", "Quick Add". Screen grab below showing the meal scan option in the menu.

I have no peer reviewed research or professional opinion, but I have tried it, so I guess that's personal experience.

Personally, I don't find it helpful, maybe because I eat weird things that it isn't good at recognizing. I think it's more of a marketing gimmick than a useful function, but I may be biased.

I don't really see how it would be accurate even in theory: We still don't have a good way to know how much oil was used in making a dish, which can be a big calorie variance; and I don't think it can necessarily know things like how thick a cheese layer is. (In general, the MFP function makes the user specify some quantities, though I'm not going to belabor how it works.)

If you'd like to see it, sign up for premium and try it.

Alatariel75

I genuinely don't see how AI looking at a photo could make a better guess at what's on the plate than me, the actual human who has the fork. AI wouldn't even know whether the white component is cream, sour cream or yogurt, but I will. AI may or may not be able to identify the type of meat, but I'll know.

This to me seems an education piece. Better to furnish people with the tools to make a best guess estimation, rather than to try and leverage (incredibly energy hungry, environmentally disastrous) technology to try and identify something the person logging should already be able to to a far greater extent.

I think that a big component of people underestimating their restaurant meals is wishful thinking - "oh, it can't possibly be that many calories! It wasn't that big! I didn't have that much!" and I don't think that using a photo scan to suggest what may have been on the plate is going to overcome that.