Accuracy of fitness apps

flsilver

What is the accuracy of, for example, Map My Walk?

yirara

In what sense? Distance walked or calories burned? I think the Map My series of apps is among the worst when it comes to calories: highly overstated, and on top of that at least in the past it only showed gross calories, which is the calories your body burns if you were in a coma anyway to provide energy for brain, organs, basically keeping you alive, plus the exercise on top. You want to see net calories, which is only the exercise as MFP already takes the body functioning bit into account.

Jthanmyfitnesspal

Another issue of all tracking apps is GPS accuracy. The GPS signal can be blocked by any building, even if you are walking on the sidewalk next to a building or on a street between two buildings. The signal is also attenuated by foliage-- particularly wet foliage. It is actually attenuated by your body: you will get somewhat better accuracy if you put your phone somewhere with a "view of the sky."

If you want to see what I mean, zoom way in on your track with the satellite image turned on. You will see where the track deviates from sidewalks, etc. It looks more like a track a dog would make, running all over the place!

Still, the distance estimates are usually OK so long as you're going in a reasonably straight line most of the time.

yirara

Yep, true. This is exaggerated by apps only using a few datapoints to create as little data as possible and to send as little data as possible to a server somewhere. It's easy to test this by doing a test walk in a 90 degree angle, or to do a u-turn. I guess overall it doesn't play much of a role unless you want to track your exact 5k, 10k, whatever running distances. The calorie estimate accuracy is a much bigger problem in many apps.

AnnPT77

Close enough to be useful in some contexts, when used with insight about their possible limitations.

Ask a general question, get a general answer?

I think if you ask a specific question, you may get better (more specific) answers. For example, if your question is "How do I get the most accurate estimate of walking calories", or "I've been using MapMyWalk and wondering if the speed/distance are accurate" or something like that, I'd answer differently.

I wear a Garmin device. When I take an intentional walk, I tell it that's what I'm doing. I've rough-checked its speed and distance results against known-distance/known-duration walks, and found it to be reasonably accurate. I don't trust its calorie estimates for walking (for a variety of reasons I won't belabor), so I put the speed/distance into this page (with the energy box set to "net"), and use that calorie estimate:

https://exrx.net/Calculators/WalkRunMETs

It's quite a conservative estimate, may even be too low. I think estimating zero calories for exercise isn't a good plan, but I do prefer to err on the low side, since error seems inevitable.

Jthanmyfitnesspal

Calorie estimates for walks and runs should be reasonably accurate and consistent using these devices. They may not be, but they should be. The formulas are very well established.

Garmin underestimates swimming, I think. (Based on my comparison to a number of other estimates.) I usually mentally add about 20%.

Cycling is also a funny case that I've discussed many times here. According to various publications, there is an unknown efficiency factor for power applied vs. calories burned. I'm going to guess that less experienced/fit cyclists are less efficient than more experienced/fit ones. This probably applies to all activities. Hmmm, it might particularly apply to swimming, which takes a lot of experience to do efficiently.

Finally, I've turned on my Garmin during various workouts (like calesthenics), but I'm really not convinced the calorie estimates are worth a hoot.

yirara

Jthanmyfitnesspal wrote: »

Calorie estimates for walks and runs should be reasonably accurate and consistent using these devices. They may not be, but they should be. The formulas are very well established.

For my, Garmin gets running fairly right using established and published equations. Cycling possibly a bit too low (difficult to say), but walking! Compared to running for the same distance (and hence longer duration) Garmin gives me nearly the same net calories for walking! That's just crazy. For a relaxed 15km stroll through a forest I might get 1200 net calories. The calculator posted by AnnP gives me about 500, which is likely a lot more reasonable.

Jthanmyfitnesspal

yirara wrote: »

For a relaxed 15km stroll through a forest I might get 1200 net calories. The calculator posted by AnnP gives me about 500, which is likely a lot more reasonable.

Hmmm. I'd put a relaxed stroll at ~20min/mile, so a 15km walk would be getting onto 3 hours. ~400kcals/hour for gross calories burned doesn't sound that high to me for walking (it would include 80kcal/hour for just breathing).

I know you have a lot of experience with this, so I assume your current plan works as expected. Maybe you are a very efficient walker!

AnnPT77

I'm just going to say up front that all of everything below is in my understanding or in my experience, but I am not a scientist in any relevant field, nor an engineer. This is a subject I've worked at understanding as best I can, but that's pretty feeble and I know it. Assume every sentence below has "in my understanding" (IMU) or "in my experience" (IME) or "in my opinion" (IMO) in it.

Also, if you don't want to read a stupid-long semi-rant, stop reading now, please.

Jthanmyfitnesspal wrote: »

Calorie estimates for walks and runs should be reasonably accurate and consistent using these devices. They may not be, but they should be. The formulas are very well established.

The formulas are "well established" in the sense of being research based. But the formulas themselves contain the seeds of inaccuracy - conceptual limitations. Those seeds will bear fruit in a larger or smaller percentage of cases depending on what the flaw is.

Garmin underestimates swimming, I think. (Based on my comparison to a number of other estimates.) I usually mentally add about 20%.

Cycling is also a funny case that I've discussed many times here. According to various publications, there is an unknown efficiency factor for power applied vs. calories burned. I'm going to guess that less experienced/fit cyclists are less efficient than more experienced/fit ones. This probably applies to all activities. Hmmm, it might particularly apply to swimming, which takes a lot of experience to do efficiently.

I've been told (by people who would seem to have reason to know) that the range of efficiency for cycling between the experienced and inexperienced is fairly narrow, that being one reason why the calorie estimate from average cycling power is reasonably accurate - probably one of the more accurate methods available.

One problem with this general "efficiency" idea (going beyond cycling) is that there are multiple types of efficiency, and they apply differently to different activities.

In many activities, a less-skilled person being inefficient means they'd burn more calories than a skilled person doing the same thing (because they waste motion in ways that don't drive the metric used to estimate calories, such as pace/speed). But it seems like sometimes we see people here thinking being "inefficient" at an activity means burning fewer calories.

That's similar to (but in the opposite direction of) the pretty common belief that calorie burn scales in parallel with how an activity feels (hard or easy), or how much sweating it causes, or that sort of thing. It's just not true.

On the flip side, when using heart rate as a metric for estimating calories, the device is more likely to overestimate calories for the less-skilled person (because of conditioning effects, loosely - those can be very activity specific - more about that later). The person's actual body is less efficient, in effect.

It matters what the estimating metric is. I'll talk about an activity I know pretty well, machine rowing. Some of the machines, including the Concept 2 that I use, measure applied power quite accurately. Applied power can be used to estimate calorie burn, because physics . . . sort of.

When I watch less-skilled people at the gym use that rower, they are often spending quite a lot of energy (I could be specific about how), but they're doing it in ways that don't make the flywheel spin faster. The monitor will show that their pace (theoretical meters per time period) is slow, and their power application low. The calorie estimate based on power will then be inaccurate on the low side.

For many activities, the best available metric is heart rate, but it isn't a very good metric for a lot of activities for which it's the best available. (More on that later.)

Finally, I've turned on my Garmin during various workouts (like calesthenics), but I'm really not convinced the calorie estimates are worth a hoot.

They're usually heart-rate based, and that has theoretical problems for estimating a thing like calisthenics.

Moderate steady state cardiovascular exercise is the type in which oxygen demand best correlates with calorie expenditure. It's oxygen demand that correlates pretty well with calorie expenditure. Heart rate is just a proxy, not always a good one, because many other things (beyond oxygen demand) can increase heart rate. (Examples: Physical strain, intra-body pressure (like Valsalva effects), higher ambient heat, dehydration (even somewhat moderate dehydration), caffeine and other drugs, alcohol, emotions, and more.)

Loosely, the more unlike moderate steady state cardio an activity is, the poorer the correlation between heart rate and calorie expenditure. Strength components in the exercise create distortion, because intra-body pressure and strain also raise heart rate in those cases. Overhead strength movements may make that even a little worse. (I'd expect those things to affect heart rate estimates for calisthenics calorie burn.)

Interval exercise breaks the algorithms (to some extent) because heart rate response lags effort, i.e., HR stays lower into the high-effort stages of the intervals, remains high into the less-intense stages of the intervals, and how high it stays for how long is variable from one person to the next (probably mainly to do with fitness adaptation, but maybe with some contribution from genetics, dunno). In addition, heart rate tends to drift upward over a longer session of steady-state cardio even at constant intensity, for a variety of reasons.

The research-based correlation of calories to oxygen demand to heart rate also breaks down because of how algorithms apply it: Generally, people are relying on age based estimates of max heart rate, but those estimates are inaccurate for a fair fraction of people. The implication is that if the device uses HR to figure out how hard a person is working, it will potentially be guessing wrong for those people who are non-average.

In that same vein, one fitness adaptation is that for experienced cardiovascular exercisers, the heart actually gets stronger, and pumps more blood per beat, thus delivering more oxygen per beat. (It's oxygen demand the correlates with calorie burn, remember.) The implication is that in 2 people who are otherwise completely identical, doing the same activity at the same objective pace and efficiency, the one who's very fit will have a lower heart rate than the one who's not fit - possibly much lower. A simple heart rate monitor will give them very different calorie estimates. They aren't burning dramatically different calories: It's a limitation of the device, or of the estimating algorithm.

(Yes, some fitness trackers adjust to personalize some kinds of estimates. That may reduce the effect of that problem, vs. a simple heart rate monitor. Dunno.)

I don't mean to be catastrophizing here, because these estimates can be plenty close enough to be useful in most normal cases. To put it in an extreme way, let's say my 250 calorie estimate for an exercise is double or half reality: A big error rate, 50% or 100%, right? So I might've burned 500 calories, or 125, but I estimate 250. Let's look at the over-estimation case.

Most people are doing their exercise fewer than 7 days a week. So, if I do that exercise 4 days a week, I'm overestimating exercise calories by 4 x 125 or 500 calories per week. If my loss rate target is a pound a week, I've wiped out one day's deficit by overestimating the exercise. Instead of losing a pound that week, I'd lose 0.86 pounds. Leaving aside the probability that my food intake estimates - no matter how meticulous - might be that far off, too, the difference between 0.86 pounds vs. 1 pound is only going to show up over multiple weeks, and ideally if I'm paying attention, I'll adjust my calorie intake to make 1 pound happen anyway.

To look at it another way, for me, that 125 calorie per day over-estimate is maybe 5% of my TDEE. In the big picture, a careful estimator will be over sometimes, under others, so I have a hard time getting excited about that 5% off in a particular case.

The point is, one has to look at the probable error in context of one's target deficit over time, and/or in context of what one's TDEE is. One also needs to look at things like frequency of the exercise.

Of course, exercise calorie estimates can be off by much more than that, but I think that's going to be primarily in cases of extremes (more exercise volume than most people do), or truly poor choice of estimating methods for a particular exercise (using heart rate to estimate pure strength workouts would maybe be an example, or an estimate from methods that may overstate calories for marketing reasons).

As I understand it, there is a flaw in how MFP applies the research-based exercise estimating method they use (METS). I believe it overstates the exercise calories by estimating gross calories during the exercise, not net calories. In the big picture, that can matter more for long, low-METS things (like slower, long walks), but less so for shorter, higher-METS things (like running sprints, maybe). The mathematical reasons why that might be so should be obvious.

Caveat: I've probably done some arithmetic inaccurately somewhere in here. I frequently do. I'm pretty good at math (understanding and applying formulas) and pretty bad at arithmetic (times tables and carrying the one, etc.).

Jthanmyfitnesspal

@AnnPT77 : Well, you warned us at the start of the post!

I agree with all of it. If I'm determined to cut, I don't eat back all the exercise calories. All of the estimates are for the average case, and no one is average!

BrianSharpe

flsilver wrote: »

What is the accuracy of, for example, Map My Walk?

For distances within the margin of error usually associated with civilian GPS Typically most apps also overestimate calorie expenditure. A good working number for net calories expended walking (assuming reasonably level terrain) is 30 cal per 100lb of bodyweight for every mile walked (eg a 200 lb person would expend an additional 60 cal for every mile walked) at speeds up to 5mph.

yirara

In the past Map My Walk and other apps of that series would only display gross calories. However, net calories is what's important. If this did not change by now and someone were to trust that number this person would be eating too much back. Of course this doesn't matter much on a 15 minute walk, but this would certainly play a role on a longer walk.

AnnPT77

yirara wrote: »

In the past Map My Walk and other apps of that series would only display gross calories. However, net calories is what's important. If this did not change by now and someone were to trust that number this person would be eating too much back. Of course this doesn't matter much on a 15 minute walk, but this would certainly play a role on a longer walk.

If a person cared, it'd be reasonably easy to back out a reasonable estimate of the BMR component of that. Also backing out the daily life activity multiplier is fairly easy if relying on MFP (or some other calculator that makes multiplier values explicit) to get the pre-exercise calorie goal. It's more complicated if using a personalized (own data) calorie estimate, but one could probably get a ballpark amount to back out.

westrich20940

I'm not sure I could find info on the accuracy of any/all fitness apps...I use RunKeeper.

It will all be an estimate though. If the fitness app you're using is using GPS...that's not entirely accurate. I've done hikes with others using Strava and our distances at the end do not match, sometimes by quite a lot.

However, I lost weight while getting back into running --- and I used a Polar HRM w/chest strap at first. You put your personal stats into the HRM and then it tracks your heartrate (which it's fairly accurate for)...so it will know things like (lowest HR during the workout, highest HR during the workout, and avg HR during the workout)...it will use that info to make an estimate of calories burned during that workout. So when I was running I'd have an estimate of calories burned from RunKeeper (which also has you put in your personal stats and makes the estimate of calories burned based on your distance, pace, elevation), an estimate of calories burned from my HRM, and I'd simply Google what the avg calories burned during a run was (with my personal stats, my avg HR during the run, distance, time, etc.). I used those THREE numbers to then manually put in a calorie burn in MFP - I'd simply pick a median number to put in. I lost weight at the expected rate...so I assume I was more/less accurate in doing it this way.

I found personally that when my workouts were more intense (when my max and avg heart rate was higher --- so a run on a route that was more hilly for example, or for whatever reason, my pace was faster)...my HRM would give an estimate that was - at times - 100-150 calories *higher* than my RunKeeper or a Google estimate. When my run was more moderate/consistent....they were essentially the same.

So, once I stopped using the HRM (I lost my wristwatch...lol) I simply would use the estimate RunKeeper gave me since it was the lower one by a bit and I figured...that'd give me some wiggle room with what I was eating to have a bit of a buffer or if I wanted a piece of candy or something I didn't worry about it.

EDIT: generally speaking...even HRM (even with chest straps) are accurate for HR, but not necessarily for calories burned based on that HR. So, I expect something like an Apple Watch or Fitbit has the same accuracy issues probably? But they are good sources of info to start with and make adjustments as necessary.

AnnPT77

westrich20940 wrote: »

(snip)
EDIT: generally speaking...even HRM (even with chest straps) are accurate for HR, but not necessarily for calories burned based on that HR. So, I expect something like an Apple Watch or Fitbit has the same accuracy issues probably? But they are good sources of info to start with and make adjustments as necessary.

Some problems same, some different. Generically, the fitness trackers can have GPS, altimeters, motion detectors, etc., in addition to heart rate. They can use those in various ways (various algorithms) in conjunction with user profile data and sometimes user-entered exercise types. That enables more sophisticated algorithms, with potentially more accurate results . . . but it doesn't necessarily achieve better estimates in practice. It's still a statistical estimate, besides.

I've heard the rumor that some fitness trackers now use METS to estimate strength training calories, same method MFP uses. METS is also available to a tracker, but relies on duration, user data, and type of activity rather than any of the other stuff.