Export and analyze your own data in Excel

1356715

Replies

  • icklistpiklist
    icklistpiklist Posts: 64 Member
    Thanks
    :flowerforyou:
  • LaLa482
    LaLa482 Posts: 82 Member
    Nerd alert

    Naaawh, it's not done in C# ;)

    Now, if he developed web app with a sql database back end and imported the data in on the other hand... ;)

    Nice stuff, though the download link is dead, anyone else upload it please? =)

    New version

    http://bit.ly/HhXhfF

    I've done web apps, and more dev languages than I can think of but I do not want to do this on a web service because I don't want to store people's data.

    I've seen that a few people have issues with exporting the weight data so I will look into that.

    Oh be still my heart! This is much more user friendly that what I put together.

    Thank you so much for doing this! Awesome contribution!
  • heybales
    heybales Posts: 18,842 Member
    Now there's the way to use a spreadsheet, awesome.
  • CarynMacD
    CarynMacD Posts: 230
    This is the most AWESOMEST thing ever. Definately beats my boring excel spread sheets. Thanks!
  • EvgeniZyntx
    EvgeniZyntx Posts: 24,208 Member
    Ok, I have been able to work out a different import method for the weight data that *might* solve the import issues for some people.
    (If you can't get the sheet to work at all and import any data - sorry this is a fix for those that could import some data)

    I will be publishing a new version shortly - it has the following updates:

    Improved TDEE calculation and outputs (it tells you the equivalent body fat based on a reverse Ketch McArdle solution)
    Improved feedback on protein and fat macro settings which take into consideration your bf% as set in the setup tab
    Two weight import methods (one used an XML file)
    TDEE calc bug repaired
    Weight trend graphing can be set to 30, 60 or 90 days.

    Known issues

    Import of data to calories and macros is slow - this is due to each date requiring a web query, I've thought about running off the XML files but that's a big rewrite with uncertain results
    Meal import is slow. C'est la vie.
    Weight import is a few minutes, really can make it faster without significant recode

    Intent for future

    I'd like to think about only loading partial dates and the possibility of just adding the latest missing data. Need to think about that fix.
    I'll also write a blog out about how this was done and possibly provide snippets for other if they want to play with the code.
  • EvgeniZyntx
    EvgeniZyntx Posts: 24,208 Member
    Latest version 2.8

    http://bit.ly/1bZtgf1

    See note above
  • Cranquistador
    Cranquistador Posts: 39,744 Member
    Latest version 2.8

    http://bit.ly/1bZtgf1

    See note above
    in
  • jofjltncb6
    jofjltncb6 Posts: 34,415 Member
    Was getting a lot of zeros for total calories (despite the food diary having a non-zero value). Came back here to see if others were experiencing the same thing and see that v8 is out. Guess I'll stop trying to figure out v3.
  • danasings
    danasings Posts: 8,218 Member
    Bumping for FL...alas, I have a Mac and can not participate. :cry:
  • deksgrl
    deksgrl Posts: 7,237 Member
    Bumping for later when I'm on my own computer.
  • The_Enginerd
    The_Enginerd Posts: 3,982 Member
    Much thanks for this. I worked on my own import for all of the items through the reports function, but it was more manual and clunky. I need to learn some VBA so I can automate more of it.
  • lilawolf
    lilawolf Posts: 1,690 Member
    The engineer in me just had an orgasm.... Thanks man!
  • jeicorsair
    jeicorsair Posts: 7 Member
    New version fixed (most) of the weight import issues for me. I tried both methods, and they got my weight stats. For some reason, it's still not calculating TDEE in the Graphs tab. Calc tab gives me eTDEE values, but nothing in the oTDEE (#DIV/0! and #N/A errors). Importing Calorie data is still just giving me dates as well, though I've been tracking that with my own spreadsheet so I'm not as worried about that. Using Excel 2010.
  • samammay
    samammay Posts: 468
    I previously could not get weight with the v6. in v8, the first method fails, the second method hangs (I stopped it at 25 minutes).
  • QuietBloom
    QuietBloom Posts: 5,413 Member
    It looks fantastic! Unfortunately, I don't have the proper version of Excel. :cry:
  • ferreteers
    ferreteers Posts: 49 Member
    Latest version 2.8

    http://bit.ly/1bZtgf1

    See note above

    Will try tonight!
  • Thanks, bump for downloading later. cheers
  • stephenson2012
    stephenson2012 Posts: 94 Member
    Will try when I get home. Thanks for the hard work.
  • mkwongh
    mkwongh Posts: 279 Member
    bump for when i get home. thanks so much. Just what i was looking for
  • jpolinisse
    jpolinisse Posts: 149 Member
    I previously could not get weight with the v6. in v8, the first method fails, the second method hangs (I stopped it at 25 minutes).

    This.
  • kyleekay10
    kyleekay10 Posts: 1,812 Member
    I previously could not get weight with the v6. in v8, the first method fails, the second method hangs (I stopped it at 25 minutes).

    This.

    Having this problem also
  • jdaley90
    jdaley90 Posts: 259 Member
    This is awesome! Thanks for sharing.
  • jollyjoe321
    jollyjoe321 Posts: 529 Member
    Nerd alert

    Naaawh, it's not done in C# ;)

    Now, if he developed web app with a sql database back end and imported the data in on the other hand... ;)

    Nice stuff, though the download link is dead, anyone else upload it please? =)

    New version

    http://bit.ly/HhXhfF

    I've done web apps, and more dev languages than I can think of but I do not want to do this on a web service because I don't want to store people's data.

    I've seen that a few people have issues with exporting the weight data so I will look into that.

    High five ;) Though I'm very much on the start of this path than as far up it as you! =)
  • micheleb15
    micheleb15 Posts: 1,418 Member
    This is so great - thanks!
  • shadow2soul
    shadow2soul Posts: 7,692 Member
    Looks awesome :drinker:

    Sadly, I only have the starter edition of excel and as such I can't make use of this awesomeness.

    Well, I was really interested in seeing the observed TDEE, so I entered all the calorie information and weight information manually.

    Really interesting to see the Observed vs Estimated TDEE chart. I kind of feel skeptical about it (maybe it didn't work right since I entered information manually).

    The chart shows my observed TDEE being right around eTDEE(1.55) and higher. While fitbit has me between eTDEE(1.2) and eTDEE (1.375).
  • heybales
    heybales Posts: 18,842 Member
    Looks awesome :drinker:

    Sadly, I only have the starter edition of excel and as such I can't make use of this awesomeness.

    Well, I was really interested in seeing the observed TDEE, so I entered all the calorie information and weight information manually.

    Really interesting to see the Observed vs Estimated TDEE chart. I kind of feel skeptical about it (maybe it didn't work right since I entered information manually).

    The chart shows my observed TDEE being right around eTDEE(1.55) and higher. While fitbit has me between eTDEE(1.2) and eTDEE (1.375).

    As long as you don't have suppressed your metabolism - you are talking about the difference between observed TDEE based on actual lost weight, and estimated calorie burn.

    The only iffy part of the equation is your logging of food eaten - was it correct, no holes or gaps, correctly weighed, ect?

    Going on assumption weight lost is fat, that means each pound lost is an assumed 3500 deficit for however long it took you to lose it.

    If your avg loss over time is 1 lb weekly, that is 1 x 3500 / 7 = 500 calorie deficit you must have had daily from your TDEE.

    So add that 500 to what you say you ate, and if accurate enough, eaten + 500 = observed TDEE.

    The only assumption problem is if you are losing a lot eating well under what you should, cardio only, not enough carbs or protein, no strength training - then you are actually burning off muscle mass. And muscle mass only provides 600 calories per pound.
  • shadow2soul
    shadow2soul Posts: 7,692 Member
    Looks awesome :drinker:

    Sadly, I only have the starter edition of excel and as such I can't make use of this awesomeness.

    Well, I was really interested in seeing the observed TDEE, so I entered all the calorie information and weight information manually.

    Really interesting to see the Observed vs Estimated TDEE chart. I kind of feel skeptical about it (maybe it didn't work right since I entered information manually).

    The chart shows my observed TDEE being right around eTDEE(1.55) and higher. While fitbit has me between eTDEE(1.2) and eTDEE (1.375).

    As long as you don't have suppressed your metabolism - you are talking about the difference between observed TDEE based on actual lost weight, and estimated calorie burn.

    The only iffy part of the equation is your logging of food eaten - was it correct, no holes or gaps, correctly weighed, ect?

    Going on assumption weight lost is fat, that means each pound lost is an assumed 3500 deficit for however long it took you to lose it.

    If your avg loss over time is 1 lb weekly, that is 1 x 3500 / 7 = 500 calorie deficit you must have had daily from your TDEE.

    So add that 500 to what you say you ate, and if accurate enough, eaten + 500 = observed TDEE.

    The only assumption problem is if you are losing a lot eating well under what you should, cardio only, not enough carbs or protein, no strength training - then you are actually burning off muscle mass. And muscle mass only provides 600 calories per pound.

    My logging isn't perfect. I totally admit that.
    I have logged everyday since March 13 of this year. Although I only entered information from Aug 20 till now into the spreadsheet. (not sure if adding more information would change the result or not).
    I don't log crystal light (even though I drink about 80 calories worth a day).
    I don't log when my son gives me one of his crackers (or anything else he offers me).
    If I weigh out food and then my son starts eating off my plate as well, I still log what I had weighed.
    If my weight loss ever stalled out, then I'd definitely start logging the little things that I currently don't.

    Trendweight is estimating I lose about 1lb of muscle for every 5lbs of total weight lost. :frown: I do strength exercises with dumbbells and resistance bands currently since I don't have access to a barbell.
    I knew my fitbit underestimated a bit, but I was thinking it was only about 250 calories off.
    Oddly enough, the lowest observed TDEE from the spreadsheet, is actually what my scale tells me is roughly my TDEE.

    I would hope I haven't suppressed my metabolism. Up until recently my average calorie intake was 1886 per day (adding 500 to this gets me roughly the same as the lowest observed TDEE). Recently I've felt like I'm starving so I've been upping my calories slowly until I find the magic number where I maintain. I'm currently averaging around 2000 calories, not working out....and still losing weight (although slower than before). :grumble:

    Talking it out like this, it might not be too far off then. It just seems high for my height and weight.
  • EvgeniZyntx
    EvgeniZyntx Posts: 24,208 Member
    I've posted a blog where I will keep the latest version and try to avoid people hunting for it throughout this thread.

    Blog can be found here:

    http://www.myfitnesspal.com/blog/EvgeniZyntx/view/mfp-extractor-and-trend-watcher-the-program-post-589839

    Latest version: http://bit.ly/H5g1zm

    I'll also adress what errors and guesses can exist in the oTDEE and how it comes up with the numbers. Just give me the time to write that up and answer your question properly. :)
  • darkestdayz
    darkestdayz Posts: 117 Member
    I'm getting an error that says macros are corrupt...the popup never comes up at all. Office 2007 and I have IE 10.
  • nz_deevaa
    nz_deevaa Posts: 12,209 Member
    Bumping for when I get smarter.