More sugar research from the USA.

stealthq wrote: »

SapiensPisces wrote: »

goldthistime wrote: »

SapiensPisces wrote: »

goldthistime wrote: »

SapiensPisces wrote: »

goldthistime wrote: »

stevencloser wrote: »

Small sample size, high daily dose (which 60 kg person drinks 2 liters of soda every single day?), sprague dawley rats that get health problems if you so much as look at them funny.

WRT sample size, perhaps you or someone else could help me unravel the statement " Sample size was chosen to yield > 80% statistical power to detect 30% between-group difference with 10% within-group difference in a phenotype using two-sided Student's t-test. " Surely this doesn't mean that the authors of the study chose only the 8 rats (out of presumably more rats) that gave them the results that they had been seeking?

Statistical power is a measure of how well a trend can be detected in your data. There are methods to determine how many samples you need to obtain a pre-set level of statistical power. That does not mean that the authors rigged the study. It just means that they were careful in determining the minimum number of samples they set out to acquire based on their choices of how strong they wanted their results to be. The caveat is that statistical power has a lot to do with expected variance. For data that has high variance, more samples are required to tease out any possible non-random trend. It also depends on what level of significance you want to achieve. Statistical significance refers to how likely it is that your data is the result of a non-random process.

IMO, as a data scientist and mathematician, a sample size of 8 is grossly insufficient to determine, with any reasonable certainty, whether or not a non-random trend exists, much less is causative.

Thanks. I certainly get variance, p-values and the like, but am still a little baffled. Does this mean that they predicted very little variance and so could choose a relatively small sample size? Wouldn't this negate any dismissal of the results using the argument that it is too small a sample size?

They probably calculated variance from their samples, which, being such a small number of them, means that they got a very weak estimation of the expected variance. It reinforces the idea of being highly skeptical of the results.

More importantly, if you look at the criterion they chose (> 80% statistical power to detect 30% between-group difference with 10% within-group difference), these are very weak criterion, since it means that they, for example, only require a sample size large enough to determine with ~80% certainty that they can detect a non-random trend that has at least a 10% difference in magnitude in their within-group data. That means that there is ~20% chance that they cannot determine, with any certainty, if a significant non-random trend of such magnitude exists.

This, combined with the very weak estimate of variance, leads to a lot of room for skepticism.

Thank you for clearing that up!

Sure! Sorry if I came across as condescending in my first reply. I never know how much background others have and try to err on the side of full explanation whenever possible.

ETA: In studies like this, it's not uncommon to see very small sample sizes being used, simply because getting reliable unbiased data on biological processes, even in a laboratory, is really expensive and difficult. High levels of uncertainty and variance make detecting meaningful trends much more difficult than in the physical sciences. So, I'm not saying that the paper is bad outright, because it's probably fairly common to see this problem, but I'm still really skeptical.

+1

Part of my job is helping with study design. There's a not very funny and unfortunately often true joke that when an Investigator asks us how many samples he needs for his particular study, the answer is: take the money you have and divide by the experiment cost per sample. That's sufficient power.