Two weeks ago, Vik wrote a column for the The Health Care Blog on the now infamous meat-and-cheese study done by a team of researchers led by folks from USC. You can read the column, and the hilarious comments, here. I sent the column to one of the researchers, using the messaging available at LinkedIn. Here is that researcher’s response in its entirety:
I feel no need to get into a debate with someone who doesn’t understand basic statistics, how research is conducted, and has written a statement that is blatantly wrong. It does worry me that you are propagating yourself as an “expert” when you can’t seem to critically evaluate or understand a study. I know that this study is not perfect, hardly any are, especially in epidemiology, but the points you bring up in your blog are completely misconstrued and show very poor understanding of research methodology.
If you had actually read and understood the paper you would see that we controlled for waist circumstance [sic] and BMI. Also, this isn’t some random population of fat, low educated, American smokers, it is a nationally representative sample–unfortunately this is what the American population looks like. Finally, the idea that you think our supplemental tables house the real results illustrates your lack of understanding about statistics or how mortality models are run.
That being said, if you come up with a legitimate critique, I would be happy to engage in a friendly debate. When you attack something, I would suggest you make sure you understand it first, otherwise it is hard to legitimize anything else you say. I find it ironic that most of the push back from this paper has been from the general public who don’t have experience doing these types of studies, while for the most part, the scientific community (at least from people at R1 universities) has been fairly receptive.
We are glad to offer this legitimate critique, beginning with what we find in the very first sentence of the Results discussion that is not in the paper itself, but in the supplementary materials: “Using Cox Proportional Hazard Models, we found no association between protein consumption and either all-cause, CVD, or cancer mortality (Table S2).” Table S1 makes the point even more clearly: all-cause mortality in the low protein group was 42.9%. All-cause mortality in the high protein group was 42.9%, meaning that there is ZERO impact on overall mortality from protein variation at the extremes.
We may not be “experts” on biostatistics and epidemiology, but we do read pretty well, and we know the difference between controlling or adjusting the data and making correlations. Put simply, controlling the data for potential confounding variables eliminates them from consideration as causative agents for the effects seen in the analysis. Thus, controlling or adjusting the data is not even remotely the same as what I charged in my THCB column.
The researcher’s riposte that the study population is representative of the US population is preposterous on its face. While demographers don’t estimate the US population’s mean age, they do track its median age, which is just over 37. Thus, it is inconceivable that the population’s mean age is anywhere near 65 because people older than that die with alarming frequency. The US population is 63% white only, 13% black, 5% Asian, and nearly 17% Hispanic or Latino. Education is a little more complex, but broadly speaking there is no way that the average educational attainment in the US is less than high school. Amongst people over 16, 28% have ONLY a high school education and 17.6% have not completed high school. This means that the remaining 55% of the population has more than a high school education.
We do appreciate all the hard work that went into trying to adjust the data for every conceivable confounding variable. But even that yeoman effort appears to have been for naught because this population of older Americans has an all-cause mortality rate that is more than double the death rate for all people in the US over age 64each year. If nothing else, that establishes quite well that data from this portion of the NHANES sample is in no way, shape or form generalizable to the broader population.
We found the conflation of human data and mouse data to be a bizarre juxtaposition. In our combined 60 years of reading the literature neither of us can think of a single large-scale epidemiology study in which researchers sought to buttress their data (which in this case needed all the help it could get) with findings from a study of mice genetically engineered to be slugs.
As many people no doubt know, cancer has been cured many times in mice, but continues to plague humans. This scientific flotsam belongs with the nut study and Ancel Keys’ ludicrous and manipulative demonization of dietary fat, which somehow became gospel and helped to make us the fattest nation on earth.
Finally, there is this: in typical journal fashion, there is an anemic disclosure at the end of the article that Valter Longo “has equity interest in L-Nutra, a company that develops medical food. The other authors declare that they have no conflicts of interest.” If that’s true, what are L-Nutra “team members” Sebastian Brandhorst and Priya Balasubramanian doing on the paper without making disclosure of their affiliation with the same company?
Did they help to secure funding? Do they have a (sweat) equity stake? Are they paid in any way? Are they working for free on hopes of a big commercial payoff down the road? Where was the adult supervision of this work and the disclosures?
Vik Khanna is THCB’s Editor-At-Large for Wellness. He is also author of THCB’s next e-book, Your Personal Affordable Care Act: Making Yourself Scarce in the Dysfunctional US Healthcare System. Along with Al Lewis, he is co-author of THCB’s inaugural e-book, Surviving Workplace Wellness With Your Dignity, Finances and Major Organs Intact.
Alan Cassels is an expert advisor with EvidenceNetwork.ca, a pharmaceutical policy researcher, and the Author of Seeking Sickness: Medical Screening and the Misguided Hunt for Disease.
Categories: Uncategorized
