Chance favors the well-prepared, and Professor Baicker jumped on this research windfall. She found that providing Medicaid–and thereby facilitating access to basic preventive medical care–for the previously uninsured did not improve physical health status, but did increase diagnoses and utilization. Because of the soundness of the methodology, the conclusion were unassailable – more access to medical care does not improve outcomes or optimize utilization, which is a proxy for spending. (We ourselves reached a similar conclusion based on a similar analysis on North Carolina Medicaid’s medical home model.)
Yet Professor Baicker herself used exactly the opposite methodology to reach the exact opposite conclusion for workplace wellness. And that’s where the identity crisis begins.
She and two colleagues published a meta-analysis in 2010 of participant-vs-non-participant workplace wellness programs. Somehow—despite her affinity for Oregon’s lottery control—she found this opposite methodology to be acceptable. She concluded that workplace wellness generated a very specific two significant-digit 3.27-to-1 ROI from health care claims reduction alone, with another 2.37-to-1 from absenteeism reduction. The title of the article–now celebrating its fifth anniversary as the only work by a well-credentialed author in a prestigious journal ever published in support of wellness ROI—was equally unambiguous: Workplace Wellness Can Generate Savings.
This article wasn’t just an academic exercise. It gave the Obama administration academic cover for what has proven to be the most unpopular, dishonest, and even hazardous component of the Affordable Care Act: allowing employers to financially and clinically punish employees with coercive directives to lose weight, get unnecessary checkups, and answer intrusive, distasteful, and counterproductive questions about (for example) checking their testicles.
Professor Baicker did not question her too-good-to-be-true conclusion. Yet the Law of Diminishing Returns clearly contradicts her finding. Compelling privately insured people to get more healthcare is very unlikely to improve health status and reduce healthcare expense if provision of basic insurance to a medically needy population doesn’t noticeably improve health status while increasing healthcare expense.
Instead, she reveled in the limelight, receiving 307 citations, vs. 18 and 9 for two other Health Affairs articles on wellness that didn’t support more spending on vendors and consultants. (Even 307 citations aren’t enough to satisfy one of the leaders of the wellness movement, Larry Chapman, who says this study should be cited much more frequently since it’s basically the one that supports the entire industry.) However, at some point in 2013, overwhelming evidence totally invalidated her findings. At that point – like Dee Edington and Al Lewis, both of whom had previously reversed positions when the data didn’t support their previous positions—she could have acknowledged that her initial findings had been wrong and moved on.
Instead, she neither defended her position nor clearly refuted it, choosing instead a yin and yang middle ground that shifted with every interview. The metamorphosis from Queen of Significant Digits into the Queen of Significant Doubt started in July 2013, when she announced on NPR’s Marketplace that “it’s too early to tell” if wellness saves money, and that employers need to “experiment” with these programs to “see what happens to participants’ weight and blood pressure.” Right there, she invalidated herself. First, by then she certainly knew that a participants-vs-non-participants methodology was invalid since the key “smoking gun” slide in our Health Affairs posting was already widely circulated and her own opposing Oregon methodology was being widely praised. Second, even if she is right, the financial payoff for the modest “weight and blood pressure” improvements that the best programs might generate is 10-20 years in the future — and even then only if the improvements are sustained.
But then came another personality change.
In February 2014, she blamed readers for focusing on her attention-grabbing headline, the certainty of her two significant digits, and the gist of the conclusion…while ignoring the fine print, such as a caution about publication bias. Publication bias? You think? Start with the standard publication bias that negative articles rarely get published because they don’t get cited and hence reduce the all-important “impact factor” – recall the difference in Health Affairs citations in her own wellness article vs. the others.
Add to that a publication bias specific to those journals: most of the articles comprising her meta-analysis were published in third-tier journals. Among them, these journals have exactly once published an article critical of wellness (twice if you include a book review by the esteemed Norton Hadler, whom a third-tier journal is thrilled to publish regardless of what he says, and three times if you include publication of an article by a graduate student at the University of Tasmania that accidentally undercut the true believers’ own storyline, that they are now having to explain away).
Weeks later, a totally different personality emerged: she told the editor of Insurance Thought Leadership that she no longer focused on wellness and consequently has no opinions to share. Leaving aside the irony that the wellness true believers continue to cite as gospel someone who says she has no interest in what they are citing her for, this spin further invalidates her next comment, delivered in December 2014 – when suddenly, as a result of yet another personality change, she has opines again. She told All Things Considered: “It could be that when the full set of evidence comes in, [wellness] will have huge returns on investment.”
Oops. First, she has just admitted she doesn’t follow wellness, so why speculate on future studies she has no knowledge of in a field she’s not involved in? Second, there is a rule of thumb in epidemiology: the bigger the impact, the smaller the sample size needed to discern it. An example would be smoking and lung cancer, a previously very rare disease whose cause was discernable from a handful of cases. A sample of only hundreds of veterans was needed to prove that very high blood pressure causes strokes, and studies of exercise almost always show either a physical or emotional benefit, even in small groups of people with significant disease. On the other hand, there have been probably close to a half-billion employee-years of wellness with nothing to show for themselves except results going the other way and a bunch of self-invalidating vendor lies.
So we are going to make a radical proposal to the true believers: you can continue to cite Katherine Baicker but must also note that she herself no longer supports the study you are citing — until and unless she says she does. In exchange for this disclosure, when do you cite her, we will acknowledge that you are telling the truth for a change.
Al Lewis and Vik Khanna are co-authors of Surviving Workplace Wellness with Your Dignity, Finances, and Major Organs Intact. Their website, They Said What, regales readers with stories of wellness innumeracy and deception.
Categories: Uncategorized
Gentlemen
Great post. Like you, I am disappointed that researchers of the caliber of Kate Baicker and David Cutler do not respond to the mounting debate about their paper. They should defend or disown their work rather than hope that the debate goes away.
In my mind, their paper is a product typical of high-end academic research. Two brilliant professors spot a gap in the evidence on a hot policy topic and decide to go after it. But the actual work gets done by a graduate student in his cubicle without windows or guidance, and then hastily published.
Then the problem arises that the paper becomes hugely influential and people start having a closer look. For our paper on the PepsiCo program, we reviewed in detail the seven publications that Baicker and colleagues called “high quality evidence”. We found that five of those analyzed programs that operated over 20 years ago and most of them had severe methodologic flaws. (John P. Caloyeras, Hangsheng Liu, Ellen Exum, Megan Broderick and Soeren Mattke. Managing Manifest Diseases, But Not Health Risks, Saved PepsiCo Money Over Seven Years. Health Affairs, 33, no.1 (2014):124-131)
Unfortunately, many defenders of the industry continue to take the Baicker paper at face value, while closely scrutinizing or ignoring more nuanced and scientifically sound findings.
So I herewith support your motion!
I have just finished my review of Vik’s new book:
http://tinyurl.com/q3nzbf6
Here’s another take, from someone who isn’t affiiated with us or Ron but is familiar with both viewpoints http://blog.hopehealth.com/2014/12/why-is-there-so-much-media-exposure-about-the-failure-of-workplace-wellness-lately-an-al-lewis-and-vik-khanna-alert/p=5445
So many targets, so few electrons on the internet… Thanks for the headsup– We’ll take a looksee at Mr. Robbins but we just need to get current business taken care of. And Mr. Robbins may be bogus science, but invalidating bogus science requires actual talent, whereas invalidating most vendor statements requires only knowledge of fifth grade math, which most of the industry seems to have been absent for.
This posting is not that situation. in this case, it’s just a question of a very talented and accomplished researcher perhaps assuming that wellness vendors did understand basic arithmetic, and not realizing that most of wellness is so ridiculously bogus that you can’t assume that peer-reviewing things in wellness promotional journals means anything, as she could have safely assumed in a legitimate corner of health services research. So doing a meta-analysis of bogus papers just gives a very bogus number.
Still, I don’t know why she didn’t cut-and-run a long time ago, like I did, when I realized my own advocacy of this activity was sorely misguided.
This posting was more so that all Professor Baicker’s comments are in one place. That way when the wellness ignorati cite her (again), and we have already drawn their attention to this posting, which we have, we can point out that they know they are misrepresenting the situation. To date, we have only been able to speculate that they are misrepresenting the situation, since all of her recantations were in different venues.
I find Lewis & Khanna’s wellness gotcha games as amusing as anyone else does – still, god help me – but I’m starting to wonder where it’s all going.
Wow, wellness is not physics, and there are people who will enthusiastically over-promote the value of its grab-bag of concepts & tactics – who, without Al & Vic’s help, would have guessed? :eyeroll:
If you want to take on big-time smoke/mirrors purveyors, guys, at leastr train your sites on bigger targets – I hear Tony Robbins has dipped a toe into the wellness/health care pool…
In fact, you can follow my exploits in this wellness debacle here:
http://khannaonhealthblog.com/the-belly-of-the-beast/
Is anybody listening? I’d say so. My wife and I are embroiled in her employer’s wellness program for simple reason that not participating was too onerous financially. This large employer, with a wellness program run by a major league (well, as major league as a wellness vendor can get, anyway) vendor who is a litigant in at least one major case we know of, is a poke-prod-pry-punish program that would make Kate Baicker proud. Because, apparently, nothing makes her contrite.
In defense of my own political affiliations, I am not a Republican. In fact, I find the major leadership of the Republican Party as repugnant, stupid, and ineffective as most of their peers across the isle. I am an Independent conservative with a libertarian bent.
That clarified, Bob’s point about government approval being a marketplace propellant is right on. Nothing can advance the pointless like an endorsement from the clueless but powerful.
Steve, thanks for your thoughts. Someone is clearly listening to these researchers and other wellness ignorati (the “Wellness ignorati” are people whose business depends on making sure facts are ignored)– wellness programs are ubitquitous, discussed at the White House, and a major agenda item for the Business Roundtable, for reasons that elude me since presumably they do have an internet connection.
By the way, one other comment was tweeted but not posted. We were attacked for this “personal attack.” The irony is that the person who attacked us didn’t do it publicly, perhaps having learned the fate of attackers from ShapeUp (Dr. Kumar, how’d that work out for ya? http://theysaidwhat.net/2014/07/23/shapeup001/ )
However, unlike ShapeUp’s personality-driven self-indulgent rant, this other guy raised a good point that we would be happy to address: why the personal attack? First, it isn’t a personal attack–we have great respect for her primary-sourced work and called her the #1 researcher. I would love to be attacked like that. Second, all we are asking is for her to answer the question: does she support her own 3.27-to-1 or not? One could argue that the fate of the industry depends on this answer: the wellness ignorati have cited her 307 times (and that’s just in journals — try googling “Wellness” and “3.27”) so I think it’s a fair question. The ignorati have no other legitimate academic source–they sink or swim on 3.27-to-1.
Finally, I did write to her directly twice in the last few months and got no response, before going public. Likewise, before we posted columns showing that the Koop Committee completely lacks the ability or willingness to identify phony outcomes when they give out awards to their sponsors, we wrote to some Committee members ahead of time to point out their incompetence and ask them to address the mistakes before going public. Even Rachel Carson didn’t go public until her protestations got totally ignored by the chemical industry.
I don’t understand the comparison between blood pressure or smoking and wellness programs. Wellness is something being designed, it is a product, and it is going to change over time. In a market economy you’d hope it’s change for the better, improving since businesses will learn and adapt to earn and keep business. I know it’s optimistic but we teach this stuff to freshmen so I sort of believe it. Blood pressure and smoking on the other hand are not going to evolve to have a different impact on your health, not much anyway.
I think that is part of what Baicker means when she says “all the evidence is in.” Right now wellness programs probably do suck on average, but will they always suck? Does they ALL suck or are their positive deviants we can learn from?
That said, I get it that wellness backers aren’t promising results some day, they are grossly overselling the benefits of their product which is their job. Similarly researchers are grossly overselling their findings and that is their job. But does anyone actually listen to these people?
Thank you Al Lewis, Vik Khanna (and Nortin Hadler cited in the piece)…..all experts who stay close to what sound research methodology tells us. It is astounding how rare that is.
This is indeed an important post…..almost like a new Rand study.
Let me make two points:
1. Medicaid probably reduced debt a lot. It may also have extended life spans by earlier detection of some cancers.\
That is about all that most health insurance does
2. I had to chuckle about absenteeism.
If a firm wants to cut absentees, all they need to do is go to hourly pay and limit sick days.
Try to run a school district for a year.
The number of sick days is stunning, despite good health insurance.
Hey guys, thanks very much for taking the time to comment on New Years Day. A couple of things. First, just to be clear, the rarely cited papers were not specifically refuting hers but rather refuting workplace wellness generally. but your point is a good one regardless.
Second, as a Democrat (Vik is a libertarian/Republican so we don’t have a political agenda about wellness), I do want to make clear that there was wellness before Obamacare came along. the ACA did raise the stakes quite a bit, for God knows what reason. I think it was to mollify the Business Roundtable, which has not wavered in its support of the wellness provision no matter what the data says. Their position reminds me of what Stephen Colbert said about George W. Bush’s consistency: “He will believe the same thinkg on Wednesday he believed on Monday regardless of what happened on Tuesday.
Sticky research.
The point about the number of citations of the original paper versus papers that have subsequently refuted the findings is an interesting phenomenon in its own right.
Hi, Gents. This is an important post. While I do not know Prof Baicker and can’t comment regarding her personal actions or motives, I do (as you may know) share your exasperation with her flawed wellness study and the way it was swallowed — hook, line, and sinker — by the wellness industry. If anything, I think you’ve understated the case, as you don’t mention the phrase that was repeated in the study, “medical costs fall by about $3.27 for every dollar spent on wellness programs,” which is more absolute than the study title and, in my analysis, was wholly unsupported by the data presented in the study findings. I’m hoping you can elaborate on a matter that is more incidental to this particular post — namely, the straight line you draw from Obamacare to so-called outcomes-based wellness programs. Certainly, it is true that ACA “allow[s] employers to financially and clinically punish employees with coercive directives.” And this inclusion in ACA is inexcusable. But readers of this and other posts, who might not know better, may conclude that the Obama administration invented these programs. As you know, ACA expanded the permissible limits of these incentives/penalties — from 20% to 30% of the cost of coverage (in some cases more) — that were established in HIPAA in 2006, prior to Pres Obama’s election. Don’t get me wrong: By the very inclusion of these programs in ACA, and by expanding permissible limits, Pres Obama and ACA endorsed these programs, and presidential and gov’t endorsement is powerful. But readers would be served well to understand that outcomes-based programs were around long before ACA or the Obama presidency, and they were codified in government regulation in HIPAA (during the administration of Pres Bush). Ironically, in light of the recent EEOC lawsuits, the 2006 HIPAA codification of outcomes-based wellness was dubbed, “Wellness Exception to HIPAA Nondiscrimination Provisions.” The RAND Workplace Wellness Programs Study (p. xx) suggested that, at the time the study was conducted, “the average amount of results based incentives for these employers [who used so-called outcomes-based models] was less than 10 percent of the total cost of health coverage,” which could bring into question whether raising the permissible limit from 20% to 30% really had such a big impact. In my opinion, outcomes-based programs are nothing but an unfounded, desperado move on the part of employers. As such, isn’t it possible that they would have continued to see uptake from desperate employers, even in the absence of ACA? And is it correct to pin these programs on ACA, when ACA simply modified something that already was enacted and had already attracted the interest of employers? I look forward to your response. (For those interested in the HIPAA wellness provisions, which were enacted during Pres. Bush’s administration, and how they were expanded in ACA under Pres. Obama’s administration, I direct you to the Federal Register at https://www.federalregister.gov/articles/2013/06/03/2013-12916/incentives-for-nondiscriminatory-wellness-programs-in-group-health-plans#h-9 )
Yikes.