By
Reed Abelson and Gardiner Harris in the New York Times are questioning some of the key assumptions behind the Dartmouth Atlas of Health, which for twenty years has documented wide variations in Medicare utilization rates across the country and used that to claim huge savings could be obtained by rooting out waste in high-spending regions. In February, Harris reported on a commentary by Sloan-Kettering’s Peter Bach in the New England Journal of Medicine that argued the Dartmouth analysis failed to adjust for illness severity. I reported on the Medicare Payments Advisory Commission’s similar analysis here.
This time, the Times’ two most thoughtful health care reporters bring quality into the discussion. After describing a map in Office of Management and Budget director Peter Oszag’s office that divided the nation into low-spending beige regions and high-spending brown regions, they write:
For all anyone knows, patients could be dying in far greater numbers in hospitals in the beige regions than hospitals in the brown ones, and Dartmouth’s maps would not pick up that difference. As any shopper knows, cheaper does not always mean better. . . The debate about the Dartmouth work is important because a growing number of health policy researchers are finding that overhauling the nation’s health care system will be far harder and more painful than the Dartmouth work has long suggested. Cuts, if not made carefully, could cost lives.
For documentation, the reporters used quality data generated by the Wisconsin Collaborative on Healthcare Quality, which I wrote about a month ago for The Fiscal Times.
This is an important debate. But as is often the case in journalism, the attempt to reduce complex realities into a single-factor analysis that can be summarized in a headline or a single “why this story is important” paragraph can leave a mistaken impression. Regional variation in Medicare spending is one indicator of gross overutilization. Something is happening when a hospital in McAllen, Texas does twice as many knee implants per Medicare beneficiary as a hospital in Baton Rouge, Louisiana. (An earlier version of this post compared McAllen to Rochester, MN, which actually has a slightly higher rate of knee implants per 1,000 Medicare enrollees.)
But that by itself tells us nothing about why that overutilization occurs. Greedy doctors or hospitals working in a fee-for-service system are part of the problem. Cost shifting due to high levels of uninsurance and illness severity also may account for some or even much of those differences. And, as the Times report points out, quality differentials have to be taken into account.
The pushback by Dartmouth Atlas proponents has already begun. The Atlas website carries a five-page rebuttal of the Times story. Meanwhile, Columbia University professor Andrew Gelman on his blog suggests the reporters could have used some help from experts in statistics to help them focus their analysis. My complaint with today’s story comes from an entirely different angle. Higher quality care lowers costs, it doesn’t raise costs.
Take a few additional steps to keep operating rooms germ free and rates of hospital-acquired infections and their attendant higher costs plummet. Do a knee implant right the first time and you don’t have a patient back within a year for a revision. A careful mapping of quality has never been done by Medicare or anyone else since good data isn’t available. The Wisconsin Collaborative is one of the nation’s few efforts to create such a database. The reform law will generate much better data, but that is years away. But when it is done, comparing those maps to spending patterns may provide researchers with crucial clues for determining what accounts for variations in spending across the U.S.
One thing is for certain. Quality data will provide patients-as-consumers with information about what hospitals and physicians to avoid — something that spending patterns by themselves can never do.
(An earlier version of this blog post incorrectly called Professor Andrew Gelman of Columbia University a defender of the Dartmouth Atlas analysis. His blog’s critiquefocused on the methdology of the Times reporters. I regret the error.)
Merrill Goozner has been writing about economics and health care for many years. The former chief economics correspondent for the Chicago Tribune, Merrill has written for a long list of publications including the New York Times, The American Prospect and The Washington Post. His most recent book, “The $800 Million Dollar Pill – The Truth Behind the Cost of New Drugs ” (University of California Press, 2004) has won acclaim from critics for its treatment of the issues facing the health care system and the pharmaceutical industry in particular. You can read more pieces by Merrill at GoozNews, where this post first appeared.
Categories: Uncategorized
Nice
smart patient- you raised interesting point of detachment from personal and financial concerns.
My own observation is that doctors who have financial worries or dissatisfaction do not tend to be most effective.
I never saw a definition before. However, I suggest the utmost concern about the first component: “I want the doctor to treat me as though I am a member of his/her own family and he/she is paying the bill out of his/her own pocket.”
Uh oh, bad idea. Detachment from relations and financial concerns is the best policy.
I think that most doctors know how to practice cost effectively. The price freezes and inane demands by payors corrupt and generate potent inflationary forces.
Thus, considering that each US doctor controls about $3.5 million in spending by his/her pen or clicker, the best advice for the policy wonks is go out there and say to each specialist and (probably to) each primary care doctor, “we know you are gaming the system, we know you know how to practice cost effectively; here is $500,000 per year. Go out there and take gopod care of the patient. Don’t screw the system. Provide cost effective care. And we will protect you from all but the most eggregious malpractice. Each of you need to save $50,000 per year.”
With $800,000 doctors, that saves $40 billion per year.
Dr. Seker — I think care quality has several aspects to it. From a patient’s perspective, here are my thoughts.
1. The care should be necessary and appropriate. Tests and procedures should not be ordered just to generate revenue or to protect against a potential lawsuit. I want the doctor to treat me as though I am a member of his/her own family and he/she is paying the bill out of his/her own pocket.
2. In the case of surgical procedures and cancer treatments, I want the risks and benefits clearly explained to me. What is the range of potential outcomes? What’s the probability of a good or at least satisfactory outcome? What are the quality of life implications and likely side effects? In short, help me fully understand what I’m signing up for so I can make an informed choice.
3. As for costs, the price of the care should be reasonable, transparent, and made known to me before services are rendered, even if insurance is paying most of the bill. If you are referring me to a specialist, especially for something that just about everyone can do well and there are big differences in cost, send me to the doctor and/or hospital that can do a good job for the best price.
4. If I’m a hospital inpatient, I would like the staff, especially the nurses, to be competent and reasonably attentive. I would like medical personnel who come into contact with me to follow basic procedures, like hand washing, so the risk of infection is minimized.
5. With respect to primary care doctors, I would like to be able to get an appointment on a reasonably timely basis when I need one and I would prefer to have a good relationship with the doctor in a personal chemistry sense.
Unfortunately, like a lot of other things in life, I would characterize good quality medical care as something that I know when I see it or experience it but I probably can’t define it in a way that I can prove in court or reduce to contract language.
Maggie and Barry,
How do you define the quality medical care? Is there a definition for cost effectiveness?
Thank you for answering.
Barry–
As I think you know, the Dartmouth research adjusts for the variables you talk about, including differences in local prices.
David Cutler tells me that he explained this, very, very clearly, to the reporters. But they ignored him.
The Dartmouth reserach also looks at use of resources–how many specialsts a hospital patient sees, how many tests he undergoes, how many procedures, how many days he spends in the hospital, what % of patients are readmitted– quite apart from how much all of this costs.
When you look at use of resource, you don’t have to adjust for differences in local prices. You do have to adjust for the severity of the illness–and Dartmouth does that.
Finally, yes, over-treatment can be a function of the medical culture (see Atul Gawande’s New Yorker article))
But we also know that high-treatment regions tend to be regions where are more hospital beds. IF they are there, doctors use them–even in cases when the patient didn’t necessarily need to be hospitalized. It’s simply more convenient. The doctor can easily call in other docs to consult. Different docs order different tests. Tests lead to procedures. One thing leads to another . .
We also know that in areas where there are more doctor-owned hospitals and surgical centers, people undergo more surgeries and procedures. In the past, I had often wondered why Louisiana was a high-treatment area . . Then I found out that it ranks #2 in the nation for the number of doctor-owned facilities in the state. Texas ranks #1–and much of Texas also shows up as high-treatment on Dartmouth maps . . .
I agree about shared decision-making. But some doctors and some hospitals resist the idea.
nice article, thanks
Barry Carol, why do you say this. Doctors are not executioners. Are you??
“This mentality needs to change. Shared decision making, with help from palliative care teams, can play an important role in helping patients to make an informed choice that is consistent with his or her wishes and value system.”
Hospitals’ use palliative care teams to bury their mistakes facilitated by CPOE and other ill designed HIT equipment.
I don’t think there is any question that medical spending per person varies significantly by region. There are lots of reasons depending on the type of care and different approaches are called for, in my opinion.
First, prices vary a lot even within the same city. Since Medicare dictates the prices it pays, this is more of an issue for commercial insurers. I would like to see disclosure of contract reimbursement rates and tiered insurance products that require higher co-pays if patients want to use more expensive hospitals for procedures that virtually all providers can do well. There is no question that the brand name medical centers and hospital systems that have amassed considerable regional or local market power are able to extract the highest reimbursement rates from insurers even though their quality is often no better and sometimes worse than their competitors.
The use of diagnostic imaging can vary both regionally and locally depending on a combination of money driven factors like whether the provider also owns the equipment, patient expectations, and defensive medicine. True tort reform, which no state has implemented yet, could address some of this including making it easier and safer for doctors to push back when patients ask for expensive imaging when it’s probably not medically necessary or appropriate. A capitated payment model for primary care could also be helpful here.
The number of surgeries performed per person including hip and knee replacement, CABG, angioplasties, back surgery, etc. probably varies a lot by region. Differences in medical culture and how and where doctors were trained is probably an issue here. I think shared decision making needs to be more universally applied in these cases.
Infections and readmission rates also probably vary quite a lot among hospitals both regionally and locally. This is an important issue for the frail elderly and very sick including those with congestive heart failure. The sunshine of transparency can help referring doctors to avoid the hospitals with the worst records on these metrics.
For cancer care, I think culture is an important issue. The big name cancer centers like Memorial Sloan Kettering and M. D. Anderson have cultures that call for aggressive treatment. Oncologists often view it as a personal failure when a patient dies. This mentality needs to change. Shared decision making, with help from palliative care teams, can play an important role in helping patients to make an informed choice that is consistent with his or her wishes and value system.
I also wonder about regional differences in aggressive surgical interventions for patients with advanced Alzheimer’s or dementia. This is probably another area where culture plays a role. Having a living will, advanced directive and ensuring that a family member, relative or friend who knows the patient well and is empowered to make medical decisions on the patient’s behalf would all be helpful.
So, spending varies considerably across regions and even within the same city for lots of different reasons. A multi-factorial approach is necessary to address these issues. As I’ve said numerous times before, there is no silver bullet but lots of silver pebbles.
I tried finding physician density map but couldn’t. So I looked at population density map, to see if there is any correlation with Medicare pay rates. Generally high density places also have higher pay rates though Western TX is big exception to that.
Another theory I have is that physicians being better paid tend to gravitate to areas that are more expensive for ordinary folks such as SoCal. Though once at that place they find a lot of competition.
How do doctors decide to where to practice? How do they decide to move? What is their strategy to maximize revenue and lead good lifestyle? How do they cope with losses? Where do they invest?
Beyond that its also about patients. Which state has sheep like patients who will be ready to get surgery for pneumonia, where are the aggressive ‘I don’t want to be your guinea pig’ etc.
Dartmouth study is so much like typical medicine- treat symptoms and not the cause. Throw in the anti-biotics and all bacterias will be killed, good and bad.
We really need extensive physician and consumer profiling because it’s their interplay that determines cost.
Dartmouth validates the ages old adage of human behavior: Monkey See, Monkey Do! Regions have doctors that trained in the regions, having learned from the senior physicians and faculty. As the wage and price freeze gripped medical care in an ill conceived effort to control inflation, Monkey Saw the Monkey Do the procedures in high volume to thwart the price freeze.
That is all, no more no less. To think that you can explain the complex inner workings of medical care systems on Dartmouth’s simplistic research model is folly. Maggie should know this, but chooses to ignore it.
Quality is not measured any place, really. What is quality medical care, anyway? Early treatment of pneumonia with antibiotics when the patient has heart failure, actually? Is that qualtiy?
The Cleveland Clinic puts on a good sales job and has excess costs, perhaps hidden by smoke and mirrors, ecaggerated severity of illness, and early discharges to SNFs and LTACS.
To base a national health care policy on Dartmouth’s Atlas is a concern the chord struck by the NY Times.
Anyone who has seriously published in NYT knows, they triple-check something this big.
This is just the tip of the iceberg.
Anyone who has seriously published in NYT knows, they triple-check something this big.
This is just the tip of the iceberg.
“And hosptials spending less–like Mayo or the Cleveland Clinic– often have clearly better outcomes measured in many different ways.”
Great example of why people should be questioing Dartmouth, why is Cleveland Clinic held up as a great example of cost efficient care? They are the highest priced system in the area. Actually look at some CC bills then tell me the rest of the county should be modeling them. Overall they provide some incredible miracles but they have issues with infections, overcharge like you woulodn’t beleive, and don’t deliver any better care then UH. I know this because I see the bills. If you are worried about spending you are not sending people to CC.
Obviouslly Dartmouth needs questions as the assumptions people like Maggie are drawing from it are 100% wrong.
go to any website like http://reflectivemedical.com/Home.aspx
and look at CC prices compared to anyone else.
There is tremendous opportunity in our field to improve upon how we measure and compare quality and outcomes.
The only issue I have with the Dartmouth work is that it can’t use quality and outcomes data sets which don’t exist as of this time.
I looked ad Dartmouth graphs and couldn’t help observing that most higher cost areas are one with tort reforms in place.
Really speaking you have to look at physician concentration in a location. Due to favorable tort laws or other reasons some places will have higher physician concentration.
Since they have to earn incomes they expect to earn, more procedures would happen. Probably another map highlighting physician density would have been apt and be placed next to Dartmouth map.
I am just exercising my imagination here, but what happens if conslusions from Dartmouth are implemented.
Efficiency measures would typically mean less procedures being approved in certain areas.
So if you are in mid-west you have most procedures available and in South patients suffer for collective profligacy. Maybe then there would be incentive for senior citizens to move to mid-west. Physicians left in South would then execute more of approved procedures, even though they aren’t relevant in all cases.
To me Dartmouth ranking is a useful scarecrow so that hospitals know that they are being watched. But any other exercise without involvement of consumers would be counterproductive. They have to be bought onboard to fact that excess spending on health or overmedicating is not always useful. Look what happened with mammogramy. Politics aside, it is general belief that more is better. I have heard only health economist argue against it.
Policy makers need to be able to translate that and into actionable items. Cutting funding and the leaving providers to determine how they want to keep lights on and maintain quality stats is not what will boost efficiency.
Merrill–
The Times reporters quote two sources that they claim are critical of the Dartmouth reserach: Harvard’s David Cutler and Halan Krumholz, a professor of medicine at Yale.
I spoke to both of them yesterday (by phone and e-mail)
Both said that the article seriously distorted what they actually said.
See http://www.healthbeatblog.com/2010/06/the-new-york-times-attacks-the-dartmouth-research-part-1.html#comments
Yesterday, I also spoke directly to a very reliable source in Washington who was interviewed by the Times’ reporters. He/she told me that “they were determined to write a story that would ‘take down Dartmouth.'”
The Times story contains many falsehoods.
For instance: “Neither patients’ health nor differences in prices are fully considered by the Dartmouth Atlas.”
As you know, the Dartmouth research does adjust for differences in local prices. Yesterday David Cutler told me : “I told Harris and Abelson that Dartmouth does adjust for differencdes in local prices.”
For more on adjusting for differences in local prices,, see this 2010 article in Health Affairs. http://content.healthaffairs.org/cgi/content/abstract/hlthaff.2009.0609
In our conversation, Cutler added: “The Dartmouth research is clearly helpful to health care refom – as part of a panolply of evidence that we can clearly save that kind of money (up to 1/3 of what we now spend on care). It was one of the first really good pieces of eivdnece—now there is a lot of evidence that we could achieve enormous savings.”
As for adjusting for patients’ health, of course,
they do. Back when I was writing my book in 2005, they were adjusting for “the underlying health of the population.”
Over at TNR, Jon Cohn makes the point, quoteing Dartmouth’s Jonathan Skinner from blog entry Skinner wrote for the Times:
“…some regions of the country experience more illness than others, and of course sick people spend more on health care. To deal with this bias, the Dartmouth group has compared expenditures and frequency of treatment across regions for people with similar diseases. The most extensive study compared spending across regions using a variety of cohorts such as people who had suffered a hip fracture or heart-attack patients. This study examined people who were equally sick, whether they lived in Louisiana or Colorado. The researchers further adjusted for any differences in patient income, race, and prior health. They still found gaps of up to 60 percent in spending among regions.”
There are, of courses, always ambiguities and questions
about data adjustment. But as Yale’s Krumholz told me yesterday: ““What I spent most of the interview trying to convey is that a lot of the back and forth [about bits and pieces of Dartmouth’s data ] is inside baseball stuff – and we are all working hard to figure out how to gauge costs and value better . But Dartmouth’s work on variation is pivotal to moving us forward – and we all agree that there is lots of waste and it is unevenly distributed across the country.”
By “inside baseball” Krumholz means pretty esoteric debates about risk adjustment within the academic community–nothing that changes the basic thrust of the DArtmouth reserach, but rather, ambiguities around the edges.
The Times’ reporters quoted none of what he spent “most of the interview trying to explain.” Instead, they plucked one line from a long interview: Krumholz saying, that “It may be that some places that are spending more are actually getting better results.”
This made it sound as if Krumholz doesn’t believe the DArtmouth reserach: not true.
Teh Dartmouth reserachers themselves have said that it could be that in particular cases a hospital spending more may be getting better results with some patients, but on average, when you look at hundreds of thousdands of cases (as they have) spending more does not lead to better outcomes. Often, outcomes are worse. And hosptials spending less–like Mayo or the Cleveland Clinic– often have clearly better outcomes measured in many different ways..
As to the notion that we can’t measure quality– I really don’t know what to say.
Do we have perfect measures of quality? No. Can we measure the quality of the care that individual doctors provide? The Dartmouth reaserachers don’t even try to do that: an indivdual doctor’s pool of patients is too small, and too easily skewed by a few very sick or non-compliant patients.
Can we measure the qualtiy of care at a hosptial that sees thousands of patients over a period of time?
Yes. Dartmouth has done exactly that in many different ways and using different measures such as patient satisfaction, avoidable-readmission rates or health outcomes
They also have measured quality by looking at the percentage of patients who receive recommended care for specific diseases. In my post, I’ve printed a Dartmouth map which shows what percentage of diabetics are receiving recommended evidence-based treatment in different parts of the country.
In addition have measured qualtiy by looking at outcomes for patients who are still alive and suffering from the same disease ( heart attack and colorectal cancer are two of the 5 diseases they looked at.) In these 2003 studies, they looked at patient satisfaction, as well as other measures of quality of care. A much-cited 2004 article showed a strong and clear negative correlation between spending and quality. (These are the studies that Dr. Donald Berwick, founder of the Institute for Health Care IMprovent and Obama’s nominee to head up the Centers for Medicare and Medicaid has called the most important health care reserach of the century. Berwick knows something about measuring quality . . . )
Finally, this is not the first piece that Harris has written about the Dartmouth reserach which garbles the facts and misrepresnts quotes. I wrote about his February piece here: http://www.healthbeatblog.com/2010/02/the-new-york-times-garbles-the-fact-about-the-dartmouth-research–.html
Let me add that I’ve never met or spoken to Harris and I’ve read other stories that he wrote on other subjects (vaccines for intance) that were fine. I’ve also never met Abelson, but have read many fine stories that she has written in the past. I’m pretty sure I quote at least one of her articles in my book.
Finally, Merrill , I agree with you: higher quality and lower costs go hand in hand. And measuring quality is complicated. But when you do it 5 different ways, looking at large pools of people, and keep getting the same answer (less expensive care can be better or at least as good; more expensive care can be worse) it’s seems pretty clear that you’re on to something. And the same regions of the country keep popping up as more expensive but no better care. (This is not to say that all hospitals and doctors in those regions are over-treating. This is why No One associated with teh Darmtouth reserach suggests cutting funding for an entire region. But they do think that we can use data on efficient hospitals that provide better care for less to set benchmarks for other hosptials.)
So I’m confused by what seems to be your defense of a pretty biased article..
My main criticism of Dartmouth work and especially some of the use it has been put to is its mostly ignoring (and then being used to deny by others) the real problem of the UNDERserved. With its focus on inpatient care and those who have already managed to get their way into care, it misses (and is then used to deny) the issues the lack of acces to primary care, the existance of medically underserved areas and populations, health professional shortage areas. Hence we get the emphasis on the “overtreated” as a convenient way to cut costs (while helping the insurance companies maintain their profits) while ignoring health disparities.
If you are not familiar with shortage areas and related issues, see:
http://bhpr.hrsa.gov/shortage/
http://datawarehouse.hrsa.gov/GeoAdvisor/ShortageDesignationAdvisor.aspx
http://bhpr.hrsa.gov/healthworkforce/default.htm
For more on the progressive critique of Dartmouth see:
http://pnhp.org/search/google/dartmouth?query=dartmouth&cx=015249405663905105964%3Aebn8t4lcngk&cof=FORID%3A11&sitesearch=#913
Nevertheless the NY Times article really sucked: it appears to have been a preconceived hatchet job, got the basic facts of it critique wrong, and misquoted folks.
“Forcing the rest of the country to adopt the medical practices of the most efficient areas will yield very little savings (that’s sort of like telling Kmart “just do it like Walmart does it”). After adjusting for the different characteristics of the states (age, income, ethnicity, etc.), the potential savings drops to only about 5 percent, which assumes orders are carried without mistakes or bureaucratic ineptitude.”
That simply makes no sense at all when one looks at the staggering degree of variations and the fact that most of these costs are calculated on a per-recipient basis (like compared to like).
There are a variety of reasons why regional variations exist. Some of these have to do with income or health status. In addition, differences in the cost of medical care by region explain some of the variation. But, in health care we have a convoluted system where third-parties pay 88% of all medical bills (it’s closer to 97% of hospital care). Where excellence exists, it’s distributed randomly – often the result of a few dedicated individuals. We tolerate variation in prices, spending and efficiency in health care that we would not tolerate in virtually every other industry – mostly because patients pay only a small portion of their medical bills so providers do not compete on price or quality. Because most providers do not compete (in the typical way that economists define competition) they have little reason to emulate other firms’ success in the health care industry.
Forcing the rest of the country to adopt the medical practices of the most efficient areas will yield very little savings (that’s sort of like telling Kmart “just do it like Walmart does it”). After adjusting for the different characteristics of the states (age, income, ethnicity, etc.), the potential savings drops to only about 5 percent, which assumes orders are carried without mistakes or bureaucratic ineptitude.
Realistically speaking, there are no savings to be expected using this approach, according to research was published by the National Center for Policy Analysis. Here is a reader-friendly blog post with links to their work. http://www.john-goodman-blog.com/can-an-independent-medicare-commission-control-health-care-costs/
The Dartmouth Data has been promoted as something it is not. The IOM report has been promoted as something it is not. HIT has been promoted as something it is not. C$HIT Certification has been promoted as something it is not.
All used flawed reasoning to influence the flow of dollars in to their coffers. Berwick makes millions $ on safety from the IOM report he wrote. Dartmouth, C$HIT, and HIT vendors do the same. This is also known as fraud.
Abelson and Harris are correct in their assertions. It is about time the self serving deception, and probable fraud, is exposed.
This is a red herring, of course. The problem with medicare-like the rest of the health care industry- is that decisions are made based on universal design and from a top down perspective- conspicuously absent from the debate- are the most important stakeholders-the public. With byzantine logic, so called experts in medicare-who dont know the basics of modeling decide without the advice and consent of the public what health services they have access to. And when the public wishes to make demands on the health care system the iron triangle of government, health care providers and researchers first make claim to benevelance and then laissez faire. For citizens the best we can hope for is for fate to give us health so we can avoid the madness and feudal mischief from so- called experts.