Researchers in John Wennberg’s shop at Dartmouth have come out with another stunning analysis of practice variation published in the new England Journal of Medicine (NEJM). This time it’s not region that makes a difference, but race of the patient. Essentially the article says that you are much less likely to receive a common type of knee surgery if you are black or hispanic than if you are white.

We’ve know for a long time that outcomes and health status are impacted by race and socio-economic status (SES).  For instance black males in Harlem have much lower life expectancy than average.  Similarly, despite the 50 years of Universal Health Insurance and care from the National Health Service in the UK, SES or "class" level there has a marked impact on health status and outcomes. In fact variation in employment status within the same SES, also has a large impact (and not surprisingly it’s better to be at the top than the bottom), as shown in the classic British Whitehall Study. And of course we also know that access to care for those without insurance is worse in the US than for those with insurance.

However, I believe that this is the first example in the US showing that access to a specific type of care for those in the same insurance category is very different.  I don’t know why black seniors in the US have knee surgery at half the rate of white seniors.  I suspect both patient demand, and physician culture have something to do with it.  I also don’t know which rate of knee surgery is better.–especially as last year the NEJM published a study that said that in the case of osteoarthritis, knee arthoscopy was no better than placebo. But it is clear that race and presumably other social factors influence the treatment that is given to patients with similar conditions for no good reason.

(Alerted to this by California Healthline)

Just when I was wondering what to write today, Matt Quinn comes to the rescue again……Linda Aitken, who’s study on nursing education I wrote about yesterday, also wrote an article last year about the impact of staff ratios. This is something Matt has looked at in depth, he writes:

Linda Aitken and (others) had an article in JAMA in Oct of last year ("Hospital Nurse Staffing and Patient Mortality, Nurse Burnout, and Job Dissatisfaction" in Oct 23/30, 2002) regarding the link between nurse to patient ratios and (suprise!) patient outcomes and whether nurses like their jobs. The long and short of it is that for every additional (surgery) patient that a nurse is in charge of, that patient’s chance of dying within 30 days of admission increases 7%.  Each patient also added 23% to the level of nurse burnout…  With the average age of US nurses north of 45 years and with the Philippines and other countries already experiencing shortages of experienced nurses (from flight to the US), the government must step in to provide strong incentives for both men and women to become nurses.  And nurses must gain (ongoing) leadership and management training.

A conversation that I had with a nurse yesterday helped me understand the difference between BS nurses and those with 2 and 3 year (vocational degrees). (Editor’s note: As you’ll notice, Matt was in the army but don’t hold it against him!) The BS nurse is like a new army 2LT: brimming with enthusiasm, long on theory/book knowledge and short on hands-on experience or credibility.  The vocational nurse is like a PFC: knows how to do tasks, but needs to be directed (a "worker bee").  The hospitals of this nation are short on the link between the two – the "sergeants" of the Army: nurses who have both years of hands-on experience and formal training in leadership / management skills.  Just as the Army is able to conduct complex operations with a few educated officers, a bunch of (perhaps) high-school-educated 18-24 year olds and a strong "backbone" of experienced sergeants to ensure the hands-on execution of things and the training of the new folks, hospitals need to develop this cadre of "nurse sergeants" and the professional "nurse officers" to lead them.  While I agree that a group of higher educated folks can probably outperform a group of less educated folks in a task like managing a group of patients, developing tightly knit teams ("squads"?) composed of nurse officers, sergeants, and PFCs could outperform both – at a lower cost and in less time than all BS nurses.

Matt has written two articles about staffing effectiveness based on his work at GE & Quantros. They are, The True Cost of Overtime and Balancing Staffing and Effectiveness . You can reach Matt at mtquinn@3fire.com.

Linda Aitken has been the leading academic looking at nursing professionals in the US for several years. (Ed O’Neill has been her opposite number on the physician side).  Aitken’s latest research is going to put the cat amongst the pigeons and maybe have a major impact. Both the abstract and some more detailed press reports (like this one that interviews Aitken) suggest that hospitals that have a greater proportion of better educated nurses have better mortality rates.  And the differences are significant both statisitically and in real life; up to 5% improvement in 30 day mortality for a 10% increase in the number of nurses with bachelor degrees.  All other features of the hospitals were corrected for, so the only difference was whether the nurse had a 2 year degree or a 4+ year degree. While representatives of 2 year nursing courses criticized the methodology, Aitken is no dummy and JAMA is no throwaway mag, so it carries the burden of proof.

So if a shift from a ratio of 50-50 to 55-45 in nursing mix (based on education) leads to a 5% decrease in mortality, how long before the lawyers/quality advocates start coming after hospitals that employ a large proportion of nurses holding only Associates degrees? You know that if a drug came out that improved mortality in the inpatient setting by 5% over a competitor, it would be adopted like a shot. The proof is in the uptake of TPA over Streptokinase 10 years ago when TPA decreased mortality only 0.2% better (see this post).  And if such a  move is made, who will end up paying for it given that we have a nursing shortage already?

The National committee on Quality Assurance (NCQA) has  new report out that funnily enough contrasts strongly with the report from the Harvard academic physicians that I was somewhat cynical about in this post yesterday.  NCQA’s mission is to improve health care via greater accountability and information. Although it’s directors and staff tend to come from the payer rather than the provider side, and its money comes from Foundations and the pharma industry, it has worked hard to maintain academic independence. In my view they have been telling the truth about health care quality all along — really all they have to do is point out the obvious. 

Anyway, enough editorial, the NCQA’s  State of health Care Quality report details what many of us have know for many years.  In order to treat heart disease, diabetes, asthma, etc, etc, etc, as John Mattison from Kaiser told me many years ago "We know what to do, we just don’t know how to make sure it gets done."  The consequence of practice variation away from best standards of care, according to the Peggy O’Kane, NCQA’s President, is "More than 57,000 people will die this year because there is a huge gap between what we know and what we do."  There’s also some 41 million sick days and billions in wasted expenditure ($1.6 billion for heart disease alone). None of this is news, John Wennberg’s Dartmouth Health Atlas has been detailing the extent of practice variation for decades.

The one area where NCQA says there has been improvement is among those health plans where they are actually measuring the impact of treatment protocols. (The report cards that NCQA promotes are part of that effort). Of course, none of this much matters if providers are not being rewarded for improving care quality.  In fact since the "end of managed care" (see my earlier post), the quality improvement movement has been struggling, even though some plans are now paying bonuses based on quality.  In the end the biggest payers (i.e. Medicare, Medicaid, FEHBP) and the government (i.e. Medicare, Medicaid, FEHBP)  must come together to promote compensation for quality if we’re ever going to make progress.  And as shown in the recent JAMA article, the provider industry has plenty of fire power with which to resist.

So the quality movement has been making slow strides and the first vestiges of a pay for performance system has appeared in California and has been going for a couple of years in Massachusetts. But not so fast! You may (as I did) have missed, while you were recovering from your labor day exertions, the September 3 JAMA article from several leading Boston doctors which explained that pay for quality and performance won’t work.  (You can see the abstract here). I quote a chunk of their press release below so you get the idea:

"Measuring a physician’s quality of care by numerical standards — such as adherence to a disease management protocol or a treatment outcome — is often invalid for a variety of reasons, say the authors of a study in the Sept. 3 Journal of the American Medical Association.

While not a general nationwide practice, several payers around the country are using quantitative quality measures as a basis for reimbursement bonuses (for Blue Cross and Blue Shield of Massachusetts’ program, see MD Practice Alert, July 30, 2003). Some medical groups also reward higher quality with higher pay. "Quality" for such incentive payments usually means adherence to well-recognized disease management or preventive care protocols or procedures. Quality measured in this way is beginning to be available on some Web-based "physician report cards" that increasingly may be the way some patients, such as those on consumer-directed health care plans, choose doctors.

Bruce Landon, M.D., researcher at the Harvard Medical School Department of Health Care Policy, and lead author of the JAMA study, says that although it looked at the use of such quantitative measurement (also called "physician clinical performance assessment" or PCPA) for credentialing doctors, many of the cautions raised in the study "are relevant for ‘paying for quality.’"

While PCPA can be valuable and is improving, Landon and his co-authors say, it has several common problems, some of which are:

–Insufficient sample size in an individual doctor’s practice. The authors suggest that 100 patients may be an appropriate sample (patients with the same disease treated by the same physician), but note that the National Committee for Quality Assurance says a 35-patient sample is adequate. "The proportion of all physicians for whom sample sizes are large enough to permit valid PCPA is unknown at this time," Landon writes.

–Systematic differences in populations of patients, who may differ in adequacy of insurance, general health status and other ways. "Health plans typically don’t adjust for health status or sociodemographic characteristics," Landon notes, although their reimbursement bonuses deal with patients who have the same insurance. To solve the problem of differing health statuses, some PCPA measures may include only "ideal candidates," he adds, but that approach could create sample-size problems.

–Poor reflection of entire practice. Obviously, adherence to one or two protocols is only a small part of what any given doctor does. Studies have shown that adherence to one protocol is a poor predictor of adherence to another not used to evaluate physicians.
Cost. "Collection [of PCPA data] in the outpatient setting would be substantially more expensive [than collecting valid hospital quality data] because of the multiple different locations and lack of funding mechanism to pay for this type of performance assessment activity," Landon says.

–Potential conflicts with quality improvement. PCPA activities may differ depending on whether they’re conducted to assess physicians’ competence or to foster quality improvement. Conflicts with patient communication and other unmeasured aspects of care also could arise, he adds. Groups focusing on a given kind of quality improvement "might pay less attention to other important features of quality that are not being measured."

–Lack of evidence-based measures for many specialties.

–Challenges in defining minimum thresholds for acceptable care.

"Many health plans," Landon says, "use arbitrary thresholds (e.g., the top 25%), when in fact there might not be much difference [in performance] between those that receive the bonus and those that don’t." Lack of uniformity among payer bonuses also is a problem, he says. "There are often so many measures from different plans that the signal to increase quality can get lost in all the noise."

The last two sentences of the abstract indicate that they are not happy with the ways they are being assessed.  "We conclude that important technical barriers stand in the way of using physician clinical performance assessment for evaluating the competency of individual physicians. Overcoming these barriers will require considerable additional research and development." And their last sentence is a thing of beauty. "Even then, for some uses, physician clinical performance assessment at the individual physician level may be technically impossible to accomplish in a valid and fair way."

Matt Quinn, who’s been working in health quality data assessment for some years now, and who’s vigilance saved me from missing this work of art, commented. "I guess that means that efforts to measure performance and inform consumers just aren’t worth it and that everyone involved should just continue to assume that all docs provide consistently excellent quality care that adheres to evidence-based guidelines." I’m sure Matt would agree that the correct performance assessment of no other human process has ever had to overcome this magnitude of challenge!

I’m reminded of Gene Wilder as the sheep-loving struck-off MD in Woody Allen’s film Everything you ever wanted to know about sex but were too afraid to ask. He’s working as a waiter and when too many customers start complaining and it all gets too much, he shouts "Don’t treat me like that–I’m a Doctor! I’m a Doctor!"

I remember mentioning in my post on pay for performance that some HMOs were paying out to medical groups based on their making certain quality indicators.  Well, I may have understated that phenomenon.  Blue Cross of California paid out over $28 million to their "favorite" medical groups last year — Hill Physicians in the SF East Bay topping the list –in a physician quality bonus scheme that looks much like pay- for-performance.

How did they score it?  Well, "the quality measures included such items as patient satisfaction surveys, waiting times for appointments, number of complaints and grievances, peer and staff reviews and patient turnover. Under the revised Quality Scorecard, more than half of a medical group’s score is now based on clinical outcomes and patient satisfaction surveys.".  So the IFTF line in 1997 which said that performance based pay would be aimed at improving "quality, customer satisfaction, patient tenure in the plan, and outcomes, as well as productivity and cost-effectiveness" wasn’t a bad forecast. 

$28 million isn’t absolutely chicken feed, but it is split between 80 medical groups.  Still, if Hill Physicians got say $1 million extra for doing well, that might encourage the others. And if that really changes the culture perhaps Blue Cross will go further towards  pay-for-performance, and advertise that fact to its members? (Something that no one ever did with capitation!)

And on the "lighter" side,  this news from Brazil shows that mistakes in health care can be made anywhere.  but if you were this patient, wouldn’t you notice?

I’ve mentioned the quality issue a couple of times, and have somewhat denigrated the quality movement as being made irrelevant by the backlash against managed care.  However, given the IOM’s "To Err is Human" report on medical errors and more recent "Crossing the Quality Chasm" (note: scroll down for exec sum), it would be unfair to ignore those who are trying to raise its public visibility, such as the CHCF promoting a series on it.  This video about the "Hospitalization from Hell" mostly by Paul Cleary (from Harvard — Here’s the article), is long (50 mins) but if you have a good connection and Realplayer, it’s well worth watching. Clearly suggests that the application of known quality processes used in most industries are still very limited in health care, and that the case he uses as an example is very applicable to most  health care organizations. One of his solutions is that local units and individuals must take responsibility for organizing their own quality process improvement, collect good and useful data, and engage the operations research specialists. This is not unlike the strategy that the German Army used to overrun the French in 1945 — command was decentralized to small units working with knowledge of wider goals (as argued in James Q. Wilson’s book "Bureaucracy"). There are also serious efforts funded by several Foundations to improve quality processes, such as this one in Washington state, funded by RWJ and "blogged" by many of its participants including Marcus Pierson, MD.

The problem here is that I heard all this in 1989, and Don Berwick, the guru of TQM in hospitals wrote the same stuff then as he is writing now.  Every other industry has been forced into using quality improvement techniques.  So this morning I heard about 2 examples of patients I know in the SF Bay Area who were served very badly.  One was not called back as promised about a potentially very serious test result. After calling the medical group, no one could locate the result or the file.  And then they couldn’t do anything other than give the patient another appointment. The other concerns a new patient who was left on hold for 25 minutes by the medical group. Let’s face it, none of us would put up with this from any other type of consumer service organization.  Why do we expect it from health care orgs, and is it our fault for doing so?

All you need to know about health care quality is encapsulated in Ian Morrison’s line–"you can tell a quality doctor when the people in the waiting room make more money than you do".  The latest Harris Poll on the subject shows that "Reputation", and the reccomendations of families and friends are still what drives doctor choice, while for hospital choice, whether the patient’s insurance plan covers it is added into the mix. (Although virtually all plans cover all hospitals unless Sutter and Blue Cross are fighting again!).  Only 20% said that they would choose a doctor because the doctor had been highly rated in a published evalutation of doctors. (Harris’s site is down at the moment, but their healthcare section should be here and is well worth cruising through). So well over a decade into NCQA and all that, we’re a long, long way from Consumer Reports