By Robert Wachter

Last week, Medicare added patient satisfaction data to its hospital reporting website. This is progress, but it raises an interesting question: should patient satisfaction scores be case-mix adjusted?

The
motivation to include patient satisfaction data comes from the
Institute of Medicine’s inclusion of “patient-centeredness” as one key component of quality.
And what could be simpler than asking patients a few questions, as the
Center for Medicare & Medicaid Services (CMS) survey does. (A pdf
of the survey, formally known as HCAHPS, or “H-CAPS”, for Hospital
Consumer Assessment of Healthcare Providers and Systems, is here).
I like the addition of the patient experience data and found the
presentation on the CMS site to be fairly reader-friendly (as did US News & World Report’s
Avery Comarow). For example, it only took a few seconds to find my
hospital’s performance on the summary question, “Would you definitely
recommend this hospital?”:

UCSF Medical Center: 80% yesAverage for Northern and Central California: 65% yesAverage for all U.S. Hospitals: 67% yes

[You’ll note that we didn’t do too badly. But it would be legitimate to
wonder whether I, being relatively fond of my job and unenthusiastic
about being shunned by my colleagues, would have shown you something
that made us look crummy. You should have the same skepticism when you
look at every hospital’s web site, a point Peter Pronovost, Marlene
Miller, and I made in this JAMA article.]

One can debate the relative value of considering patient experience vs.
harder measures of quality and safety forever. Personally, I want both:
great technical quality (which few patients will be able to judge) as
well as a clean room with nice people who listen and communicate well.
There’s no reason that the dexterous surgeon needs to be a jerk, nor
that the empathic internist needs to be a diagnostic imbecile.

But,
like most things in healthcare, patient satisfaction measurement and
reporting is trickier than it looks. Think about it in terms of the Donabedian triad
of quality measurement: structure, process, and outcomes. One of the
advantages of using processes (did the patient get a beta blocker?) and
structure (are there intensivists available?) is that outcome
measurement requires case-mix adjustment to avoid apples-to-oranges
comparisons. Just comparing raw 30-day post-CABG mortality rates, for
example, would clearly be misleading, since the superb surgeon who
operates on older diabetic vasculopaths might well have a
higher-than-average mortality rate, notwithstanding his excellence.
Just as importantly, if you don’t employ scientifically bullet-proof
case-mix adjustment, the Pavlovian response of every provider and
hospital whose outcomes are worse than average is… “But–but–but… You
don’t understand… My patients are sicker and older!”

Not everybody’s patients can be sicker and older. Except in some Bizarro-World version of Lake Wobegon. But believe me, that is what everybody will claim.

Anyway,
it is largely for this reason that most publicly reported quality
measures to date have not been of outcomes – the science of case-mix
adjustment has not been ready for prime time. But this science is
getting better, and the world is clearly moving toward outcome
measurement and reporting: CMS and several states now report case-mix-adjusted CABG mortality, and California is now reporting case-mix-adjusted ICU outcomes via its CalHospitalCompare project.

If you think about it, patient satisfaction is simply another outcome measure.

But
do satisfaction survey responses need to be adjusted? Well, yes. For
example, maternity patients tend to rate their experience more highly
than do medical and surgical patients (no surprise there). Well
educated people tend to be more critical, and older patients are more
forgiving.

Impressively, the H-CAPS folks thought of some of this, and the data you see on hospitalcompare have been adjusted
for the following variables: service line (medical, surgical, or
maternity care), age, education, self-reported health status, language
other than English spoken at home, emergency room (ER) admission, and
the time between discharge and survey completion.

But is this enough? Probably not. As reported in the New York Times,
states showed substantial variation in their average satisfaction
scores. For example, 79% of patients in Alabama hospitals “would
definitely recommend” their hospital to friends and family, while only
64% of folks in New Jersey, 61% in Florida, and 56% in Hawaii would do
the same. I’m guessing that these differences are more likely to be due
to the characteristics of Floridians (spend a day visiting my family in
Boca if you doubt this) or in Hawaii (“hey, dude, I’m missing some gnarly grinders”) than differences in the niceness of nurses and doctors.

Would
it be possible to capture the personal characteristics that would fuel
a robust satisfaction “case-mix adjustment” engine? I’d guess that
insurance status would be a predictor of satisfaction; income might be
as well. I’d also wager that Nordstrom shoppers are more demanding than
Target shoppers, and that people with young kids or busy jobs are less
tolerant of long waits than retirees.

The point is that we
don’t understand the interactions between these subtle sociocultural
and economic variables and the likelihood of ranking your hospital or
doctor more highly. For now, this isn’t a big deal – the data are just
being put out there and folks can draw their own conclusions. (Medicare
penalizes hospitals – about $100 per hospital admission – for not
reporting, but there is no change in payment based on performance on
these measures. Yet.) But if satisfaction is ultimately tied to
reimbursement, or if patients or insurers begin making decisions based
on satisfaction data, it will be important to either adjust for these
variables or at least understand and describe them.

Some day,
the presentation of patient satisfaction scores may be similar to that
of presidential polling results: “Independents, single mothers, and
Asian men over 55 really adored Hospital X.” Or perhaps it will be more
like Amazon.com: “Customers like you prefer Hospital Y.”

Robert Wachter is widely regarded as a leading figure in the modern
patient safety
movement. Together with Dr. Lee Goldman, he coined the
term "hospitalist" in an influential 1996 essay in The New England
Journal of Medicine. His most recent book, Understanding Patient
Safety, (McGraw-Hill, 2008) examines the factors that have contributed
to what is often described as "an epidemic" facing American hospitals.
His posts appear semi-regularly on THCB and on his own blog "Wachter’s World."