By Bob Wachter, MD

The human capacity to deny reality is one of our defining characteristics. Evolutionarily, it has often served us well, inspiring us to press onward against long odds. Without denial, the American settlers might have aborted their westward trek somewhere around Pittsburgh; Steve Jobs might thrown up his hands after the demise of the Lisa; and Martin Luther King’s famous speech might have been entitled, “I Have a Strategic Plan and a Draft Budget.”

Yet when danger or failure is just around the corner, denial can be dysfunctional (see Karl Rove on Fox News), even suicidal (see climate change and Superstorm Sandy).

Healthcare is no exception. Emerging evidence suggests that patients and their surrogates frequently engage in massive denial when it comes to prognosis near the end of life. While understandable – denial is often the way that people remove the “less” from “hopeless” – it can lead to terrible decisions, with bad consequences for both the individual patient and society.

First, there is evidence that individuals charged with making decisions for their loved ones (“surrogate decision-makers”) simply don’t believe that physicians can prognosticate accurately. In a 2009 study, UCSF’s Lucas Zier found that nearly two-thirds of surrogates gave little credence to their physicians’ predictions of futility. Driven by this skepticism, one-in-three would elect continued life-sustaining treatments even after the doctor offered their loved one a less than 1% chance of survival.

In a more recent study by Zier and colleagues, 80 surrogates of critically ill patients were given hypothetical prognostic statements regarding their loved ones. The statements ranged from “he will definitely survive” to “he will definitely not survive,” with 14 statements in between (including some that offered percentages, such as “he has a [10%, or a 50%, or a 90%] chance of survival”). After hearing these statements, surrogates were asked to interpret them and offer their own survival estimates.

When the prognosis was optimistic (“definitely survive” or “90%” survival odds), surrogates’ estimates were in sync with those of the physicians. But when the prognosis was pessimistic (“definitely not survive” or “he has a 5% chance of surviving”), surrogates’ interpretations took a sharp turn toward optimism. For example, surrogates believed that when the doctor offered a 5% survival chance, the patient’s true chance of living was at least three times that; some thought it was as high as 40%. Remarkably, when asked later to explain this discordance, many surrogates struggled. Said one, “I’m not coming up with good words to explain this [trend] because I was not aware I was doing this.” The authors identified two main themes to explain their findings: surrogates’ need to be optimistic in the face of serious illness (either as a coping mechanism for themselves or to buck up their loved one), and surrogates’ beliefs that their loved one possessed attributes unknown to the physician, attributes that would result in better-than-predicted survival (the “he’s a fighter” argument).

Religious beliefs also play a part. In Zier’s 2009 study, nearly half the surrogates harbored religious objections to the notion of futility. These individuals were far more likely than those with other objections to favor continued aggressive care in the face of a physician’s dire prognosis. Even more striking, in a 2008 study of more than 1000 adults by Jacobs and colleagues, 61% believed that a person in a persistent vegetative state could be saved by “a miracle,” and 57% believed that divine intervention could rescue a patient even after a physician had characterized the situation as hopeless.

A study in last month’s New England Journal of Medicine by Weeks and colleagues further illustrates the gulf between reality and perception when it comes to terminal care. The authors surveyed 1193 patients with metastatic lung or colorectal cancer who were receiving palliative chemotherapy. Approximately half of the lung cancer patients and two-thirds of the colorectal cancer patients believed that a cure was “very” or “somewhat” likely. (Sadly, the chance of a cure is essentially nil in such patients.) Unrealistic expectations were particularly common in nonwhite patients. Interestingly, patients who reported better physician communication were also more likely to have inaccurate expectations – perhaps because the doctors were telling the patients what they wanted to hear.

This literature illustrates that patients’ and surrogates’ religious beliefs color their perceptions, as do doubts regarding whether their physicians are accurate prognosticators. The latter concern is not entirely without merit. All experienced physicians can recall circumstances in which things did turn out better than advertised. I recently renewed my acquaintance with a former colleague who had a huge MI – which resulted in heart failure and a scary-low ejection fraction – while in his 40s. His physicians told him to retire from practice and enjoy his last few years on earth. He quit his UCSF faculty position and moved to Montana. That was 20 years ago; he’s doing just fine today. My dad had stomach cancer in his early 50s, with a positive lymph node found at the time of his gastrectomy. He was told to get his affairs in order, which he promptly did. He needn’t have rushed: he’s alive and reasonably well today, 30 years later. In both cases, the doctors weren’t wrong, per se; the odds were fairly terrible. But both men managed to beat them. These are the cases that we all remember – both patients and doctors need only to see a few like them to soften everyone’s sense of prognostic certitude.

But there is something subtler going on here: a type of collusion between doctors and their patients, neither of whom really wants to confront some painful truths. Weeks and colleagues cite a 2000 ethnographic study, in which the investigators observed both oncologists and patients rapidly pivoting from discussions of prognosis to matters of treatment options and logistics. “In other words,” Weeks wrote, “a focus on chemotherapy was the instrument that facilitated prognostic misunderstanding.”

You may recall Atul Gawande’s poignant New Yorker piece on end of life care from 2010. Gawande described a post-op visit with one of his patients, after he found her abdomen filled with widely metastatic colon cancer at surgery, a situation that offers no hope of cure.

I said that it had not been possible to remove all the disease. But I found myself almost immediately minimizing what I’d said. ‘We’ll bring in an oncologist,’ I hastened to add. ‘Chemotherapy can be very effective in these situations.’

She absorbed the news in silence, looking down at the blankets drawn over her mutinous body. Then she looked up at me. ‘Am I going to die?’

I flinched. ‘No, no,’ I said. ‘Of course not.’

Every physician I know, including me, can recall getting weak-kneed at moments like these. It is only natural.

While there are some valid reasons for undue optimism and misunderstanding, the impact can still be highly destructive. Patients and surrogates need accurate prognostic information in order to make informed choices about their treatment options, whether it is chemotherapy, intensive care, surgery, dialysis, or palliative care and hospice. As Zier writes, “not all optimism is ethically benign.”

Moreover, in an era in which the costs of healthcare have become a central concern for all Americans – one-in-four Medicare dollars are spent in the last year of life – we can no longer afford to swim in a sea of wildly inaccurate prognoses. Patients and family members who believe that chemotherapy for metastatic cancer can be curative will not only favor it when they shouldn’t, but they and their advocates will demand that it be covered by insurers, even if its meager benefits are not worth the costs and toxicity. And, while patients have every right to their own religious beliefs, should those whose beliefs are grounded in science be forced to contribute to the costs for patients who insist on aggressive care because they believe in miracles?

Fixing this is going to take a multifaceted approach. Both physicians and patients need access to better tools to help them prognosticate accurately. A wonderful example of such a tool is ePrognosis, developed by our VA-based geriatrics group. Clinicians also need better training in communicating prognosis with patients and surrogates, particularly in how to offer accurate information without robbing patients and families of (realistic) hopes.

As everybody who either read or watched Moneyball knows, baseball scouts who used to base their recruiting decisions on gestalt now rely on advanced statistics to make data-based prognostications. Similarly, decisions by patients and surrogates about treatments near the end of life should be based on empirical data and science, not on hope, theology, or magical thinking. And, in a healthcare world in which costs must be cut, policies – including insurance coverage for specific treatments – should be based on accurate information about prognoses and outcomes.

Can we have a calm, rational discussion about all of this, built on a foundation of science? In today’s America, I’m sad to say that that would be a miracle. But a person can always hope.

Robert Wachter, MD, professor of medicine at UCSF, is widely regarded as a leading figure in the patient safety and quality movements. He edits the federal government’s two leading safety websites, and the second edition of his book, “Understanding Patient Safety,” was recently published by McGraw-Hill. In addition, he coined the term “hospitalist” in an influential 1996 essay in The New England Journal of Medicine and is chair of the American Board of Internal Medicine.  His posts appear semi-regularly on THCB and on his own blog, Wachter’s World.