The unfortunate reality is that there is no body of expertise with responsibility for addressing the science of performance measurement. The National Quality Forum (NQF) comes closest, and while it addresses some scientific issues when deciding whether to endorse a proposed measure, NQF is not mandated to explore broader issues to advance the science of measure development, nor does it have the financial support or structure to do so.
An infrastructure is needed to gain national consensus on: what to measure, how to define the measures, how to collect the data and survey for events, what is the accuracy of EHRs as a source of performance, the cost-effectiveness of various measures, how to reduce the costs of data collection, how to define thresholds for measures regarding their accuracy, and how to prioritize the measures collected (informed by the relative value of the information collected and the costs of data collection).
Despite this broad research agenda, there is little research funding to advance the basic science of performance measurement. Given the anticipated broad use of measures throughout the health system, funding can be a public/private partnership modeled after the Patient-Centered Outcomes Research Institute or a federally-funded initiative, perhaps centered at AHRQ. Given budgetary constraints, finding the funding to support the science of measurement will be a challenge. Yet, the costs of misapplication of measures and incorrect judgments about performance are substantial.
Moreover, the science of performance measurement and improvement needs an academic home. While many medical and health policy societies and associations have sections on quality or quality measurement, no professional society primarily focuses on the science of quality measurement and improvement. Such an entity could set standards for and advance the science of quality measurement, thereby moving the policy discussion from whether measures are good enough to use despite their flaws to a more fundamental discussion of how to achieve good measures, how to assess whether current measures measure up, and whether the costs of attaining good measures are worth the benefits. Professional societies, such as the American Heart Association, have an important role in speaking authoritatively about the science of clinical issues; performance measurement lacks a similar authoritative voice.
Such an endeavor needs to explicitly consider the unintended, yet harmful, consequences of misapplication of performance measures, whether resulting from the measures themselves, in how they are reported and assessed, or in the costs of collecting invalid performance data. There is substantial literature detailing such untoward consequences [28], some from measures experts who promote the use of performance measurement. For example, some have expressed concern that unless carefully designed, public reporting and pay-for-performance programs will increase racial and ethnic disparities [29].
Robert A. Berenson, MD is an institute fellow at the Urban Institute.
Peter J. Pronovost, MD, PhD is the director of the Armstrong Institute for Patient Safety and Quality at Johns Hopkins, as well as Johns Hopkins Medicine’s senior vice president for patient safety and quality.
Harlan M. Krumholz, MD, is the director of the Yale-New Haven Hospital Center for Outcomes Research and Evaluation, director of the Robert Wood Johnson Foundation Clinical Scholars program at Yale University, and the Harold H. Hines, Jr. professor of cardiology, investigative medicine, and public health.
The authors thank Lawrence Casalino, MD, PhD, chief of the Division of Outcomes and Effectiveness Research and an associate professor at Weill Cornell Medical College, and Andrea Ducas, MPH and Anne Weiss, MPP of the Robert Wood Johnson Foundation for their helpful comments on this paper. This research was funded by theRobert Wood Johnson Foundation, where the report was originally published.
Notes
28. Casalino 1999; Curtis JR and Levy M. “Improving the Science and Politics of Quality Improvement.” Journal of the American Medical Association, 305(4): 406-407, 2011; Baker DW and Qaseem A. “Evidence–Based Performance Measures: Preventing Unintended Consequences of Quality Measurement.” Annals of Internal Medicine, 155(9): 638-640, 2011; Pogach L and Aron DC. “Sudden Acceleration of Diabetes Quality Measures.” Journal of the American Medical Association, 305(7): 709-710, 2011; Wachter, 2006.
29. Casalino LP, Elster A, Eisenberg A, et al. “Will Pay-For-Performance and Quality Reporting Affect Health Care Disparities?” Health Affairs, 26(3): w405-w414, 2007; Chien, et al, 2011.
Categories: Uncategorized
