Hospital rankings may rely on faulty data

https://www.eurekalert.org/pub_releases/2016-10/rumc-scr100616.php

Public Release: 6-Oct-2016
Study: Hospital rankings may rely on faulty data
Greater transparency and validation needed
Rush University Medical Center

The October 2016 issue of The Joint Commission Journal on Quality and Patient Safety, features the article "Consumer Rankings and Health Care: Toward Validation and Transparency" by Bala Hota, MD, MPH, and co-authors, Rush University Medical Center (RUMC), Chicago. After RUMC received a lower than expected ranking for patient safety in the 2015-16 U.S. News & World Report's (USNWR) "Best Hospitals" rankings, the authors compared the data that USNWR used for their hospital to their own internal data.

The authors found that the USNWR data showed many more patient safety events, such as pressure ulcers, almost all of which had actually been present at the patient's admission. Suspecting a broader problem, Hota et al. analyzed data on a sample of hospitals and found that RUMC was not the only organization with discrepancies in data. False-positive event rates were common among high-transfer and high-volume hospitals.
[So some were being blamed for things that were not their fault.]
•••••

In an accompanying editorial, "The Quality Measurement Crisis: An Urgent Need for Methodological Standards and Transparency," David M. Shahian, MD, Elizabeth Mort, MD, MPH, and Peter J. Pronovost, MD, PhD, reflect on the Hota et al. article to conclude, "Just as health care providers have ethical and moral responsibilities to the public they serve, rating organizations and journalists that grade providers have similar obligations--in their case, to ensure measure validity and methodological transparency."

•••••

Rating hospital quality is not as simple as rating the quality of a car, though. One reason is that patients vary in terms of how sick they are when they arrive at the hospital or undergo surgery. Older patients with multiple health problems or advanced diseases are more likely to develop an infection or complication than younger, healthier patients -- even though they receive the same quality of care.

The best hospital rating systems try to account for severity of illness. However, a Rush data expert's recent analysis of U.S. News & World Report's "Best Hospitals" rankings shows that these rating systems often are dependent on faulty data sets and performance metrics.

•••••

"When we compared the data that U.S. News used for our hospital to our own internal data, we found big differences," says Hota, Rush's chief research informatics officer. "U.S. News showed many more patient safety events than our actual rates."

For example, U.S. News data showed that 25 patients at Rush had developed pressure ulcers, or bed sores, over a particular time period. In fact, only one patient had developed a pressure ulcer while at Rush.

•••••

"When we compared the data that U.S. News used for our hospital to our own internal data, we found big differences," says Hota, Rush's chief research informatics officer. "U.S. News showed many more patient safety events than our actual rates."

For example, U.S. News data showed that 25 patients at Rush had developed pressure ulcers, or bed sores, over a particular time period. In fact, only one patient had developed a pressure ulcer while at Rush.

•••••