HHS site invites public to compare mortality rates at 4,700 hospitals
■ Most facilities are listed with rates no different from national norms, leaving skeptics questioning the data's value.
For the first time, physicians and patients can directly compare hospitals' mortality outcomes for heart attack, heart failure and pneumonia care using the Dept. of Health and Human Services' Hospital Compare Web site.
The risk-adjusted data on Medicare patients from 2005 to 2007 were unveiled in August and represent the first set of outcomes metrics made available in such detail on the site. Government officials and patient-safety advocates want the new information to help patients make medical decisions and spur hospitals and physicians to make systemic changes to lower their patient death rates.
The hope is that the mortality data will solidify Hospital Compare's role "as a key driver in improving the quality and the reliability of care in the nation's hospitals," according to a statement by Centers for Medicare & Medicaid Services Acting Administrator Kerry Weems.
By measuring 30-day mortality, as opposed to in-hospital mortality, HHS is seeking to encourage better discharge planning and communication with long-term-care facilities and primary care physicians.
But some doctors and experts on health care reporting say the source of the information -- administrative claims data -- does not capture all of the relevant clinical facts, making apples-to-apples comparisons questionable. And, they argue, because nearly all of the approximately 4,700 U.S. hospitals are listed on Hospital Compare as achieving mortality outcomes "no different than the national rate," the value of the information is limited.
Some physicians said the new data are unlikely to drive patients' decisions about where to seek care.
"I am not sure how important the Web site is going to be, especially for my patients," said James Cunnar, MD, a family physician in Naperville, Ill. "My experience is that patients go where it's convenient."
The new mortality metrics are the latest additions to the HHS site. Hospital Compare also lists data on dozens of process measures for heart attack, heart failure, pneumonia and surgical care. HHS added information last spring about patients' experience of care at hospitals. In August, the agency unveiled metrics of pediatric asthma care processes.
Process measures are less controversial than outcomes metrics because such process measures look only at whether clinically recommended care was delivered. But studies have found that using process measures as a proxy for outcomes is problematic. For example, a July 5, 2006, Journal of the American Medical Association study of 962 hospitals found that performance on heart attack process measures accounted for just 6% of differences in risk-adjusted acute myocardial infarction mortality rates in hospitals.
"We say loud and clear this is just one tool," said cardiologist Harlan M. Krumholz, MD, director of the Yale-New Haven Hospital Center for Outcomes Research and Evaluation, with which CMS contracted to develop the mortality metrics.
Dr. Krumholz and his team tested whether Medicare administrative claims data could be used reliably to adjust raw mortality data for patients' illness severity. When the team ran models predicting death rates using claims data versus medical records, there were differences in accuracy, but "results were very close" -- good enough to use in public reporting.
The American Medical Association, in the context of policy addressing pay-for-performance programs, says that "accurate data and scientifically valid analytical methods" should be used to assess physician performance. The AMA is involved with the AQA Alliance, a multistakeholder group working to develop physician performance measures for public reporting.
Will scrutiny improve care?
Some hospital leaders expressed doubts that the new mortality metrics offer meaningful quality comparisons. "Administrative data was designed for billing and is very limited in detecting quality differences," said Evan Benjamin, MD, vice president and chief quality officer at Baystate Medical Center in Springfield, Mass. "There are often many conditions that put a patient at higher risk of dying, but these conditions may not be captured in these administrative databases."
Dr. Benjamin said widely varying documentation and coding standards present another obstacle to fair and accurate ratings. "We can compare ourselves to ourselves, but when you compare our hospital to another, we wonder about the standardization to make this comparable."
Baystate's performance on the mortality metrics surpassed national rates, but the differences were not statistically significant. In fact, more than 97% of hospitals are listed as "no different than the national rate" on pneumonia mortality; 98% are so classified on heart failure mortality; 99% are "no different" on heart-attack mortality.
It's a problem that so few hospitals are listed as having rates no better or worse than the national rate, said Jim Conway, senior vice president of the Institute for Healthcare Improvement, a quality-improvement organization in Cambridge, Mass.
"We're only shining a light at either extreme end of the spectrum," he said. "We'll have to be much more discriminating going forward if we really want to drive improvement."
As the push for more public reporting moves ahead, there are doubts about the impact on quality improvement and patients' decision-making. A review in the Jan. 15 Annals of Internal Medicine found that public reports stimulate quality activity in hospitals, but there is little evidence that they improve the effectiveness, safety or patient-centeredness of care. Moreover, there is not much evidence to show that patients rely on public health care reports.
"This kind of information matters to probably less than 1% of my patient population," said Joseph C. Perkinson, MD, a family physician in Victoria, Texas. "The reason they go to a facility isn't based on mortality data. They're going to want to know that the nurses were 'nice to me,' or 'my dinners were always hot.' "