Sometimes. But quite often not.
Indicator use is often misguided
Broad hospital ratings (and other rankings based on specific indicators) regularly make leading news. Consumers & administrators alike gather anxiously to see how their local hospital performed. But at deeper levels, performance indicators have become embedded in the fabric of our healthcare system. They are used ostensibly to ensure accountability in accreditation, public reporting, pay-for-performance, measuring “value,” certification, executive compensation and dozens of other places.
Leading quality organizations have endorsed a myriad of “evidence-based measures of quality of care” such as re-admissions, pressure ulcer rates, infections and hand washing. Unfortunately, in their continuing efforts to “move the metrics,” many hospital leaders, government officials and consumers have lost perspective…
Great Idea: Poorly Implemented
The original theory behind using indicators is reasonable enough and actually a pretty good idea.
Consider vital signs. In clinical care, temperature, pulse, respiration & blood pressure are the vital signs, used to provide a quick assessment of the overall well-being of a patient. While these measurements don’t provide any sort of in-depth analysis, generally speaking, if all of the vital signs are good, then the person is usually doing OK. But when one or more of these vital signs deviate from the norm, they can give clues as to what’s wrong and where to look further.
Similarly, a balanced set of hospital quality indicators can be used as “vital signs,” providing an easy way to get a broad perspective on overall performance. If hospital leadership chooses a relatively representative set of indicators and monitors them regularly, the indicators can provide early warning of problems brewing.
So what went wrong?
Hospitals are chasing the numbers, not overall quality
Instead of being a means to an end, hospital quality indicators have become an end in themselves – especially now that hospital ratings affect reimbursement.
Instead of working to maintain overall quality of care, some organizations have focused on just getting better numbers. The goal has become to get the numbers that will generate the best hospital ratings, the good report card, the required payments, the accreditation level or other stamp of approval. Overall quality is sometimes secondary.
A hospital may have great pressure ulcer rates, but that’s really all it means – that they have great pressure ulcer rates.
Check out some hospital web sites. Look at the hospital ratings and other publicly reported indicators and it’s hard to find an organization that performs well across the board. Scores may be great in one area but then bad in another. In several instances recently, hospitals have had great scores on specific safety indicators but had high profile and pervasive safety issues blow up in the news.
So, what does the actual “evidence” say about performance indicators?
- Little research looking at the value or ROI of indicator use
- Only 1 out of 21 quality measures reliably indicated a hospital’s patient safety profile
- No hard evidence of system quality improvement as a result of indicator use
- Evidence that patient satisfaction correlates poorly with overall clinical quality of care
- Some indicators turn out not to measure what we think they measure
- Performance improvements from pay-for-performance don’t last
- “Value-for-money” is highly subjective
Don’t Give Up!
Does this mean we should abandon performance indicators?
Not at all.
In a system that prizes value, we must create and pursue meaningful performance goals. But, organizations need to use indicators as they were originally intended, remembering that these metrics are not a substitute for a good overall quality improvement approach.
Consumers should use performance indicators as a starting point in evaluating care, not an end.
Any conclusions about hospital quality and safety performance should be based on an overall body of information, collected over time, not a single indicator or ranking.
Hospital ratings and other performance indicators provide good foundations for conversations with provider, leaders and consumers. Along with talking about performance indicators, stakeholders should add their own observations about facilities, clinicians, support staff and the way that they are treated.