As quality measures are developed and quality monitoring programs seek to refine their measure list to those that are the most impactful and least burdensome, there will continue to be discussions about the appropriateness of different measures for different programs, clinical areas, and provider types. There will also be a lot of opinions and assessments of current measures, as individuals and groups advocate for certain measures and discourage the use of others. Although many of the examinations of measure appropriateness typically focus on the specifics of an individual measure, you have probably noticed that from time to time there are arguments for or against “types” of measures, including those comparing and contrasting process measures versus outcome measures. Therefore, a closer examination of these types of measures is warranted, including the strengths and weaknesses of each.
These are, in general, measures regarding the steps providers take (or don’t take) during a patient encounter and/or in the course of providing care. Process measures are intended to encourage providers to do things such as screen for a condition or risk-factor, take the time to go over a surgical safety checklist, provide discharge instructions, etc. Proponents of process measures laud several of their attributes:
When people talk about the drawbacks of process measures, they usually discuss the following:
There is such a variety of process measures that both the pros and cons listed above are more applicable to some measures than to others. And, certainly these are not exhaustive lists. But in general process measures can be effective in encouraging certain types of behaviors and facilitating certain steps or actions which can improve care delivery and the patient’s experience. However, directly linking improvements in process to the subsequent health and outcomes of patients may be challenging.
Ultimately, some would argue, the purpose of measuring aspects of healthcare delivery is to improve patient outcomes, whether it means reducing utilization (hospitalization, readmissions, ED visits) or adverse events (infections, complications, death). So, when it comes to promoting the use of outcome measures, proponents usually note the following:
However, just as with process measures, outcome measures have potential limitations that should be noted:
Not all of these are applicable in all situations - but these are the kinds of potential drawbacks referenced when there are concerns regarding some outcome measures. Just as was the case with process measures, the limitations of outcome measures do not mean that these types of measures are not or cannot be effective for tracking and comparing quality. They are simply things to be considered when selecting appropriate measures and interpreting results.
So, which is better to measure, a process or an outcome? It depends; each type of measure has its strengths and weaknesses, and different situations, goals, resources, and time-frames call for different measures - or sometimes even a combination of the two. Be careful when someone universally praises one type over the other, because they each have their place if they are appropriately defined, created, implemented, and interpreted. Also be aware of the strengths and weaknesses of each as you determine what is best for your organization and situation. Consider how a chosen measure will allow you to provide continuous feedback to your team and facility, so that you can track progress, celebrate successes, and quickly address areas of need. Understanding how each type of measure may facilitate change or encourage innovation can help you design and implement the most effective intervention possible.
The Role of Risk-Adjustment
When using outcome measures (like mortality or readmission rates) to compare quality between providers or facilities, the notion of risk-adjustment is usually part of the conversation: should it be done, how best to do it, how well it performs, etc. There is a lot of literature out there devoted to the methods and effectiveness of risk-adjustment, and for good reason: there is enormous variation in provider and facility characteristics (size, capabilities, available resources, staffing, location, case-mix, etc.), and it is well-known that some factors can significantly impact access to care, timing of care, and ultimately patient outcomes. This variety and these differences should be acknowledged, studied, understood, and when possible, used to adjust measures or stratify those being measured. When done well, risk-adjustment methods can do an excellent job of mitigating the impact of uncontrollable factors and make comparisons far more fair. However, there are a couple of things to keep in mind when it comes to risk-adjustment:
This doesn’t invalidate risk-adjustment as a meaningful tool to aid in making meaningful comparisons of quality, but it is certainly not a panacea, especially in a complex system like healthcare. Additionally, we are continuously learning about new things that can impact the quality of care delivered and patient outcomes; there are studies that claim that stroke outcomes can be impacted by EMS care before the patient even reaches the hospital, and the quality of SNFs near a hospital may influence the likelihood of hospital readmission. This underscores the notion that there are very likely unknown factors that could be influencing patient outcomes, and no amount of risk-adjustment can completely level the playing field. That is why as users and interpreters of measures (unadjusted and risk-adjusted alike) we need to understand the background and the factors at work, and make judgments that take into consideration more than just the single piece of information of a risk-adjusted rate. Gaining an understanding of risk adjustment methods - as well as basic concepts of measure development - are crucial for all of us working to improve care and outcomes of patients.
Those of us who work in quality improvement understand these concepts, including risk-adjustment, to some degree. Developing a deeper understanding of the tools commonly used to identify, develop, and adjust measures can only improve our collective ability to assess and evaluate the quality of care delivery. By seeking ways to further our aptitude and proficiency with data-related aspects of quality measurement through continuous learning and frequent discussions, we will improve improving patients’ health and experience of care.