Risk adjustment for home and community-based services quality measurement

Home and community-based services are support services provided to persons with disabilities that enable them to live in the community. With a goal of improving recipients’ health and quality of life,¹ home and community-based services include a wide array of services and supports, such as personal care; care management; round-the-clock services, such as group homes, residential habilitation, and shared living; supported employment; day services, such as day habilitation, and adult day care; nursing; home-delivered meals; durable medical equipment; home modification; and transportation.²

The growing demand for home and community-based services has led to an increase in efforts to measure the quality, including the effectiveness, of these services and supports in helping persons served to achieve their desired health and quality-of-life outcomes.¹ States and other entities paying for home and community-based services want to understand which types of services or service arrangements optimize outcomes, including health care and person-centered outcomes. In this paper, person-centered outcomes refers to outcomes related to living a more meaningful life as measured using person-centered constructs, such as having more choice and control in everyday living.³

Some of the existing data collection efforts are limited in measuring the effectiveness of person-centered services because they may not distinguish whether home and community-based services outcomes are due to differences in the quality of provider services or differences attributable to the characteristics of the persons served. Monitoring the quality of home and community-based services requires being able to compare quality outcome measure data across programs or providers, and for some outcomes data, that requires adjusting the data for the characteristics of persons served, such as varying levels of motor functioning, cognitive abilities, or behavioral conditions. For example, outcome data about level of community inclusion could be collected for one assisted living facility that serves people who are medically fragile and have limited mobility and for a second assisted living facility that serves people who are generally healthier and have few impairments and disabilities. When community inclusion data that have not been risk adjusted are compared for these two facilities, differences in data results could reflect differences in how the program operates (i.e., operations that lead to improved community inclusion) or differences in the characteristics of the people they serve or a combination of both of these factors. Risk adjustment is intended to control for differences in the characteristics of persons served across providers so that meaningful comparisons of organizational outcome quality measure results can be made.

The purpose of this brief is to provide a nontechnical description of the purpose of risk adjustment, and approaches to risk-adjusted data, including the application of exclusion criteria, stratification, and the use of regression model results. We begin by describing quality measurement.

Measuring quality involves the collection and analysis of data using surveys and other measurement instruments as part of business operations (e.g., by staff or an independent contractor). While surveys and measurement instruments must have demonstrated item and/or multi-item scale reliability (measures the concept consistently) and validity (measures what it is designed to measure),⁴ measures intended to quantify levels of quality (i.e., quality measures) also need to have demonstrated reliability and validity.⁵ For a quality measure to be a valid measure of quality, it may require risk adjustment if there is a concern that differences in the providers’ quality measure results may be driven by the characteristics of the persons served in addition to the quality of the services provided.⁶ This is typically a concern for home and community-based services quality measures that aim to evaluate the extent to which a system is effective in improving person-centered outcomes, because the disabilities that people have may impact the amount of control they have over their lives and over their ability to be integrated with the community.

If persons receiving home and community-based services were randomly distributed across programs so that every program served persons with similar characteristics, then we could fairly compare outcomes data across these programs and make inferences about the quality of the services provided. However, in reality, some providers serve persons who have more medical conditions or have more complex conditions. This means the characteristics of persons served across providers and service arrangements differ, and each group’s unadjusted outcomes, such as the percentage of people admitted to the hospital, are likely to differ because of these differences in the mix of persons served by each provider. Risk adjustment, sometimes referred to as a process of “leveling the playing field,” is used to allow comparisons of outcomes data across organizations by controlling (i.e., adjusting) for differences in the characteristics of persons served.⁷

Risk adjustment is used in quality measurement to control for differences in the relevant characteristics of persons when the outcomes of two or more groups are compared. The process of risk adjustment is similarly applied in research studies that adjust for differences when comparing study outcomes for two or more groups of people. The term risk adjustment has been defined by the Agency on Healthcare Research and Quality as “a statistical process used to identify and adjust for extraneous variables.”⁸

The characteristics of the persons served that can result in different outcomes can be called “risk factors.” There are several ways that quality measures can adjust for these risk factors, and more than one method can be used for a single quality measure. Three methods we will describe are: use of regression coefficients, stratification, and application of exclusion criteria.

A common method used to risk adjust quality measures involves developing regression models with the outcome of interest as the dependent variable to identify the relationship between each risk factor and the outcome for the population of interest. The coefficient for each risk factor from the regression model, which represents the average association between the risk factor and the outcome, is then used to adjust data for calculating the quality measure result. This is done by using the risk factor data (e.g., whether the risk factor is present or not at the person level or the percentage of patients with the risk factor at the provider/program level) and regression coefficients to calculate the expected result given each provider’s mix of persons served, which is compared with the observed (i.e., actual) result.

Stratification is another method that can be used to control for differences in the mix of the characteristics of persons served when comparing outcomes. For example, provider-level quality measure results for persons with different types of disabilities (e.g., severe mental illness, physical disabilities) can be analyzed and reported separately (i.e., stratified) if outcome results are expected to vary across these populations.

A third method that can be used to control for differences in the mix of characteristics of persons served involves applying exclusion criteria. For example, a measure of opportunities for meaningful employment may exclude persons below or above certain age thresholds. Because home and community-based services outcomes include person-centered concepts, such as choice and control over aspects of life, some quality measure concepts may not be considered important to all users, and an exclusion criterion may be that the measure concept is not important to the person.

Compared with risk adjustment for health care quality measures, risk adjustment for home and community-based services measures is in its infancy. A recent literature review by Houseworth et al.⁹ found that several characteristics are risk factors for home and community-based services quality measures, including chronic conditions, level of mobility, and cognitive functioning.

When a quality measure focused on an outcome is being developed, a literature review should be conducted to identify factors present at the start of services that would affect the outcome of interest. Consulting with individuals with lived experience and other experts to identify these risk factors could also be helpful. Clinical factors may include medical conditions, co-existing conditions, level of motor functioning, or level of cognitive functioning. Demographic factors, such as age, are often used to adjust for health care outcomes because older age is often associated with outcomes, such as functional outcomes.^10,11 Social factors, such as socioeconomic status, are sometimes used as risk adjusters; however, there are concerns about adjusting quality measure results for social factors. As noted in the report entitled Social Risk Factors Performance under Value-Based Purchasing,¹² adjusting quality measures for social factors may mask differences in the quality of care provided, which would reduce our ability to identify and reduce these differences. Further, when these differences in quality are related to poor performance, bias, or discrimination, adjusting quality measure results for social factors could excuse the delivery of worse services to persons with those social risk factors. However, if a social risk factor is associated with worse outcomes, and there is no risk adjustment, providers who offer services to persons with this social risk factor may have worse quality outcome measure results compared with other providers due to the lack of risk adjustment.¹³ A 2022 report from the National Quality Forum¹⁴ recommended considering adjustment for social factors when there is a conceptual rationale and empirical relationship. To mitigate concerns that risk adjustment masks disparities in care, the report recommended stratification of measure results by the social factor.

Once candidate risk factors are identified, they need to be tested using data to demonstrate evidence of the association between the risk factor and the outcome. The risk factors can be used as exclusion criteria, to stratify the populations, or to adjust for the factor such as with a regression model. Each outcome should have its own set of risk factors that are considered and tested.

For quality measures that rely on regression models for risk adjustment, there is a two-step process. The first step is to run a regression model to determine the coefficients for the intercept and each risk factor. The second step involves applying the coefficients to the provider data to create an expected result. The expected results are compared with the observed results.

All approaches require collecting data for the outcome of interest, which may be a measurement instrument, as well as the characteristics that are risk factors. For home and community-based services, this could include data routinely gathered as part of a comprehensive assessment.

For implementing quality measures, specifications may require random sampling from the population of persons served, or routine data collection may mean the entire population is included. Quality measure reliability testing is typically conducted to determine a minimal sample size for calculating reliable quality measure results.

Although risk adjustment is very important for relevant quality measures, including most outcome quality measures, we acknowledge it is complicated. The selection of risk adjusters will vary based on the quality measure and requires thoughtful consideration of the outcome and how a risk factor affects that outcome and whether the provider has control over that factor. Not adjusting for a risk factor could result in providers avoiding serving persons with those risk factors because of anticipated worse outcomes and the consequent impact on the provider’s quality measure results. However, adjusting for a factor inappropriately would result in masking (i.e., hiding) quality differences.

Another issue relates to the availability of risk factor data for testing and then for routine data collection. Data may not be routinely available for each risk factor, which means adjustment for that factor is not possible. Further, we may not know all the relevant risk factors for each outcome, so no adjustment for these unknown factors is possible. For these reasons, risk adjustment is often imperfect.¹⁴

Risk adjustment is warranted for some quality measures, particularly quality outcome measures, because each provider serves a unique mix of persons with varying characteristics, and a provider’s case-mix can confound their outcome results. The purpose of risk adjustment in quality measurement is to adjust for these differences in a provider’s mix of persons served. Risk adjustment can be accomplished through application of exclusion criteria, stratification, and the use of regression model results.

In this publication, we do not report on, analyze, or generate any data.