Supplemental Material to the CMS MMS Blueprint
September 2021 Page 1
Population Health Measures
The United States (U.S.) spends nearly twice the average of other Organization for Economic Co-
operation and Development (OECD) countries expenditures on health, but has the lowest average life
expectancy, performs worse than average on many population health outcomes, and has more
outcome-related disparities compared to peer OECD countries (OECD, 2019 ; Tikkanen, & Abrams,
2020, January ). A recent analysis of 2020 Commonwealth Fund International Health Policy Survey
data found lower income adults in the U.S. fare relatively worse on affordability and access to primary
care and income-related disparities across domains than those in ten other high-income countries
(Doty, Tikkanen, FitzGerald, Fields, & Williams, 2020 ).
The U.S. and CMS acknowledge the importance of quality measurement and that quality reporting and
incentive programs have improved outcomes and how measured entities deliver care. Additionally,
population health measurement is critical to improving the nation’s overall health. As such, CMS is
committed to four principles for improving population health:
Establish health equity as a strategic priority.
Empower and enable measured entities and other stakeholders to take a data-driven approach
to measuring and improving population health.
Leverage state
1
innovation and local leadership through partnerships.
Address all determinants of health including clinical, social, behavioral, and environmental
factors.
1
References to states include the District of Columbia and the territories.
1 Introduction to Population Health ........................ 2
2 Approach to Population Health
Measurement and Improvement .......................... 3
3 Considerations for Developing, Evaluating,
and Maintaining Population Health
Measures ............................................................... 5
3.1 Measure Conceptualization ......................... 5
3.2 Measure Specification .................................. 8
3.3 Measure Testing ......................................... 10
3.4 Measure Implementation .......................... 11
3.5 Measure Use, Continuing Evaluation, and
Maintenance .............................................. 12
4 Key Points ............................................................ 12
References ................................................................ 13
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 2
This document provides a high-level overview and definition of population health. It addresses
considerations for population health quality measures with respect to the Measure Lifecycle. As
population health measures evolve, so will this document.
1 INTRODUCTION TO POPULATION HEALTH
CMS defines population health as health behaviors and outcomes of a broad group of individuals,
including the distribution of such outcomes affected by the contextual factors within the group. The
definition is a slight variation from the widely cited 2003 Kindig & Stoddard definition of population
health
2
adopted by many including the U.S. Department of Health and Human Services (HHS), Office of
the National Coordinator for Health Information Technology (ONC) (ONC, 2020 , p.7). Note that CMS’s
definition does not delineate how to define the groups themselves. Therefore, when developing
population health measures clarity of the denominator is critical for measurement. The definition also
does not delineate the contextual factors. The current approach for commonly published summary
measures of population health, such as mortality rates, primarily uses geopolitical areas. However,
other population identifiers may include panels of patients (e.g., persons assigned to a specific
measured entity or measured entity team), members of a health plan, or members of a specific social
demographic (e.g., women of color). Social determinants of health (SDOH) (e.g., economic stability,
education, social and community context, health and health care, and neighborhood and built
environment), and social risk factors (e.g., food and housing insecurity, lack of transportation), also
impact population health significantly (Green & Zook, 2019 ).
CMS defines a population health measure as a broadly applicable indicator that reflects the quality of a
group’s overall health and well-being. Examples of measure topics include access to care, clinical
outcomes, coordination of care and community services, health behaviors, preventive care and
screening, and utilization of health services. Without guidance as to how to define a group, these
working definitions reflect important distinctions between population health measures and quality
measures. The current intent of quality measures is to assess the quality, cost, or efficiency of particular
services to individuals by healthcare setting, so there is an attachment of quality measures to particular
services and specific types of measured entities. Population health measures would not necessarily have
these restrictions. Population health measures are more expansive in that they include what is
happening outside the direct healthcare system.
Section 1890 of the Social Security Act (the Act) requires the consensus-based entity (CBE), currently
National Quality Forum (NQF), to report annually on its work to Congress and the HHS Secretary. Section
§1890(b)(5)(A) of the Act also requires the CBE to include descriptions of matters related to convening
multistakeholder groups to provide input on national priorities for improvement in population health.
The 2019 report of the NQF Prevention and Population Health Standing Committee identified six
population health measure gaps in the NQF portfolio:
measures that detect differences in quality across institutions or in relation to certain
benchmarks, but also differences in quality among populations or social groups
measures that assess access to care
measures that assess environmental factors
measures that address food insecurity
2
the health outcomes of a group of individuals, including the distribution of such outcomes within the group
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 3
measures that address language and literacy (e.g., health literacy)
measures that address social cohesion
2 APPROACH TO POPULATION HEALTH MEASUREMENT AND IMPROVEMENT
The population health of a group is dependent upon the interplay of several factors (e.g., economic,
social, environmental, cultural, behavioral), of which clinical care represents only a portion
(Stoto, 2014 ). As such, population health depends on a multiplicity of factors, many of which are not
within CMS’s traditional role to address as a healthcare services payer. Because of this, the achievement
of measurement and improvement in population health depends upon innovation, collaboration, and
coordination across stakeholders. These include local, tribal, state, national government agencies, and
the community, including but not limited to, members of the care team, payors, hospitals, and nursing
homes in delivering care to the target population(s), as well as community members and organizations.
Figure 1 reflects this overlap of roles in improving population health and showing the joint influence on
population health outcomes by healthcare, government, the community, and the private sector.
Figure 1. Population Health and the Triple Aim
No single entity in the public or private sector has sole capacity or responsibility for overall population
health improvement. Multiple organizations, public and private, perform public health activities. As
opposed to other sectors with interorganizational partnerships and alliances, these public health
activities are largely uncoordinated, leading to gaps, inefficiencies, and inequities (Mays & Scutckfield,
2010 ). Systems thinkingunderstanding the collective effect of multiple actors and actionsis
necessary to organize and sustain population health improvement (Woulfe, Oliver, Zahner, & Siemering,
2010 ). There must be “a shared measurement system.By extension, a multi-sector approach is
essential to addressing the multiple determinants of population health. Emerging partnerships between
Population health improvement
requires a multisector approach.
Government agencies, including tribal
agencies, measured entities and
payors, community service providers,
and the private sector can join
together to improve the health of
every person and population in their
communities, together through
measurement, innovation,
collaboration, and improvement to
achieve the triple aim goals of better
care, smarter spending, and healthier
people and communities.
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 4
measured entities, federal, state, tribal, and local health agencies, community service providers, and
multiple other organizations (e.g., education systems and justice system), and the private sector can
help call attention to underlying problems, shift resources to increase returns on investments, and
sustain population-level improvements in health.
Peter Drucker, among others, stated that “you cannot improve what you do not measure.
Stoto (2014 ) noted measurement is critical to improving population health. The Institute of Medicine
(IOM), now known as the National Academy of Medicine, said “Without a strong measurement
capability, the nation cannot learn what initiatives and programs work best, resources cannot be guided
toward the most promising strategies, and there is little ability to promote accountability in results”
(IOM, 2013a , p. 2).
Parrish (2010 ) identified three approaches to measuring population health:
aggregating health outcome measurements made on people into summary statistics, such as
population averages or medians
assessing the distribution of individual health outcome measures in a population and among
specific population subgroups
measuring the function and well-being of the population or society itself, as opposed to
individuals
In 2013(b), the IOM , identified criteria for selecting and prioritizing measures of quality for use in
population health improvement:
Conditions or outcomes for measurement should be
reflective of a high preventable burden
actionable at the appropriate level for intervention
Measures should be
timely
usable for assessing various populations
understandable
methodologically rigorous
accepted and harmonized
Of particular importance is CMS’s partnerships with state agencies, Medicaid in particular. Because all
healthcare is local, states are in the best position to assess the unique needs of their respective
Medicaid-eligible populations and drive reforms that result in better health outcomes. CMS is
committed to ushering in a new era for the federal and state Medicaid partnership, where states have
more freedom to design programs that meet the spectrum of diverse needs of their Medicaid
population. CMS aims to empower all states to advance the next wave of innovative solutions to
Medicaid’s challenges – solutions that focus on improving quality, accessibility, and outcomes in the
most cost-effective and equitable manner. Working together, through local organizations, tribal
agencies, state agencies, other parts of HHS (e.g., Indian Health Service), and federal partners such as
the Departments of Education, Agriculture, Transportation, Housing and Urban Development, and
Veterans Affairs, CMS believes they can collectively manage and improve population health for all
individuals and families served by CMS programs.
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 5
Current CMS initiatives that seek to focus on improving population health and do not focus solely on the
quality of care rendered by a singular measured entity include
Accountable Care Organizations (ACOs): ACOs are responsible for clinical care, costs, and
outcomes in a particular population of Medicare patients.
Accountable Health Communities (AHCs) Model: AHCs address a gap between clinical care and
community services in the healthcare delivery system by testing whether identifying and
addressing the health-related social needs of Medicare and Medicaid beneficiaries through
screening, referral, and community navigation services will impact healthcare costs and reduce
healthcare utilization.
Medicare Advantage Organizations (MAOs): MAOs are responsible for care in the population of
enrollees. MAOs may provide additional services not covered by traditional Medicare
(Tompkins, Higgins, Perloff, & Veselovskiy, 2013 ), such as transportation to appointments and
non-permanent home modifications to allow beneficiaries to age in place.
Program of All-Inclusive Care for the Elderly (PACE) : PACE is a Medicare and Medicaid
program that provides comprehensive medical and social services to certain frail, elderly people
living in the community. PACE helps people meet their healthcare and social needs in the
community instead of going to a nursing home or other care facility.
3 CONSIDERATIONS FOR DEVELOPING, EVALUATING, AND MAINTAINING
POPULATION HEALTH MEASURES
3.1 MEASURE CONCEPTUALIZATION
Conceptualization of population health measures should identify opportunities for improvement at the
population level, rather than only seeking to identify gaps or variations in clinical care. Similarly,
information gathering and business case development should be at the population level to identify
health differences among populations, including disparities among subpopulations. During measure
conceptualization, measure developers should always consider whether to stratify and/or risk adjust the
measure(s). Measure developers should estimate the potential for population level improvement as well
as the potential benefits, burdens, and costs of achieving the population health goals.
Conceptualization of population health measures presents unique challenges for measure developers.
CMS notes that the current healthcare delivery system lacks an incentive structure to support local
problem-solving. For example, insurers do not usually pay measured entities and systems for their
efforts, e.g., screening for social risk factors or coordinating with local community providers and
governments beyond the clinical setting. Existing value-based purchasing programs do not reward
coordinated community health improvement efforts. Although population health improvement is a
priority goal, there are limited incentives tied to improvements or disincentives to worsening of
population health.
Although the focus of population health measures differs from clinical quality measures, measure
development should address alignment of the population measures with existing or potential measures
of clinical care and other drivers of population health improvement. These may include individual
behaviors, prevention, and social determinants of health, e.g., housing, transportation, food security,
economic stability, education, social and community context, access to healthcare, and neighborhood
environment.
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 6
3.1.1 Environmental Scan
Where should measure developers go to find population health areas needing improvement? Measure
developers need to expand their environmental scan search criteria beyond their usual sources for
quality measures, e.g., the CMS Measures Inventory Tool and NQF Quality Positioning System . For
example, the OECD has 80 key indicators for population health and health system performance. These
key indicators use data from official national statistics to compare countries in terms of health status
and health-seeking behavior, access to and quality of healthcare, and the resources available for health
(OECD, 2019 ). These key indicators may provide insights to population health areas needing further
investigation and offer ideas for measure concepts.
3.1.1.1 Community Health Needs Assessment
The Patient Protection and Affordable Care Act of 2010 requires a community health needs assessment
(CHNA) and implementation strategy every three years for all Section 501(c)(3) nonprofit hospitals
(charitable hospitals) working with public health agencies and community members (Internal Revenue
Service [IRS], n.d. ). The IRS requires hospitals to submit their needs assessment and implementation
strategy with their IRS Form 990 and provide an annual description of how the hospital is addressing the
needs identified in their CHNA and implementation strategy. The IRS also requires hospitals to make
their CHNA and implementation strategy publicly available, which is usually only on the individual
hospital’s website. There is no oversight on the content of these CHNAs, no central repository, no state-
based repository, and no widely available measures focused on measuring the impact of the hospitals’
implementation strategy on the population of the community. These CHNAs and implementation
strategies have the potential to provide ideas for improvement opportunities in population health.
However, a review of the first CHNAs in Texas (Pennel, McLeroy, Burdine, Matarrita-Cascante, & Wang,
2016 ) found that few included health improvement or program goals and objectives. The Association
of State and Territorial Health Officials (ASTHO) website provides information on how to conduct
CHNAs. The ASTHO website also includes some case studies. These case studies provide a high-level
overview of the project, steps taken, results, lessons learned, long-term goals, and sustainability. All
these projects have the goals of improving coordination of community benefits and improving the
health of the community. Similar to the Texas study, these case studies lack measures to determine
whether the projects met their goals, especially for improving the health of the community. Measure
developers can review these case studies and identify measure concepts to assess whether the
community’s health has improved.
3.1.1.2 Research Social Determinants of Health and Social Risk Factors
Research into the effects of SDOH and social risk factors may also provide measure developers with
ideas for population health measure concepts. For example, a recent analysis of state and local
government spending on non-healthcare services (e.g., education, social services, environment, and
housing) noted an association between increased spending and lower infant mortality rates among
certain high-risk populations (Goldstein, Palumbo, Bellamy, Purtle, & Locke, 2020 ).
3.1.1.3 Healthy People
Healthy People 2020/2030 provides a set of broad population level goals and objectives broken down
into five categories: health conditions, health behaviors, populations, settings and systems, and social
determinants of health (HHS, Office of Disease Prevention and Health Promotion, n.d. ). These goals
and objectives may also be concepts for population health measure developers to consider. Healthy
People 2020 also addresses SDOH and includes a list of examples of social determinants and links to
other federal SDOH initiatives and resources. Healthy People 2030 identifies five SDOH domains each
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 7
with multiple objectives: economic stability, education access and quality, healthcare access and
quality, neighborhood and built environment, and social and community context.
3.1.1.4 Existing Population Health Measures
Measure developers should look for existing measures currently identified as population health
measures, but these may not meet the CMS definition. In 2015, IOM identified several existing
measures that are indicators for population health, for example, life expectancy, overweight and obesity
rates, and teenage pregnancy rates. The usual databases of measures, e.g., CMS Measures Inventory
Tool (CMIT) and Quality Positioning System (QPS) , have historically not included these types of
measures. CMIT and QPS can also help identify existing measures that are proxies for population health,
e.g., communication between measured entities and patients for patient-centered care.
A recent scan focusing on existing clinician-level measures applicable to population health identified 248
measures. The reviewers then categorized the measures according to level of analysis using
clinician/clinician group, facility, health plan, integrated delivery system, and population. Some
measures applied to more than one level of analysis. Table 1 shows the number of measures identified
for each conceptual topic and the number of measures with population as the level of analysis (CMS,
Health Services Advisory Group, 2020). For brevity, Table 1 omits numbers for other levels of analysis.
Table 1. Existing Quality Measures at Population Level of Analysis
Conceptual Topic
Number of Quality Measures
Identified
Number of quality measures with
population as level of analysis
Access to care
21
3
Clinical outcomes
108
19
Coordination of care and
community services
39
13
Health behaviors
26
5
Preventive care and screening
50
5
Utilization of health services
4
0
3.1.2 Stakeholder Engagement
Given the broad nature of population health measures, it is critical to include community members and
organizations early in the development process. These would include local community organizations and
local governments that address social needs. Community organizations, e.g., soup kitchens and
homeless shelters, can provide important input as to the gaps in population health, the drivers of
improvement, and the benefits of improvement. Measure developers should adhere to the latest Person
and Family Engagement (PFE) toolkit at the earliest stages of developing population health measures.
3.1.3 Public Comment
Soliciting public comments for population health measures should occur frequently throughout the
Measure Lifecycle. The timing in which the measure developer solicits comments in the Measure
Lifecycle may be different than other quality measures. The measure developer may want to target
representatives of the populations and/or communities for measurement. These groups are not the
usual readers and responders to the Federal Register and websites requesting comments from the
public, so concerted targeting may be necessary.
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 8
3.1.4 Technical Expert Panel (TEP)
The composition of a TEP for a population health measure may vary from other measure development
TEPs. The TEP should include representatives of the proposed population, group, and/or community for
measurement. Again, as with public comment, concerted outreach may be necessary.
3.2 MEASURE SPECIFICATION
The general processes for specifying population health measures are no different than other types of
measures. There are, however, some specification building blocks that need closer consideration. The
measure developer must distinguish population health measures from clinical quality measures.
3.2.1 Target/Initial population
3.2.1.1 Population level specification geography-based
The geographic level of specification may include zip code, county, city, state, national, or other
geographically-based areas.
Example all adults, 18 years and older, living in zip code 20500
3.2.1.2 Population level specification patient panel-based
Denominator specifications may be beneficial for use in assessing population health improvement based
on the work of healthcare delivery systems. When specifying at a patient panel level, it is critical to avoid
limiting the measure denominator to only patients who receive specific services. In other words, the
denominator should include all patients in the patient panel without regard to particular services
rendered or patient encounters that occur. Otherwise, the measures no longer address the health of the
entire patient panel population, but rather only those that receive certain services. As such, they
become clinical quality (measure entity-focused) measures assessing quality of the services rendered
rather than an assessment of the health of the population.
Example all Medicare beneficiaries in Acme ACO
3.2.2 Stratification
Most population health measures will need a stratification plan. As noted by IOM (2015 ), factors
outside of healthcare substantially shape the health of populations, e.g., social, environmental,
individual behaviors. Stratification is necessary to provide actionable information to measured entities
and policymakers, beyond the collection and sharing of data. Stratifying the data by race, ethnicity,
language preference, sexual orientation, gender identity, age, disability, and SDOH, including those
related to education and literacy, social and community context, economic stability, and neighborhood
and built environment, can enable focused quality improvement activities.
As CMS moves toward population-based payments and shared risk, it is increasingly important for
measured entities and payers to be able to quantify and address differences and disparities among the
communities and populations served by CMS programs. Stratified data is the critical first step to
improving the health of all individuals and families.
As part of the Reducing Provider and Patient Burden proposed rule released December 10, 2020, CMS
included a request for information on barriers to adopting standards, and opportunities to accelerate
adoption of standards, related to social risk factors. CMS acknowledges that healthcare “providers in
value-based payment arrangements rely on comprehensive, high-quality data to identify opportunities
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 9
to improve patient care and drive value.” The goal is to standardize and liberate these data for multiple
reasons, such as to decrease patient reporting burden and increase the chances of connecting patients
with appropriate community care and support.
3.2.3 Data Sources
As with clinical quality measures, measure developers must specify data sources for population health
measures. Data sources may include clinical data (electronic health records [EHRs], registries, paper
patient records), claims, surveys, patient assessments (e.g., Minimum Data Set), screening tools (e.g.,
the Accountable Health Communities Health-related Social Needs screening tool [Billioux, Verlander,
Anthony, & Alley, 2017 ]), and administrative data that can include census data, crime data, birth and
death records, etc. The Gravity Project is working to change inconsistencies in existing SDOH data
collection processes. The Gravity Project "convenes stakeholders across the country through an open
and transparent collaborative process where they develop and test consensus-based standards to
facilitate SDOH data capture and exchange across a variety of systems and settings of care and social
services” (The Gravity Project, n.d. ).
However, medical record and other clinical data are unable to describe population health for the total
population group, and therefore, are insufficient. For example, although the measure developer may
consider immunization status a measure of population health, if a measure denominator includes solely
patients who receive certain clinical services or have documented patient encounters, the measure
becomes one of clinical care quality. Immunization status for a population must consider the population
as a whole. As such, public health data sources, such as those provided by CDC, or other repurposed
data, e.g., crime statistics, number of grocery stores, community health assessments, and community
health needs assessments, may be more valuable than the typical patient care data used for clinical
quality measures. CDC’s National Center for Health Statistics administers and/or collects data from
multiple surveys, e.g., National Health and Nutrition Examination Survey (NHANES) and the National
Health Interview Survey, and vital statistics which can provide population-level data for comparison. In
the absence of appropriate data sources, survey development and implementation may be necessary.
3.2.4 Level of Analysis
The level of analysis for population health measures should be at the population level and not limited
solely to patients who receive particular services. The measure developer must clearly define the
population in the target/initial population.
3.2.5 Time Interval
What is the appropriate time interval for population health measures? Is one year long enough or do
these measures require a longer time interval to determine significant changes? Measure developers
should consider the appropriate time interval for measurement, bearing in mind a longer period of time
may be necessary to identify significant changes at the population level.
3.2.6 Risk Adjustment
Outcome measures typically require risk adjustment where the purpose is to assess clinical quality.
The intent of population health measures is to produce true values without adjustment. However,
measure developers should consider risk adjustment.
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 10
3.3 MEASURE TESTING
Measure testing may be challenging due to the potential use of multiple data sources in a single
measure and a lack of data, especially SDOH data. Lack of interoperability among data sources is likely.
Consider the (in)completeness of data sources and data elements (e.g., incomplete SDOH data). The
measure developer needs to be creative with their testing plan and should partner with a variety of
stakeholders including data owners.
For the purposes of testing measures of population health, the nature of the quality construct
(inferences about underlying processes or structures [Messick, 1987 ]) determines the testing
approach (Table 2). The first consideration is which system owns the quality construct. Until recently,
the distinction in attribution was more well defined. Hospitals and physicians provided clinical care;
public health and social service agencies did not provide clinical services or they were very limited. With
the increased emphasis on the importance of social risk factors, healthcare systems are more directly
involved in addressing these social risk factors through the direct provision of or referral to services like
housing or food security, and/or are formalizing collaborations with entities outside the healthcare
system. For the purposes of defining the quality construct for measure testing, the measure developer
needs to expand attribution for traditional measured entities to include the delivery of public health and
social services, and/or to expand the measured entity to include both healthcare and non-healthcare
providers.
Table 2. The nature of the quality construct for population health measures
System owning the quality construct
The quality construct is a public
good
Healthcare
Population health
Non-healthcare
Public health
Assuming the quality construct is not a public good, once the measure developer defines the quality
construct, then measure testing would proceed as with any quality measure with the focus on
importance, scientific acceptability (reliability and validity), feasibility, and usability. A
public good is one for which consumption of the good by one person does not preclude consumption of
the same good by another person (e.g., a city park, clean air).
Measure validation is critical. If the intervention is a public good, then the validation should
demonstrate that. If the focus is on allocative efficiency,
3
then there is no need for validation. If the
focus is on some population characteristic like “cohesion,” then the focus is on that construct. The
reliability focus would be whether there is detectable variation in the quality construct across measured
entities (signal), or whether there is overwhelming variation in factors independent of the quality
construct (noise). Because there is an extended pathway between, for example, food (in)security-to-
clinical care-to-outcome, one might surmise a greater influence of factors independent (e.g.,
transportation availability) of the quality construct, thereby reducing reliability. The measure developer
might need to consider strategies to increase reliability, such as increasing the effective sample size,
e.g., borrowing statistical strength of related process or outcome measures (National Quality
3
Allocative efficiency is when the right share of resources is being devoted to healthcare versus other goods in the economy (Baicker & Chandra,
2011 ).
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 11
Forum, 2015 ) or enhancing the information context (e.g., incorporating structural measures in
reliability adjustment
4
).
The second consideration is whether the quality construct is a public good. For example, testing for lead
poisoning and removing lead from city water pipes are both interventions that might improve adverse
outcomes. The second intervention is a public good, the first is not. In the testing for lead poisoning
intervention, the evidence for validity is the same as any quality measure. There must be a person-level
process/outcome relationship established with rigorous evidence and there must be an entity-level
(e.g., ACO) demonstration of validity of the quality construct which shows that there is alignment
between the behavioral response of persons and clinicians to the measure with the end user intent.
However, in the second public good intervention there is no person-level process/outcome relationship
(e.g., an individual’s health might improve whether that individual consumes the good or not). The
effectiveness of the intervention is only determined by examining population level outcomes. Measure
testing might be cross-sectional (e.g., geopolitical areas with and without the intervention that are
similar in other measurable aspects) or temporal (e.g., the same geopolitical area before and after
implementation of the intervention). The focus on measure testing is more about characterizing the
attributes of the population and attempting to demonstrate that the attribute that matters for variation
in population level outcomes is the intervention of interest: removing lead from city water pipes. Testing
might focus on statistical significance of the assertion about the attribute, but reliability conceptualized
as to whether there is detectable variation in the quality construct across measured entities is not
relevant (i.e., there is no within and between variation).
A final testing consideration is the nature of the population level outcome. In productive efficiency,
5
the
focus is on maximizing the individual’s outcome for a given amount of healthcare or social services. A
population level outcome might be like those used by OECD such as cancer deaths per 1,000 persons. An
intervention to improve the population level outcome would be to improve maternal mortality for
individuals. In allocative efficiency, the focus is on maximizing the outcomes for a population by
allocating or distributing a given amount of healthcare or social services to the best marginal use. A
population level outcome might examine the distribution of outcomes across population subgroups and
consider whether increasing access to healthcare or social services for certain subgroups would have the
largest impact on outcomes. Testing would focus on demonstrating the hypothesis by, for example,
demonstrating that the geopolitical areas where those subgroups have better access have better
outcomes.
3.4 MEASURE IMPLEMENTATION
Because population health measures are not setting specific, their adoption would primarily be into CMS
programs such as the Medicare Shared Savings Program (MSSP), Marketplace Quality Rating System,
and Medicare Advantage program. CMS is proposing to use population health measures in Merit-Based
Incentive Payment System Medicare Value Pathways. Commercial insurers could adopt population
health measures for continuous quality improvement and serve as a comparison with other commercial
insurers. Communities could adopt population health measures to assess the success of pertinent
4
Although risk adjustment considers differences in patient disease severity and case mix, reliability adjustment allows for repeatability of
estimates related to the relative number of cases and outcomes used to calculate the indicator of interest (Wakeam & Hyder, 2016 ).
5
Productive efficiency is when health care resources are put to the best use possible and produce as much health as they can
(Baicker & Chandra, 2011 ).
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 12
implemented community programs. Measure results could also serve as input to community resource
and intervention planning and impact policy decisions at the local, state, and national levels.
The ideal plan is to use population health measures broadly, report them at the community level, and
share results among participating clinicians, public health, community, and other organizations.
Consistent with the notion of a “shared measurement system,” this arrangement increases the
likelihood that the multiple stakeholders coalesce around addressing the measure concept.
3.5 MEASURE USE, CONTINUING EVALUATION, AND MAINTENANCE
Population health measures are subject to the same three types of measure maintenance reviews as
other types of measures annual, triennial comprehensive, and early maintenance using the measure
evaluation criteria outlined in the CMS MMS Blueprint .
4 KEY POINTS
Most current population health measures summarize population outcomes at a geographic level.
Generally, these encompass health outcomes based on mortality or life expectancy, and survey-based
measures of subjective health status, psychological state, or ability to function (Parish, 2010 ).
Measure developers may need to expand their stakeholder outreach, e.g., community organizations.
Multiple data sources may be necessary to include non-healthcare sources.
There is much more to learn about population health and population health measurement. While the
principle of rigor in measure development remains the same, learning the details will come by doing and
iterating.
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 13
REFERENCES
Baicker, K., & Chandra, A. (2011, September 9). Aspirin, angioplasty, and proton beam therapy: The
economics of smarter health care spending.
https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.662.4232&rep=rep1&type=pdf
Billioux, A., Verlander, K., Anthony, S., & Alley, D. (2017). Standardized screening for health-related social
needs in clinical settings. The accountable health communities screening tool. National Academy
of Medicine Perspectives, 1-9. https://nam.edu/wpcontent/uploads/2017/05/Standardized-
Screening-for-Health-Related-Social-Needs-in-Clinical-Settings.pdf
Centers for Disease Control and Prevention. (n.d.). National Center for Health Statistics. Retrieved
October 28, 2020, from https://www.cdc.gov/nchs/index.htm
Centers for Medicare and Medicaid Services. (n.d.). PACE. Retrieved December 17, 2020, from
https://www.medicare.gov/your-medicare-costs/get-help-paying-costs/pace
Centers for Medicare & Medicaid Services. (2020). Person and family engagement (PFE) toolkit.
Retrieved October 28, 2020, from https://www.cms.gov/Medicare/Quality-Initiatives-Patient-
Assessment-Instruments/MMS/Downloads/Person-and-Family-Engagemen.pdf
Centers for Medicare & Medicaid Services, Health Services Advisory Group. (2020). 2020 CMS quality
measure development plan environmental scan and gap analysis report.
https://www.cms.gov/Medicare/Quality-Payment-Program/Measure-Development/Measure-
development.html
Doty, M. M., Tikkanen, R. S., FitzGerald, M., Fields, K., & Williams, R. D. (2020). Income-related inequality
in affordability and access to primary care in eleven high-income countries. Health Affairs.
Advance online publication. https://doi.org/10.1377/hlthaff.2020.01566
Goldstein, N. D., Palumbo, A. J., Bellamy, S. L., Purtle, J., & Locke, R. (2020). State and local government
expenditures and infant mortality in the United States. Pediatrics, 146(4). Advance online
publication. https://doi.org/10.1542/peds.2020-1134
Green, K., & Zook, M. (2019, October 29). When talking about social determinants, precision matters.
Health Affairs. https://www.healthaffairs.org/do/10.1377/hblog20191025.776011/full/
Health Level Seven International. (n.d.). The Gravity Project. Retrieved December 18, 2020, from
https://confluence.hl7.org/display/GRAV/The+Gravity+Project
Institute of Medicine. (2013a). Core measurement needs for better care, better health, and lower costs:
Counting what counts: Workshop summary. The National Academies Press.
https://doi.org/10.17226.18333
Institute of Medicine. (2013b). Toward quality measures for population health and the leading health
indicators. The National Academies Press. https://doi.org/10.17226/18339
Institute of Medicine. (2015). Vital signs. Core metrics for health and health care progress. The National
Academies Press. https://doi.org/10.1726/19402
Internal Revenue Service. (n.d.). Community health needs assessment for charitable hospital
organizations section 501(r)(3). Retrieved September 23, 2020, from
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 14
https://www.irs.gov/charities-non-profits/community-health-needs-assessment-for-charitable-
hospital-organizations-section-501r3
Kindig, D., & Stoddard, G. (2003). What is population health? American Journal of Public Health, 93, 380-
383. https://doi.org/10.2105/AJPH.93.3.380
Mays, G. P., & Scutchfield, F. D. (2010). Improving public health system performance through
multiorganizational partnerships. Preventing Chronic Disease, 7(6),
A116. http://www.cdc.gov/pcd/issues/2010/nov/10_0088.htm
Messick, S. (1987). Validity. Educational Testing Service.
https://onlinelibrary.wiley.com/doi/pdf/10.1002/j.2330-8516.1987.tb00244.x
National Quality Forum. (2015, September). Performance measurement for rural low-volume providers.
https://www.qualityforum.org/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=80442
National Quality Forum. (2019, March). NQF report of 2018 activities to Congress and the Secretary of
the Department of Health and Human Services. National Quality Forum.
https://www.qualityforum.org/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=89478
OECD. (2019), Health at a glance 2019: OECD indicators. OECD Publishing.
https://doi.org/10.1787/4dd50c09-en
Parish, R. G. (2010). Measuring population health outcomes, Preventing Chronic Disease. Public Health
Research, Practice, and Policy, 7(4), A71. www.cdc.gov/pcd/issues/2010/jul/10_0005.htm
Pennel, C. L., McLeroy, K. R., Burdine, J. N., Matarrita-Cascante, D., & Wang, J. (2016). Community health
needs assessment: Potential for population health improvement. Population Health
Management, 19(3), 178-186. https://doi.org/10.1089/pop.2015.0075
Stoto, M. (2014). Applying performance measure concepts in population health settings. eGEMS
(Generating Evidence & Methods to Improve Patient Outcomes), 2(4), 6.
http://doi.org/10.13063/2327-9214.1132
Tikkanen, R., & Abrams, M.K., (2020, January). U.S. health care from a global perspective, 2019: Higher
spending, worse outcomes? Commonwealth Fund.
https://www.commonwealthfund.org/sites/default/files/2020-
01/Tikkanen_US_hlt_care_global_perspective_2019_OECD_db_v2.pdf
Tompkins, C., Higgins, A., Perloff, J., & Veselovskiy, G. (2013, April 2). Population health management in
Medicare Advantage. Health Affairs.
https://www.healthaffairs.org/do/10.1377/hblog20130402.029363/full/
U.S. Department of Health and Human Services. (2017). Letter to the nation’s governors: Secretary Price
and CMS Administrator Verma take first joint action: Affirm partnership of HHS, CMS, and States
to Improve Medicaid Program. https://www.hhs.gov/about/news/2017/03/14/secretary-price-
and-cms-administrator-verma-take-first-joint-action.html
U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion.
(n.d.-a). Healthy People 2020. Social determinants of health. Retrieved December 17, 2020, from
https://www.healthypeople.gov/2020/topics-objectives/topic/social-determinants-of-health
Supplemental Material to the CMS MMS Blueprint Population Health Measures
September 2021 Page 15
U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion.
(n.d.-b). Healthy people 2030. Retrieved September 22, 2020, from
https://health.gov/healthypeople
U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion.
(n.d.-c). Healthy People 2030. Social determinants of health. Retrieved February 9, 2021, from
https://health.gov/healthypeople/objectives-and-data/social-determinants-health
U.S. Department of Health and Human Services, Office of the National Coordinator for Health
Information Technology. (2020, October). 2020-2025 Federal Health IT Strategic Plan.
https://www.healthit.gov/sites/default/files/page/2020-
10/Federal%20Health%20IT%20Strategic%20Plan_2020_2025.pdf
Wakam, E., & Hyder, J. A. (2016). Reliability of reliability adjustment for quality improvement and value-
based payment. Anesthesiology, 124, 16-18. https://doi.org/10.1097/ALN.0000000000000845
Woulfe, J., Oliver, T. R., Zahner, S. J., & Siemering, K. Q. (2010). Multisector partnerships in population
health improvement. Preventing Chronic Disease, 7(6), A119.
http://www.cdc.gov/pcd/issues/2010/nov/10_0104.htm