How to leverage your recently released CAHPS® survey results for maximum impact

2023 Stars is around the corner. Here's an action plan to design your CAHPS® improvement strategy.

In recent months, I’ve urged plans to take action to improve the Patient Experience Measures (comprised of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) (except flu), disenrollment, appeals, call center, and complaint measures) that will be more heavily weighted in the 2023 Medicare Part C & D Star ratings.

Now that the Centers for Medicare & Medicaid Services (CMS) has moved two 3x-weighted HOS measures to display for 2022 and 2023, that urgency has become an existential imperative for many plans, as the patient experience measures (and CAHPS in particular) eat up an even greater slice of the Star ratings pie.

Your CAHPS Scores are in … now what?

Earlier this month, Medicare Advantage (MA) plans received their 2021 CAHPS survey results. Instead of simply sharing the results throughout your organization and promising to do better next year, seize this moment to dive deep into your data to uncover actionable insights, identify opportunities to improve your member experience and precisely target where to direct your resources (people, time, and investment) for the rest of this coverage year and into Q1 of next year. Each measure could be approached from a variety of different directions, so it’s important to figure out which aspects of the member experience need improvement and, of those, which ones your plan is actually capable of tackling. Begin by evolving the way you think about your CAHPS survey results, from viewing them as a discreet set of stand-alone data to scrutinizing the results in combination with other forms of member feedback.

Evaluate your plan’s results against one or more benchmarks

Not all benchmarks are created equal. Choose benchmarks with your plan’s long-term strategy and improvement goals in mind. Here are some good sources of comparative benchmarks* to consider:

• CAHPS Health Plan Survey Database contains results for the Medicaid survey submitted by various sponsors, including public employers, State Medicaid agencies, State Children's Health Insurance Programs (CHIP), and individual health plans, as well as Medicare Survey results provided by CMS. To access this data, visit https://datatools.ahrq.gov/cahps
• NCQA’s Quality Compass® presents quality improvement and benchmark plan performance through online access to health plan HEDIS® and CAHPS performance data. Plans can create custom reports for up to three trended years, comparing measures and benchmarks (averages and percentiles). To access this data visit, https://store.ncqa.org/data-and-reports.html
• Comparative Data from CMS’ Health Plan Survey includes scores calculated from responses to the MA and fee-for-service (FFS) survey versions and reports state means using both the original survey scale and the 0-100 scale. This source also includes detailed current and historic response rate information for MA and PDP CAHPS, and overall rates for MA, PDP, and FFS CAHPS. To access this data visit, https://www.ma-pdpcahps.org/en/comparative-data/
• Your survey vendor: Most vendors will provide comparative data for their book of business.

* Keep in mind that in 2020, CMS did not collect Medicare CAHPS data due to the COVID-19 pandemic and therefore, results for 2020 are not presented in many of the above data sources.

Understanding the various ways to analyze your data

There’s a myriad of ways to dissect your CAHPS results that make use of benchmarks as reference points from which to view your plan’s performance. The key is choosing the methods that are the most applicable with the understanding that not all comparative data will be relevant to your unique organization. For example, comparing your plan’s performance to regional or national averages may not be as meaningful as comparing yourself to the average scores for similar organizations. And, if you need to make significant leaps, comparing yourself to the top performers (looking at top box scores) may be a more effective way to set the bar than comparing yourself to the averages.

• Comparing averages: Compare your plan’s mean scores for composite and rating measures with the average mean score for other plans (nationally or by state). See what trends you can identify. Check out which individual measures are dragging down your composite scores and how your performance compares to that of other plans.
• Comparing top box scores: The percent of respondents that chose the best possible response option (i.e., yes, definitely or always) is referred to as a “top box” score. You can compare your plan’s top box score with that of other plans, again paying close attention to composite measures and the individual measures that make up the composite.
• Comparing percentile scores: The distribution of scores across all health plans included in a benchmark are referred to as percentile scores. For this calculation, scores for all participating plans are ranked from low to high. The percentile (i.e., 90^th percentile, 25^th percentile) indicates the percentage of organizations that scored at or below a particular survey score. To understand where your score falls, find the highest percentile where your score is higher than the percentile score. In the example below, the plan’s score in the composite measure “Getting Needed Care” is 61, which is higher than the score for the 75^th. That means the plan scored higher than the 75 percent of plans in this measure. The plan’s score for “Easy to get necessary care, tests or treatment” was 68, better than 90 percent of plans. However, the plan’s score for “Got appointment with specialist as soon as needed” is 56, better than only 50 percent of plans, suggesting that this particular measure is driving down the composite measure score. Doing this type of analysis allows you to more precisely pinpoint where to find opportunities for improvement.

• Comparing your current performance to past performance: When available, comparing your most recent performance to your past performance can be a useful tool to determine whether interventions are working or whether you need to make changes to your strategy or tactics. Use this type of analysis to find trends in top box scores as well as comparing your mean scores.
• Figuring out what’s most important to your members: Understanding what issues are important to your members will help you tactically choose the areas to focus on for improvement. This type of analysis, referred to as “key driver” analysis, entails examining the correlation between a particular individual question or composite measure and the member’s overall rating of the health plan. The stronger the association, the higher the importance.

Prioritizing measures to work on

Once you understand your performance and what’s important to your members, you can create a priority list of measures by placing measures of low performance and high member importance higher on the list. It might be helpful to create a visual representation by organizing all Star rating CAHPS measures within a priority matrix such as the one below.

Putting it all together

Once you’ve prioritized the measures you want to improve upon, it’s time to figure out what actions you can take.  Get curious about what qualitative info you can use to better inform what you’re seeing in the quantitative data. For example, if you notice that the measure “Got Appointment with specialists as soon as needed” is dragging down your “Getting Needed Care” composite measure, look at other sources of information, such as your call logs, grievance and appeals data, and provider accessibility studies to see if the issue is popping up in any of those places as well. What are members and providers saying? Are you seeing grievances related to this issue? How about appeals related to requests for out-of-network coverage? Then, brainstorm potential reasons why members may be having trouble getting appointments when they need them. If the answers aren’t immediately obvious, dive deeper to get at the inherent causes of performance issues by trying one or more of these proven methods for problem solving: 

• Root cause analysis
• Process mapping
• Process observation
• Walkthroughs
• Small-scale surveys

The results of the above efforts should help you understand what you can fix right away and what will take more time and effort. That awareness should shape your strategy for approaching the specific measures you want to tackle with improvement activities.

Finally, zero in on what specific improvement activities you will ultimately focus your time, money, and energy on by considering the prevalence of the issue you’re hoping to correct, how far away your score is from others (how big is the opportunity for improvement), your existing improvement projects, and where these improvement activities fit within the framework of other plan strategies, priorities, and initiatives.

As always, if you’re charged with championing these measures at your health plan, I remind you to not feel daunted by the work ahead. Each of the steps above takes a concerted effort and time, but none of it is rocket science. Get started as soon as possible, and in the words of Franklin D. Roosevelt, “above all, try something.”  Take the first step and get help if you need it. 2023 Stars is around the corner. Designing your CAHPS improvement strategy is too important to wait another day. Your investment of time, money, and resources in these critical Star ratings measures, if leveraged smartly, will pay off in spades.

About the author

Ana Handshuh, principal at CAT5 Strategies, is a government programs executive with expertise in creating and implementing corporate programs for the health care industry. Her background includes quality, core measures, care management, benefit design and bid submission, accreditation, regulatory compliance, revenue management, communications, community-based care management programs and technology integration. Handshuh currently serves on the board of the Resource Initiative and Society for Education (RISE) and is the chair of RISE's Quality & Revenue Community. She is a sought after speaker on the national health care circuit in the areas of quality, Star ratings, care management, member and provider engagement, and revenue management. Her recent consultancy roles have included assisting organizations create programs to address the unmet care management needs in the highest risk strata of membership, document their processes and procedures, achieve accreditation status, design and submit government program bids, institute corporate-wide programs and create communications strategies and materials.