Eliminating the Fee-for-Service Adjuster from the Risk Adjustment Data Validation methodology would likely have significant implications for plan payment and could change plan incentives and behavior, including plans’ willingness to assume the risk of participating in the program, writes Sean Creighton, managing director of Avalere, who also serves as a RISE board member and the chair of the RISE Risk Adjustment Policy Advisory Committee.
Medicare Advantage (MA) is a growing program accounting for 22 million beneficiaries in 2019, or nearly one in every three enrollees in Medicare. This program has achieved significant growth and stability during the past decade, despite payment reductions under the Affordable Care Act. CMS is now considering a revised methodology for calculating payment to MA plans that could put the stability of the program at risk. Specifically, the proposal would change a key component of the Risk Adjustment Data Validation (RADV) methodology called the Fee-for-Service (FFS) Adjuster and, if finalized, would introduce inaccuracy into CMS’ payment to plans.
CMS uses a risk adjustment model, called the CMS Hierarchical Condition Category (HCC) model (CMS-HCC model), to pay plans appropriately for the health status of their enrollees. The CMS-HCC model calculates a risk score for each individual based on an individual’s demographics and disease profile. The higher the risk score, the sicker the individual. In general, diagnosis codes are assigned to HCCs (e.g., diabetes, COPD, CHF), and each HCC has a coefficient that represents that disease’s incremental contribution to overall costs. CMS converts these coefficients into relative factors through a process called “normalization” by dividing them by the average expected costs for the population. For example, if the coefficient is $3,000 for diabetes, and average costs are $10,000, then the relative factor is 0.3 ($3,000 divided by $10,000).
The CMS-HCC model is based on diagnoses and program costs for individuals in Medicare FFS. In their proposed new methodology, CMS does not account for the difference between how this model is developed—in which coding errors are allowed and included—and how money is recovered under a RADV audit—in which each code is held to a 100 percent accuracy standard. The difference in documentation standard introduces an actuarial challenge that the “FFS Adjuster” was intended to address. The FFS Adjuster accounts for the difference in documentation standards between the FFS data used to calibrate the risk adjustment model and the RADV standard.[1] Because FFS claims data are not audited to determine if the diagnoses are supported by a medical record, there are undocumented errors in FFS data as research shows.[2] Absent an adjuster to account for such errors, RADV audits would be contrary to the statutory requirement, § 1395w-23(a)(1 )(C)(i) of Title 42 of the United States Code, which states that risk adjustment must achieve actuarial equivalence between FFS and MA.[3]
The revised payment methodology would remove the FFS Adjuster. CMS has proposed that the FFS Adjuster is not warranted based on a study that we have previously critiqued. CMS suggests it assessed the difference in risk scores that would occur from a model based on unaudited FFS data (the current model) versus a model estimated on audited data (the corrected model). CMS’ methodology is inconsistent with standard risk adjustment methodology, relies on assumptions that are not reflective of actual diagnosis coding error rates or of the underlying claims distribution in the FFS and MA populations, and appears to be designed to minimize differences in payment under the two models.
In particular, this blog focuses on two methodological decisions in the CMS study that undermine its validity and should be considered when evaluating the final recommendation:
- CMS minimized the error rates used in its study by calculating them using assumptions that do not accurately reflect either coding practices or patients’ claims experience
- CMS applied an adjustment, which it calls the Inflated Post-Audit Risk Score (IPARS) adjustment, that eliminates any difference between the model calibrated on corrected data versus the model calibrated based on uncorrected data
The design of CMS’ study results in estimates of the rates of error in FFS coding that are much smaller than in practice, and then adjusts the results to altogether remove the impact of the errors.
This methodology raises series questions about the ability to make accurate policy decisions based on the findings of the study. Public comments were due August 28, and multiple stakeholders—including leading risk adjustment actuarial experts in the field—raised significant concerns about CMS' methodology. [4]
Methodology for assessing impact of FFS Adjuster detailed in recent study
In its FFS Adjuster study, CMS measured the error rates in FFS data and applied these error rates to a new model, which it called the ‘corrected’ model. As a first step, CMS determined errors in the FFS data by reviewing 8,630 outpatient claims from 2008. For example, 488 claims in this data set had COPD (HCC 108), and 96 were not supported, for an error rate per COPD claim of 19.8 percent. In other words, about one in every five times COPD was coded, it was in error.
CMS then used the claim-level error rates for each HCC to create three error rate groups: low (20.9 percent), medium (33.8 percent), and high (46.2 percent). Each HCC was assigned to one of these three groups, where assignment was based on that HCC’s error rate’s relationship to the overall error rate of 33.9 percent. For example, COPD, which had an error rate of 19.8 percent, was assigned to the low category.
CMS next converted claim-level HCC error rates to beneficiary-level error rates, based on the average number of claims for a given HCC. In the case of COPD, it would have been placed into the “low” category, and the claim-level error rate used for COPD was 20.9 percent. The beneficiary-level error rate was then the probability that all claims for that enrollee would have that condition recorded in error. CMS assumed each claim was a distinct event because “each enrollee HCC potentially has multiple claims with independent supportive medical records.” In addition, CMS assumed that all beneficiaries with that condition had the same number of claims (specifically, the average number of claims per beneficiary triggering the HCC), regardless of the person’s actual number of claims triggering that HCC. The combination of these two assumptions led CMS to calculate beneficiary error rates by taking the claim-level error rate and raising it to a power, where the power was the average number of claims. For example, if the average number of claims triggering the COPD HCC was five, then the beneficiary-level error rate used by CMS in this study is (20.9 percent*20.9 percent* 20.9 percent* 20.9 percent* 20.9 percent), or 0.09 percent.[5]
To estimate the impact of these assumed beneficiary-level error rates, CMS then estimated versions of the CMS-HCC model using the original data and also data that incorporated a reduced set of HCCs due to identified errors. Because it is not possible to identify actual errors, CMS simulated errors by removing HCCs, randomly on an HCC-by-HCC basis, according to the beneficiary-level HCC error rates calculated according to the assumptions above. For example, if 10 percent of beneficiaries were identified as having diabetes coded in error, then diabetes was removed from 10 percent of the beneficiaries with the diabetes HCC. Because the beneficiary-HCC exclusion is a random simulation, CMS repeated this process 50 times to account for sampling error in the random simulation process.
For each simulation, CMS then applied an adjustment, called the IPARS adjustment, that modified the coefficients of the model such that the average risk score of this model would be 1.0. Using these new models and the baseline model, CMS calculated risk scores for a population of MA enrollees and found that there is little difference between these risk scores. CMS therefore concluded that the FFS Adjuster is unnecessary.
FFS Adjuster study inaccurately minimizes error rates in Medicare FFS data
In converting claim level error rates to beneficiary level error rates in the FFS data, CMS made two simplifying assumptions. In particular, CMS calculated beneficiary-level error rates by assuming that:
- Each beneficiary with an HCC had the average number of claims to support the HCC
- Each claim supporting an HCC is independent of each and every other claim for that same diagnosis
These assumptions, when used together, dramatically reduce the documentation error rate used in their study from a weighted average of approximately 14 percent to 2 percent. The assumption that each beneficiary has the average number of claims is crucial because of how it is used to convert claim level errors to HCC level error. The assumption of independence is critical because it allows CMS to treat each diagnosis code error as a distinct event.
The distribution of Medicare medical claims and costs are skewed with a small number of beneficiaries having very high claims counts and associated costs and many beneficiaries having significantly fewer claims than the average for a particular HCC. However, if the average number of claims for an HCC is 5.0, the CMS methodology assumes that each beneficiary with that HCC had five claims, even though most beneficiaries have only one or two claims triggering the HCC. Using the average number of claims in the formula means that this study dramatically overstates the number of claims for a substantial number of beneficiaries, and thereby reduces the HCC level error rate used in estimating the “corrected” model.
In March 2019, Avalere Health conducted an analysis to test CMS’ assumptions by modeling the impact of the FFS Adjuster using the actual number of claims per beneficiary. By capturing the variation in the data, Avalere’s study more closely approximates the real-world impact of the FFS Adjuster. By changing the assumption about the number of clams per beneficiary, Avalere found that eliminating the FFS Adjuster would lead to an average underpayment to plans of nearly 8 percent. Others’ actuarial analyses based on slightly varying assumptions and different years of data support an estimate of the FFS Adjuster in the same range as Avalere’s.
In a follow-up study, Avalere Health found that the impact of documentation-related miscalibration could be even greater for MA plans that enroll a significant number of beneficiaries who are dually eligible for Medicare and Medicaid (i.e., the mean underpayment would be approximately 9.1 percent for a plan with 100 percent duals) or beneficiaries with certain conditions (e.g., kidney disease or congestive heart failure).
Regarding independence of claim errors, CMS asserts that each claim supporting an HCC is independent of other claims supporting that HCC. However, coding errors will not be independent from one claim to the next, especially if the patient is seeing the same provider or if the HCC presents coding or documentation challenges. By virtue of assuming independence, CMS reduces the beneficiary level error rate. For example, if the probability of an error is 50 percent, and the person has two claims, the beneficiary level error would be 50 percent2, or 25 percent. Likewise, if the person has six claims, the beneficiary level error rate would be 50 percent6, or 1.56 percent. In the formula used by CMS to determine the beneficiary level error rate, the exponent is the average number of claims. In other words, even though a beneficiary who has one erroneous claim may be more likely to have another (for instance, if the same provider submits more than one claim for the beneficiary, or if the coding error is one that is made by many providers), CMS’ methodology assumes this beneficiary would be no more likely to have multiple error claims than any other beneficiary.
CMS’ assumption of independence is contradicted both by real world considerations and the results of the HHS National RADV studies that suggest that claim level diagnostic error translates to beneficiary error rates at a ratio of about 1.5 to 1. In the CMS FFS Adjuster study, the ratio is calculated to be approximately 7:1—in other words, the claim-level error rates used by CMS in the FFS Adjuster study translate to person-level error rates at a much lower rate than they do in CMS’ National RADV studies. The assumption of independence has the effect of reducing the person-level error rate and in combination with the assumption that each enrollee has the average claims count generates unrealistically low error rates.
The methodology includes steps that minimize the impact of the FFS Adjuster regardless of the error rate in the underlying FFS data
CMS applied the IPARS adjustment factor to the coefficients of each of the 50 model simulations, normalizing them such that the average predicted risk from each of the 50 models applied to the full national data is set equal to the actual average payment of the unaudited data. The IPARS adjustment is critical because its calculation guarantees that no differences will be found between the original model and the corrected model. Essentially, CMS determines the difference between the uncorrected and corrected models, and then divides all coefficients by that difference. If CMS had found that the difference was 10 percent, it would have divided all coefficients by 1.1 (in other words, in effect, CMS’ approach is similar to multiplying a number by 10 and then dividing by 10). Unsurprisingly, the risk scores for the corrected and uncorrected models are virtually identical because of this step.
CMS describes the IPARS adjustment as a way to correct for the effect of claim amounts associated with erroneous diagnoses have on the regression model. Under their theory, Medicare costs (the dependent variable) should be lowered to account for claims associated erroneous diagnoses. CMS did not actually test this argument, which seems inconsistent with Part B payment policy, under which providers are paid based on the services they provide, not the diagnosis submitted on the claim. The implementation of the IPARS amounts to a complete negation of the findings of the study because CMS normalized the coefficients resulting from audited data by using the mean expenditure of the uncorrected dataset.
The IPARS formula produces a factor that, when applied, reduces the risk score produced from the audited model on average to be equal to the risk score from the unaudited model. With such an adjustment in place the revised model will never show a difference from the original model. Indeed, as a result of the IPARS adjustment, even extraordinarily high FFS error rates would be effectively negated and would have no impact on the models CMS used for its study.
The impact of removing the FFS Adjuster warrants further study before CMS moves forward with its proposed approach
Eliminating the FFS Adjuster would likely have significant implications for plan payment and could change plan incentives and behavior, including plans’ willingness to assume the risk of participating in the program. Implementing RADV in the absence of a FFS Adjuster may undermine the actuarial certification of the plan bids on which CMS bases payment to health plans. In addition, the proposed change is retrospective and changes MA payment methodology without advance notice as required by law, which prevents plans from adjusting their bids to account for CMS’ proposed change in policy. Stakeholders should consider asking CMS to conduct and publicly release additional studies in response to critiques before finalizing any proposal to change stated policy.
RELATED ARTICLE: CMS proposed rule: Big changes to RADV audits could lead to hefty penalties for Medicare Advantage plans
[1] Per CMS’ memorandum: “The FFS Adjuster accounts for the fact that the documentation standard used in RADV audits to determine a contract’s payment error (medical records) is different from the documentation standard used to develop the Part C risk-adjustment model (FFS claims).”
[2] The Centers for Medicare & Medicaid Services (CMS) measures the Medicare Fee-for-Service (FFS) improper payment rate through the Comprehensive Error Rate Testing (CERT) program. https://www.cms.gov/Research-Statistics-Data-and-Systems/Monitoring-Programs/Medicare-FFS-Compliance-Programs/CERT/Downloads/2018MedicareFFSSuplementalImproperPaymentData.pdf
[3] Of note, this was the conclusion of the United States District Court for the District of Columbia: UnitedHealthcare Ins. Co. v. Azar, 330 F. Supp. 3d 173 (D.D.C. 2018) (Collyer, J.), appeal docketed, No. 18-5326 (D.C. Cir. Nov. 14, 2018)
[4] https://www.regulations.gov/docketBrowser?rpp=25&po=0&dct=PS&D=CMS-2018-0133&refD=CMS-2018-0133-0001
[5] Fee for Service Adjuster and Payment Recovery for Contract Level Risk Adjustment Data Validation Audits - Technical Appendix.
About the Author
Sean Creighton, Managing Director
Creighton is responsible for leading advisory services work tied to Medicare Advantage, risk adjustment, and related issues at Avalere, an Inovalon company. His extensive experience with claims data and application of Avalere’s modeling and analytics functions enable him to advise clients on their strategic goals.
Prior to Avalere, Creighton was a senior vice president at Verscend Technologies, where he led the development and management of risk adjustment products. Prior to that, he spent 15 years at the Centers for Medicare & Medicaid Services, leading the policy development and implementation of major public programs.
Creighton holds graduate degrees in sociology and statistics from the London School of Economics and Trinity College, Dublin, Ireland, and a BA from the University of Limerick, Ireland.
About Inovalon
Inovalon is a leading technology company providing cloud-based platforms empowering data-driven healthcare. Through the Inovalon ONE® Platform, Inovalon brings to the marketplace a national-scale capability to interconnect with the healthcare ecosystem, aggregate and analyze data in real-time, and empower the application of resulting insights to drive meaningful impact at the point of care. Leveraging its platform, unparalleled proprietary data sets, and industry-leading subject matter expertise, Inovalon enables better care, efficiency, and financial performance across the healthcare ecosystem. From health plans and provider organizations, to pharmaceutical, medical device, and diagnostics companies, Inovalon's unique achievement of value is delivered through the effective progression of “Turning Data into Insight, and Insight into Action®.” Supporting thousands of clients, including 24 of the top 25 U.S. health plans and 22 of the top 25 global pharma companies, Inovalon's technology platforms and analytics are informed by data pertaining to more than 964,000 physicians, 519,000 clinical facilities, 264 million Americans, and 42 billion medical events. For more information, visit www.inovalon.com.