On the Friday afternoon before the Memorial Day weekend, the Centers for Medicare & Medicaid Services (CMS) issued their Organ Procurement Organization (OPO) Public Aggregated Performance 2024 and 2025 Reports for data years 2022 and 2023, and, yes, CMS data lags two full years.
According to these reports, New England Donor Services’ performance was ranked as a Tier 2 OPO, that is, it was above average but below the top 25% of performers. This ranking stands in juxtaposition to NEDS setting annual records for organ donation and transplant in the region in 2021, 2022, 2023 and 2024, and increasing the number of organ donors by a whopping 80% between 2020 and 2024. The number of life-saving organ transplants rose by 53% in this same time period, as well. This record level of clinical activity now places NEDS as among the top 2 organ procurement organizations in the nation by donor and transplant volume. Further, NEDS has achieved the highest rate of kidney utilization in the nation for the past three years and in 2023 recovered organs from more donors than the top three ranked “Tier 1” OPOs combined.
What then accounts for the disconnect between the outstanding performance of NEDS and the CMS assigned Tier 2 ranking? Well, in short, the metrics used by the federal government to rank OPO performance are faulty measures, out of date and unvalidated.
The CMS metrics claim to rank order OPO performance by donation rate and transplant rate and use those rankings to assign each OPO to one of three tiers. OPOs below the median are placed in Tier 3. OPOs in the top 25% are assigned to Tier 1. OPOs performing above the median but below the top 25% are labelled as Tier 2.
The rankings are of critical concern because if an OPO is ranked below the median, they are automatically decertified, and they can no longer do the work of an organ recovery organization. Tier 2 OPOs are not recertified but instead forced to enter an undefined “competition” of some sort to continue their life-saving organ donation work. The stakes are very high for the patients awaiting transplant who may be left waiting longer because the system of donation in their area will be thrown into chaos and disarray if the sole OPO serving that area loses its ability to recover organs or spends its time and resources on a competitive process.
Also, because the OPOs metrics are inaccurate and do not reflect actual OPO performance, there is a high likelihood that OPOs which actually perform well will be closed by the federal government and later replaced by a recovery agency that, in fact, performs at a lower level than the previous OPO. This is evident given actual examples such as OPOs being assigned to Tier 1 despite substantially decreasing numbers of organ donors over the past several years.
There are two main reasons why the metrics do not accurately measure and rank OPO performance:
The CMS metrics are not risk adjusted.
Unlike most metrics in health care, the CMS metrics for OPO performance are not risk adjusted for the underlying sociodemographic factors within a patient population. This risk adjustment is almost universally considered when comparing healthcare providers, especially across different geographies. According to CMS’ own quality measurement guidelines, risk-adjustments – including socioeconomic and underlying health factor adjustments – should be made to metrics when they are used to compare health provider performance in order to account for differences in the patient populations health providers serve. Specifically, CMS materials state, “risk adjustment increases the likelihood of fair comparison of measured entity performance, which is to compare ’apples with apples’.” However, in the measure CMS uses to compare the performance of Organ Procurement Organization Donation Rates, there is no risk adjustment by CMS; no adjustment on socioeconomic factors known to impact authorization rates, no adjustment for whether the donation was a brain death or DCD circumstance which is known to impact organ utilization, no adjustment for cause of death; and no adjustment for co-morbidities that impact medically suitability for organ donation or for any adjustment using factors CMS metrics routinely adjusts for when comparing health providers. Ranking OPOs without adjusting for socioeconomic factors results in a ranking of OPO performance that is faulty and at odds with health care metrics best practices.
Here are four peer reviewed studies that demonstrate how OPO donation rates and rankings are impacted by underlying population characteristics that are outside an OPO’s control. The studies show that performance rates need to be risk adjusted to account for these underlying population characteristics so that only OPO performance itself is being measured.
–Population Characteristics and Organ Procurement Organization Performance Metrics
–Adjusting for race in metrics of organ procurement organization performance
CMS tier placement is biased against large OPOs.
Not only is the ranking of the nation’s OPOs inaccurate, but the method CMS uses to assign Tier placement is also incorrect and biased against OPOs with the greatest volume of potential organ donors. In fact, the way CMS assigns tiers results in OPOs with the SAME performance being assigned to DIFFERENT tiers. Specifically large OPOs with similar performance to small OPOs will disproportionately be assigned to Tier 2 while smaller OPOs of similar performance to large OPOs will be assigned to Tier 1. The CMS methodology risks decertifying many of the US donation system’s largest and highest performing OPOs. A peer reviewed study of this flawed methodology is available here.
Concern with the flawed CMS performance metric is shared by the transplant centers and surgeons of the New England region. In a letter to the Department of Health & Human Services in September 2023, leaders of all 14 transplant centers in the region expressed their collective concern that the current CMS performance measurements. The letter notes that, as the measurements currently stand, they are on course to disrupt the nation’s transplant system through the anticipated forced closure of many of the nation’s OPOs using a metric that is based on flawed data and measures the OPO on the transplant rate which is not within the OPO’s control. In the letter, transplant centers noted that CMS is “arbitrarily defining nearly half of the OPOs as ‘failing’ regardless of significant year-over-year growth in donation rates.”
There is broad agreement across the donation and transplant communities that performance metrics are necessary to drive improvement and hold institutions accountable for their outcomes. There is also broad agreement in the community that such important metrics be accurate! However, the currently flawed measurement system does not meet that goal, and it is It is time for CMS to reform and improve the regulatory metrics so that our nation’s world-leading donation and transplant system isn’t hindered by bureaucratic missteps.