Racial bias in mortality prediction scores

Mortality prediction models alone could divert scarce critical care resources away from Black patients.

The COVID-19 pandemic has forced medical professionals to face uncomfortable realities. For the last year, critical care physicians have been working in intensive care units (ICUs) that have grown to accommodate large numbers of patients who have become seriously ill from COVID-19. The issue of scarcity looms—hospital staff worry whether there are not enough ICU beds or ventilators for all the patients who need them. In such a situation, how will hospitals fairly choose who gets access to these scarce resources, knowing that the patients who are turned away may die?

African American COVID patient wearing a respirator in a hospital bed.

Ethicists have developed crisis standards of care (CSCs) to answer exactly this question during pandemics like COVID-19 or other health crises like natural disasters or mass casualty situations. CSCs generally promote two main ethical goals—to save the most lives, and to do so fairly. To achieve the first goal, CSCs suggest prioritizing patients with the best chances of survival to receive scarce resources. Most CSCs operationalize this by using a mortality prediction model, the Sequential Organ Failure Assessment (SOFA) score, to rank patients based on their predicted probability of surviving their hospital stay. However, whether using the SOFA score helps achieve the second goal—allocate resources fairly—is not known. Therefore, against the backdrop of a pandemic that has exposed the structural disadvantages experienced by racial and ethnic minorities, Penn Medicine researchers set out to determine whether the SOFA score is equally good at predicting mortality among Black and white patients.

The research team found that the SOFA score is “miscalibrated” and racially biased. Specifically, it overestimates mortality among Black patients and underestimates mortality among white patients. In other words, using the SOFA score would lead physicians to believe that Black patients are sicker or more likely to die than they actually are. This matters because, within the framework of CSCs, patients who are most likely to die are last in line to receive scarce resources. Therefore, using the SOFA score could systematically and unfairly divert scarce critical care resources away from Black patients. In a rudimentary simulation, the researchers estimated how many Black patients would be affected by this miscalibration, and found that 81% of Black patients from lower priority CSC categories, and 9% of all Black patients, would be improperly excluded from the highest priority CSC category. In the paper, researchers also test the accuracy of other scores, and share general principles to consider when choosing a mortality prediction model to include in CSCs.

This story is by Deepshikha Charan Ashana. Read more at Penn LDI.