UT Southwestern urges model-by-model triage checks after study finds stigma bias

UT Southwestern urged testing each AI triage tool against real patient outcomes in an October 20, 2026, announcement highlighting bias in emergency-care rankings. Its researchers found that adding stigmatizing patient details could move priority away from people who later deteriorated. The warning emphasizes that bias varied by model, rather than following one uniform pattern.

The announcement highlights a study published online September 19 in npj Digital Medicine. It tested three open-weight language models—Gemma, Qwen and DeepSeek—using patient records. These were controlled comparisons, not decisions made during live AI-directed care.

Similar starting conditions, different outcomes

Researchers used emergency-department records from Stanford Medicine and Beth Israel Deaconess Medical Center. They paired patients with matching Emergency Severity Index levels, a measure of initial urgency. One patient in each pair deteriorated within six hours; the other did not. The models had to choose which patient should receive priority.

Before extra demographic, social or potentially stigmatizing information appeared, all three models identified higher-risk patients at rates well above chance. Researchers then inserted one attribute and checked whether the ranking changed. Across 221,556 comparisons and 18 conditions, they tested neutral and stigmatizing formulations of the same concepts.

Frequent visits showed the clearest wording effect

The stigmatizing frequent-visit description produced statistically significant harmful reprioritization in all six combinations of model and hospital dataset, reaching 9.4%. It produced more harmful shifts than neutral wording in every combination, with gaps of 1.4 to 7.1 percentage points.

Those percentages describe model rankings shifting away from patients who worsened—not patients harmed by an operating triage system. Other attributes showed different patterns:

• Psychiatric history produced significant harmful shifts reaching 9.5%, but the difference between stigmatizing and neutral wording was significant for only one model.
• Race, language and insurance information produced no consistent harmful shifts in this experiment.

Age also influenced some decisions, although researchers believed those effects likely reflected greater medical risks rather than age-related bias. The team’s planned next step is to develop emergency-department deterioration models that combine structured data with clinical text.