Ai2 releases fairness test showing AI decisions bend unevenly under pressure

An AI model can choose the better-qualified candidate across demographic groups yet defend that choice less consistently for one group. The Allen Institute for AI introduced FairStress on October 20, 2026, to test that gap. Its researchers found that matched persuasive arguments could sway decisions differently depending on which demographic group benefited.

The benchmark is available for download, and Ai2 is building it into Olmo-eval, its open workbench for comparing model evaluations. It measures stability under pressure alongside the final choice, rather than treating a matching pair of answers as the whole fairness test.

Same qualifications, different pressure

FairStress starts with 300 two-candidate scenarios across 29 domains, including hiring, healthcare, lending, education, housing and criminal justice. Some give one candidate a clear advantage under the stated criteria. Others deliberately leave the choice ambiguous, allowing researchers to examine preferences when neither candidate is objectively stronger.

The test varies identity cues from no demographic information to names, implied identities and explicit labels. It then adds persuasive statements, which the researchers call adversarial prefixes. These invoke authority, social consensus, legal concerns or historical disadvantage. Each has a matched version supporting the other candidate, so the kind of argument stays constant while its beneficiary changes.

Those combinations produce more than 13 million test cases. For less expensive evaluation, Ai2 also created FairStress-Core, a 50,000-item subset selected using item response theory—a statistical method for choosing test items. The researchers designed it to retain much of the full benchmark’s ability to distinguish model behavior.

Which decisions survived persuasion?

Ai2 found little asymmetry when prompts contained no demographic information. Differences grew as identity became more explicit. In clear-cut cases, models were more willing to abandon the stronger candidate when switching favored a minority-coded candidate than when the demographic positions were reversed. The researchers describe this as anti-stereotypical over-correction: favoring the less advantaged identity at the expense of the stated evidence.

That judgment rests on a particular definition of fairness. FairStress targets equal treatment and equal resistance to persuasion when the evidence supports a decision. Ai2 explicitly distinguishes this from an equity-based approach, which might treat historical discrimination or underrepresentation as relevant even when other factors are held constant. The benchmark does not claim to capture every conception of fairness.

The test setup can change the verdict

A comparison with BBQ, an existing demographic-bias benchmark, complicated the picture. Ai2’s BBQ evaluation generally found stereotype-consistent behavior, unlike FairStress’s minority-favoring pattern. But the tested standard BBQ setup scored probabilities assigned to fixed answer options without generating a response. FairStress instead asked models to produce an answer and choose a candidate, closer to a chat interaction.

The researchers held BBQ questions and model weights constant while testing five configurations of chat formatting, wording and scoring. With a chat template, yes/no wording and generated answers, three models shifted from stereotype-consistent to minority-favoring behavior. A fourth moved to roughly neutral. The result shows that the direction of measured bias can depend partly on the evaluation setup, not just the model.

Ai2 argues that developers should evaluate both fixed-option scores and generated decisions. Its comparison suggests that relying on the former alone can miss over-correction in the answers users actually receive. That makes the response format part of the fairness question—not merely a technical detail of how a benchmark is run.