AI models penalized Chinese and Nigerian accents despite accurate transcripts, study finds

An accurate transcript did not mean a fair score. In a University of Washington-led study, four AI models generally rated Chinese- and Nigerian-accented speakers below American- and British-accented speakers in interviews, presentations and English tests. The gap reached beyond delivery: identical answers received different content scores depending on the speaker’s accent.

The university described the findings in a November 3 research release. Researchers presented the study on October 25 at the Conference on Empirical Methods in Natural Language Processing in Budapest. The experiment examined a consequential use of audio AI: judging a person’s performance, rather than simply recognizing the words they said.

Same scripts, different scores

The team tested Google’s Gemini 2.5 Pro and Gemini 2.5 Flash, OpenAI’s GPT-Audio 1.5, and Qwen 3 Omni. These models can process audio as well as other kinds of input. Senior author Yulia Tsvetkov said the researchers chose them because they represented systems companies would deploy in real-world settings.

To compare accents at scale, researchers used ElevenLabs voices to read the same scripts in American, British, Chinese, Indian and Nigerian accents. Each accent was represented by male and female voices. The scripts covered three kinds of evaluation:

• Hiring: responses to job-interview questions.
• Presentations: scripts for a class presentation and a conference presentation.
• English proficiency: responses to spoken-language test questions.

Each model scored delivery and content separately on a one-to-seven scale and supplied a brief explanation. Across the tasks, American and British accents generally ranked highest, Indian accents fell in the middle, and Chinese and Nigerian accents ranked lowest. Holding the words constant did not remove the difference, even when the models transcribed those words accurately.

Human voices widened the gap

The researchers then repeated the interview test with 52 human participants from the five countries. Each read four scripted answers and also answered questions without a script. This human validation covered the interview task, rather than all three evaluation settings.

American and British accents remained favored, with larger disparities between human accents than between the synthetic voices. Researchers also compared acoustic features—the sound characteristics of speech. Accents that were more acoustically distant from American accents often received lower scores.

Looking beyond average accuracy

The university’s account identifies several possible routes for this bias to enter training. Data may describe accented speakers as hard to understand or underrepresent particular accents. Training methods may also reinforce conventional expectations about what a professional speaker should sound like. These are possible explanations, not an identified cause of the score gaps.

Tsvetkov called for more representative training and evaluation datasets, and for developers to publish results broken down by population and dialect rather than relying only on average performance. Ahia pointed to hiring, oral exams and English tests used in immigration as settings where fairness in audio evaluation carries particular weight.

The team plans to examine more human accents, including regional differences within countries, and study audio AI in medical settings such as automated emergency response. Google and OpenAI, whose models were included in the experiment, partly funded the research through gifts.