Boston University selects AI reading tutor Amira for trial against human tutoring

An AI reading tutor used by millions of students is headed for a test against human help—not just the absence of extra instruction. Boston University’s EVAL Challenge has selected Amira Learning for its first randomized controlled trial, with trained BU undergraduates providing live tutoring to one of the three groups.

The selection, described in BU Wheelock’s October 23 article, followed 35 applications. The EVAL Challenge invited educational technology companies to apply for a free trial conducted by BU faculty.

A human alternative, not just a no-help baseline

• Amira tutoring: Students will use the AI-based reading tutor selected through the challenge.
• No supplemental instruction: Students will receive no extra instruction, providing a comparison without either of the study’s tutoring options.
• Live undergraduate tutoring: Students will receive help from trained BU undergraduates, creating a human comparison rather than evaluating the software alone.

BU describes the undergraduate group as a way to test whether a scalable human option requiring limited training can rival an AI tool. The comparison is therefore specific: Amira versus trained undergraduates, not Amira versus every form of professional reading support.

Widespread use leaves a causal question

According to BU, Amira has served millions of students but has no published causal evidence of effectiveness. That is a narrower statement than saying the tool does not work. It identifies the question this trial is intended to address: whether using the product causes better educational outcomes.

Ola Ozernov-Palchik, director of BU’s Evidence-Based AI in Learning Industry Collaborative, argues for a much higher evidence bar across classroom technology. She says her research shows that less than 10 percent of the technology children use in school is backed by studies demonstrating that it works. That estimate concerns educational technology broadly, not AI reading tutors alone.

Basically, we don’t integrate a tool into classrooms unless we have the causal evidence that it works.

Ola Ozernov-Palchik, describing her proposed standard for classroom technology

Making evidence a competitive advantage

Her proposed standard draws on pharmaceuticals: randomized controlled trials that establish whether an intervention works. She likens the oversight she wants to an FDA for education.

Ozernov-Palchik also hopes the challenge changes companies’ incentives. If the trial finds Amira effective, she says, the company would gain an advantage over tools that have not undergone evaluation. That could put pressure on competitors to seek research of their own. Her stated goal is friendly competition over demonstrated educational outcomes, with evaluation becoming something companies want to obtain.