Chatbots gave wrong or outdated voting information in 13.9% of tested answers, study finds

Chatbots gave outdated or inaccurate voting instructions in 13.9% of answers tested in October, according to Innovating for the Public Good findings covered by NPR on November 2. The nonprofit found that 5.9% of voting-logistics answers contained errors that could cause someone to miss a deadline, vote on the wrong day or skip a required step.

The ongoing study examines seven chatbot services: Google AI Mode and AI Overviews, Claude, Microsoft Copilot, Gemini, ChatGPT and Perplexity. Researchers use authentic voter questions covering election predictions, candidates’ backgrounds and policy positions, comparisons, controversies and voting rules. That gives the research two distinct kinds of answers to assess: information about the choices on a ballot and instructions for casting it.

Across the full set of questions, 5.1% of answers contained at least one clear factual error. The share rose to 10.9% when researchers also counted misleading claims. Those measures cover the broader election-question set, not just voting logistics, and distinguish a plainly wrong fact from the wider category of answers that could mislead.

A deadline that arrives too late

One example reviewed by NPR shows how a procedural error could interfere with voting. Asked when the 2026 election would take place in Michigan’s 4th congressional district and how to vote, Claude supplied incorrect deadlines for registering online and by mail. Both dates were later than the actual deadlines.

A voter following that answer could miss the registration cutoff. The example concerns a basic instruction for participating, rather than an argument about a candidate. IFPG’s primary researcher, Tim Chambers, warned that convincing, authoritative answers with meaningful errors could derail parts of the voting process when users trust them.

The researchers found that accuracy depended more on which chatbot answered than on the questions posed. Claude and Copilot together produced three-quarters of the clear errors across the study. That concentration is a separate finding from their voting-question error rates: it describes their contribution to the study’s total count of clear mistakes.

Anthropic questions the testing route

Anthropic spokesperson Michael Aciman told NPR that some key Claude safeguards may not have been active because IFPG conducted part of its research through an application programming interface, or API. An API lets researchers send questions through software rather than manually entering each one into a chatbot’s website.

That testing route allows researchers to automate prompts, standardize experiments and collect large numbers of responses. It also makes results easier to compare and reproduce. Anthropic’s objection concerns whether those experiments captured the protections available to someone using Claude through its consumer interface, not simply whether researchers could collect answers at scale.

IFPG said it addressed that concern with a separate internal study comparing Claude’s API responses with answers from the web interface. It found comparable results. The researchers’ account therefore challenges the suggestion that the testing route explains the findings, while Anthropic’s response raises a qualification about which safeguards the original tests exercised.

Microsoft spokesperson Jessica Dash said the company had improved how Copilot responds to election-related questions. She said Microsoft was continuing to test and refine those responses, with a focus on consistently delivering reliable information.