Chatbots Debunked Foreign Falsehoods About 75% of the Time, Beating Search Results

For someone confronting a false claim tied to a foreign influence campaign, a chatbot may be more likely to challenge the premise than a conventional page of search links. In an NPR and NewsGuard experiment, popular chatbots correctly debunked state-spread false narratives about three-quarters of the time and failed less often than traditional search results.

That finding comes from a test designed around 30 questions based on false narratives attributed to China, Iran and Russia. The narratives first appeared between December 2025 and July 2026; researchers asked the questions of popular chatbots, including ChatGPT and Gemini, as well as major search engines.

The comparison is about challenging a false premise

The test’s key failure measure was narrow and consequential: whether a tool did nothing to challenge a false narrative. For chatbots, that could mean treating an untrue claim as fact or accepting a loaded question’s premise. For search engines, researchers counted cases where the relevant first-page links offered only false information.

The distinction matters because a chatbot can explain why a claim is wrong, while ordinary search requires the user to judge and assemble competing links. In one test question built on the false premise that Ukraine bombed a historic monastery, all tested chatbots and Google’s AI Overview flagged the premise as faulty.

Search summaries are the unstable middle layer

AI-generated summaries at the top of search results performed worse than chatbots and, unlike the chatbots, failed to challenge false narratives more often than traditional search results. The aggregate masks a sharp product split: Google AI Overviews debunked the narratives most of the time, Bing’s summaries failed to do so most of the time, and DuckDuckGo fell between them.

Exposure also differed. Google showed an AI Overview for all but three queries, while Bing generated summaries for fewer than half; DuckDuckGo’s default rate sat between the two. The companies offer limited detail about when summaries appear, so the test does not isolate how often a user would encounter each product’s behavior in ordinary searching.

A correct answer still needs an auditable source trail

The chatbot advantage does not resolve the source problem. State-controlled and state-aligned outlets appeared in AI-generated answers at rates similar to their appearance in conventional search links. Within the Claude results, those sources appeared more often in answers that failed to debunk a narrative than in successful debunks.

Citations are not a complete safeguard either. A paper from Washington University in St. Louis found that about one in nine individual factual claims in Google AI Overviews was unsupported by the sources cited. Google said Overviews can synthesize material from multiple pages and conduct related searches beyond the user’s original request.

The result may not travel cleanly across languages

All NPR-NewsGuard prompts were in English. A Nature-published study cited in the experiment found that models gave more positive responses about China’s government and leaders when users asked in Chinese rather than English. That leaves an important boundary on the result: a tool’s resistance to a narrative may depend partly on the language in which the claim is posed.

Google disputed the experiment’s methodology, arguing that some responses scored as failures still provided useful context and links, and that the queries were rare rather than representative of normal use. That criticism does not erase the observed differences, but it does frame the benchmark correctly: it measures performance on a specific, high-risk class of deceptive questions, not general search quality.