Researchers warn missing chatbot records hide harms from AI health advice

When someone is harmed after following chatbot health advice, doctors may be unable to recover the conversation that explains what happened. A Nature Health Perspective published October 9, 2026, argues that inaccessible records and missing reporting channels leave those harms hidden from public-health oversight—and calls for changes to how AI companies handle health guidance.

The paper, “The Invisible Risks of AI-Generated Health Information,” is a policy argument, not a new experiment measuring chatbot accuracy or patient outcomes. Its authors focus on what happens after potentially harmful guidance reaches someone: whether the advice can be inspected, the problem reported and its consequences independently assessed. They recommend both voluntary platform changes and regulatory measures.

The authors acknowledge the benefits of AI health information: plain-language explanations, round-the-clock availability and help for people facing language or literacy barriers. Their concern spans two settings. People can actively ask a chatbot or search engine for medical guidance, or encounter AI-generated health claims incidentally while browsing online. Oversight needs to account for both.

A poisoning case without the conversation

A Binghamton University account published by Medical Xpress describes a case the paper uses to illustrate the problem. A 60-year-old man reportedly asked ChatGPT how to reduce chloride in his diet, replaced table salt with sodium bromide and was hospitalized with bromide toxicity after experiencing hallucinations and paranoia. He could not retrieve the chatbot conversation, leaving his doctors unable to determine exactly what advice he had received. The account therefore does not supply a verified transcript of ChatGPT’s instructions.

The authors’ proposed response starts with access: AI companies should let users retrieve their health conversations and share them with clinicians investigating a harmful event. They also want a disclosure system through which health guidance can be flagged, reported and investigated. Those recommendations address the evidence trail, rather than relying solely on preventing every incorrect answer.

Harms are not only possible, but structurally hidden from the clinicians, researchers, and regulators who could otherwise detect and correct them

The Perspective’s authors, quoted in Binghamton University’s account

Safeguards before health claims reach people

The paper also addresses how harmful information is delivered. Chatbots can confidently produce wrong answers, oversimplify medical information or draw on outdated information and false claims. The authors warn that inaccurate guidance can harm people at scale, while malicious actors can generate personalized health misinformation at negligible cost. Their delivery-focused recommendations extend beyond the companies operating chatbots.

• Social platforms should make labels on AI-generated health content clearer, so people can recognize its origin.
• They should withhold that content until medical governing bodies have vetted it—a recommendation that goes beyond labeling alone.
• Search engines should use only vetted information in their summaries, placing a check on the material used to generate health answers.

Malpractice liability enters the proposal

The authors suggest policymakers extend physician malpractice liability to AI chatbot companies. This is a proposed accountability measure, not a legal change announced by the paper. Alongside it, they call for independent third-party monitoring. Their recommendations therefore combine ways to inspect individual interactions with outside scrutiny of the systems producing health advice.

Co-author Kaicheng Yang, an assistant professor in Binghamton’s School of Computing, argues that companies’ financial incentives make voluntary oversight insufficient. In the university account, he says building a monitoring system runs against those incentives and calls for independent evaluation. That is the authors’ case for involving outsiders: platforms should not be the only institutions able to examine the health guidance they provide.