Researchers propose a formula to predict harmful AI replies, but real-time use remains untested

A formula intended to warn before an AI chatbot gives a harmful answer remains a research tool, not a live safety system. In October 26 interviews with The GW Hatchet, George Washington University researchers explained how their approach tracks changing internal priorities—and why applying it to commercial chatbots still requires more research.

Physics professor Neil Johnson and doctoral student Frank Yingjie Huo published the underlying study in Patterns on October 8, 2026. Their central finding is that conversation history can influence whether—and when—a model shifts from desirable responses to undesirable ones. Even changing the order of questions can change its behavior.

Following the model’s shifting priorities

The approach looks inside a model rather than judging only the answers it has already produced. It examines attention, the mechanism that helps determine which parts of a conversation influence the next response. The researchers describe competing information patterns that can move the model toward different kinds of output.

Johnson explained the equation to the Hatchet through a cliff-edge analogy: estimate the distance to dangerous behavior and divide it by the speed of approach. The aim is to estimate when a transition might occur, not simply label an answer as harmful after a user has received it.

One experiment used GPT-2 and questions about vaccines, harming others and self-harm. As Live Science described on October 14, the researchers found that placing the vaccine question first rather than last changed subsequent response behavior. Their explanation centers on the sequence of information, not just the content of an individual question.

What the laboratory result measures

The university’s research announcement describes seven open-weight models ranging from 124 million to 12 billion parameters. Open-weight models make their underlying numerical settings available for examination. The researchers chose publicly accessible models because they could inspect how those systems process information and generate responses, rather than testing commercial chatbot products.

The Hatchet reported that the formula correctly identified whether an undesirable response would occur immediately, after a delay or not at all in 18 of 19 evaluated cases, excluding two borderline cases. That figure concerns those experimental outcomes; it is not a measured success rate for screening everyday chatbot conversations.

“Undesirable” also covers more than factual errors. The study’s framing includes misleading information, encouragement of harmful behavior and accurate information that could endanger someone. Johnson emphasized that distinction for sensitive advice: a technically correct response can still be inappropriate or dangerous for the person asking.

The authors reduce a huge system to a single tiny component, like explaining a whole car by studying one spark plug.

Wei Xing, University of Sheffield AI lecturer, speaking to Live Science

A warning is not a prevention system

Xing, who was not involved in the study, questioned whether the simplified framework could become a practical safeguard. He told Live Science that it could not represent safety training, a primary defense in modern AI, and that tests on older, basic models did not provide significant evidence for the authors’ conclusions.

Johnson’s response is that the tested models share the same mechanical blueprints as frontier systems. In the newer Hatchet interviews, he said the mathematical principles could apply to larger models, while acknowledging that additional testing would be needed to establish accurate predictions. Huo likewise described commercial applications as requiring research into more complex systems.

The proposed destination is a lightweight monitor that watches internal processing and flags an approaching harmful response. The researchers describe it as a supplement to existing safeguards, not something that would automatically prevent dangerous answers. Detecting a possible transition and stopping the resulting output remain distinct tasks in that proposal.