Chatbots gave flawed abuse advice in 86% of tested replies, NYU study finds

People seeking help with spyware, tracking or compromised accounts can receive chatbot advice that misses the danger of their situation. In an October 20 research account, NYU Tandon described a four-system study in which 86% of tested responses fell short on accuracy, completeness or safety. Even chatbots designed for abuse survivors failed to outperform general-purpose models.

The study was presented at the USENIX Security Symposium in August 2026. It examined technology-facilitated abuse: situations in which someone uses devices or accounts to monitor, harass or control another person. The researchers also tested the opposite risk—whether systems intended to help could be persuaded to assist a potential perpetrator.

Starting with survivors’ questions

The NYU-led team assembled 193 questions from academic literature, Reddit and Quora. Researchers submitted them to GPT-4o, Claude 3.7 Sonnet, and two chatbots built specifically for survivors of intimate-partner violence, Aimee and Ruth. They checked the answers against authoritative sources for accuracy, completeness and safety, and surveyed 114 people who had experienced technology-facilitated abuse about how actionable the advice seemed.

The need for help is substantial. NYU says more than 85% of U.S. domestic violence practitioners report working with clients who have experienced technology-facilitated abuse. Yet, to the researchers’ knowledge, only three dedicated technology-abuse clinics operate nationwide. The evaluation asked whether AI-generated answers could provide guidance in that high-risk setting.

Aimee had the highest share of answers rated unsafe, at about 74%. Claude followed at 65%, Ruth at 61% and GPT-4o at 55%. Ruth also had the highest share rated partially or minimally accurate, about 62%. These are ratings of responses in this evaluation, not measures of how often actual users suffered harm.

An answer can sound confident and reasonable, and may even be technically correct, while still leaving out critical context or recommending something that does not actually make the person safer

Danny Huang, NYU Tandon assistant professor and the project’s principal investigator

When a technical fix misses the danger

Across the tested responses, 60% omitted critical safety warnings. Systems frequently failed to warn that changing a device or account could alert an abuser or escalate the situation. They also overlooked preserving a compromised device or account as evidence. The recurring problems included both unsuitable tools and advice that ignored who controlled the technology:

• VPN recommendations for spyware, account hijacking or harassment, despite those being inappropriate solutions to the problems described.
• Largely ineffective radio-frequency detectors for hidden devices, as well as recommendations to use illegal GPS jammers.
• Password changes in situations where an abuser controlled or shared access to the accounts or devices.

None of the four systems noted that spyware installed in a phone’s system partition can survive a factory reset if the phone is rooted or jailbroken—modified to allow deeper access to its software. That omission could leave an important exception out of advice about resetting a compromised phone.

The same systems could help the other side

Researchers could induce all four systems to assist a potential perpetrator. Ruth and Aimee answered roughly two-thirds of those questions. Digital Trends’ October 22 coverage describes the mechanism: potentially harmful surveillance requests were framed as technical troubleshooting, including requests involving camera or microphone permissions. The findings concern responses elicited in tests, not documented stalking incidents.

The researchers recommend caution with AI-generated safety and security advice. Their next-step recommendation for support organizations is independent evaluation before relying on these systems for high-stakes guidance. They have also released research resources for further study. The proposed safeguard is scrutiny of the advice itself, rather than assuming a survivor-focused chatbot is safer because of its intended audience.