Harvard paper identifies two safeguard failures that could enable AI terrorism

AI developers have tools to reduce catastrophic misuse, but having safeguards is not the same as deploying protections that hold. A new Harvard Kennedy School Belfer Center paper identifies two likely failure paths for a major AI-enabled terror attack: a closed model with inadequate defenses, or a downloadable model whose safeguards users defeat.

Published October 15, the paper works backward from a hypothetical disaster rather than documenting a new attack. Its “pre-mortem” examines decisions by developers, distributors and governments that could enable—or interrupt—biological, chemical or cyber terrorism. The authors offer nine testable predictions and two illustrative scenarios, not a measured probability that an attack will happen.

In an October 19 Harvard Gazette interview, co-author and computer scientist Stephen Casper explained the threshold: more than 50 deaths or serious injuries, or more than $1 billion in damage. He said the project was not prompted by recent breach disclosures; its purpose was to help prevent such an event or improve the response if one occurs.

Two ways protection can fail

Closed-weight models keep their underlying parameters under the provider’s control. Users reach them through an interface, giving the company opportunities to filter requests, monitor outputs and longer sequences of actions, and restrict suspicious accounts. The paper argues that, as of summer 2026, no known attack strategy reliably bypassed a fully implemented, state-of-the-art layered defense.

That is not a clean bill of health for closed systems. The authors distinguish the best available defenses from what companies actually use. Sophisticated attacks can still exploit models with incomplete protections. They also warn that spreading a harmful project across providers and accounts fragments visibility: each company sees only part of the activity.

Useful capability has a safety cost

The researchers reject an easy separation between useful and dangerous knowledge. Training that helps a model support legitimate biological work can also give it capabilities useful for harm. Removing hazardous material or limiting those capabilities can reduce legitimate usefulness. The paper argues that competitive pressure gives companies reasons to avoid paying that safety cost.

Measuring the danger remains difficult. The authors say almost no developers publicly report randomized studies testing how much AI improves people’s ability to carry out harmful tasks. They also identify no rigorous way to translate benchmark scores into proportionate deployment decisions. A strong test result alone therefore does not supply a clear rule for when access should be restricted.

Intervention before an incident

The recommendations assign responsibility beyond model developers. The paper proposes action at several points in the chain, including:

• Developers: implement the strongest available safeguards, publish third-party adversarial testing, and offer narrow, sensitive capabilities through vetted access rather than unrestricted release.
• Distribution platforms: proactively moderate versions modified to remove safeguards, rather than waiting for requests to take them down.
• Policymakers: require disclosure and implementation of key safeguards and evaluations, establish deployment thresholds, and develop threat-information sharing between companies and government.

Researchers and funders have a separate assignment: support independent studies of how much models increase harmful capability and improve protections against tampering. That would address two gaps the paper treats as central—understanding what assistance an attacker actually gains, and keeping safeguards effective after a model leaves its developer’s control.