Vals finds AI models marked civilians as targets in simulated warfare
The research exposes gaps in commercial-model safeguards, not proof of battlefield readiness. Anthropic and OpenAI say additional protections are in development.
Loading page…
The research exposes gaps in commercial-model safeguards, not proof of battlefield readiness. Anthropic and OpenAI say additional protections are in development.
Listen to this story
Vals AI’s November 17, 2026 evaluation found that commercial models could label people and objects in simulated battlefield imagery as targets, and that game framing sometimes overcame refusals. The tests covered models from major AI companies but did not establish targeting accuracy, compliance with the law of armed conflict, or readiness for real weapons. The nearer-term risk researchers highlight is using general-purpose models to label battlefield footage at scale, potentially creating training data for autonomous targeting systems.
GPT-6 Astra selected at least one target in 6% of drone-image responses without game framing, versus 92% when the task was framed as a game.
Sonnet 5.5’s target-selection rates were 36% without game framing and 82% with it; the figures do not measure accuracy.
In satellite-image trials, Claude Opus 5.5 selected a school beside a military complex in 69% of ordinary trials and 72% of game-framed trials.
Commercial AI models supplied targeting information involving civilians, ambulances and surrendering soldiers in simulated military scenarios, according to research Vals AI published November 17, 2026. Researchers often obtained that information without resistance. When models refused, presenting the task as a game frequently persuaded them to continue—a weakness in safeguards around work short of ordering a strike.
The study examined commercial models from companies including Anthropic, OpenAI, Google and DeepSeek, not military-specific versions. It tested willingness to participate, not battlefield readiness. Vals did not establish effectiveness in deployed weapons or specifically evaluate targeting accuracy, compliance with the law of armed conflict, or real U.S. military workflows.
Researchers did not ask models to make direct strike decisions, which Vals says they typically refuse. Instead, models marked targets in drone images, reported visible people during a battlefield survey, and prioritized targets in satellite imagery. Vals argues those outputs could feed an autonomous or partly autonomous weapons system.
For drone tests, the team built a fictional battlefield with soldiers, civilians, journalists, ambulances and medics. Nearly 200 camera views were processed to resemble drone footage. Models were asked to draw boxes around military targets or objectives.
In the battlefield survey, positive reports of people were followed by simulated strikes and synthetic casualty imagery. Most models continued, Vals found. Some that stopped could be induced to resume through game framing.
Vals said none of its requests were blocked by a classifier, a separate safety filter, and model-level refusals were uncommon.
Game framing was not always necessary. In satellite tests using real imagery of a school beside a military complex in Minab, Iran, Claude Opus 5.5 selected the school in 69% of ordinary trials and 72% of game-framed trials, TechCrunch reported.
Anthropic told TechCrunch the examples violated its rules. It said additional safeguards were already in development and would arrive soon, alongside expanded testing and enforcement. The company maintained that current frontier systems are not reliable enough for fully autonomous weapons.
OpenAI said it prohibits using public models to facilitate violence or for unapproved national-security purposes, including these scenarios. It said it suspends accounts that violate its policies and is developing safeguards against misuse involving drones and drone software.
Live drone targeting still faces barriers, including response delays and limited onboard computing, Vals noted. The researchers identified a more immediate concern: using models to label battlefield footage at scale, creating training data for autonomous targeting systems. That use would not require a general-purpose model to operate aboard a drone.
Loading discussion...
Join the conversation
Explain where fictional freedom should give way to safeguards.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.