Dragos finds evidence of three AI provider breaches in two-million-prompt test
The five-month experiment also traced credential misuse to marketplace sellers and found secret links being accessed despite no-retention settings. The affected providers remain unnamed.
Dragos’s five-month test suggests prompt exposure can happen well beyond the model host: attackers used stolen test credentials to probe cloud services, while resellers and provider systems also accessed planted links. The findings span three unnamed hosting providers and include requests marked for Zero Data Retention, but the breaches have not been independently confirmed and Dragos could not determine whether some privacy failures were intentional or bugs. For teams buying AI access, each router and reseller adds another party that can see prompts.
01
In the largest incident, attackers tested some stolen AWS keys within hours and tried to access storage, secrets, and hosted AI services.
02
Some marketplace sellers exposed prompts through ordinary request logging, without any software flaw or exploit.
03
An unnamed Chinese AI lab fetched links from more than half of test prompts, including roughly two-thirds marked no-data-collection or Zero Data Retention.
Secrets planted in AI prompts became credentials for attempted break-ins. In findings published November 19, security firm Dragos described a five-month experiment involving nearly two million prompts. It found evidence of breaches at three AI hosting providers, access through reseller logs, and data handling that appeared to violate customers’ privacy settings.
The findings come from research led by Dragos chief research officer HD Moore. The providers are unnamed, and the reported breaches have not been independently confirmed. The share of other customers’ prompts affected remains unknown.
July: planting secrets nobody should use
The experiment began in July and ran through November 2026. Each request carried unique, trackable material, including an AWS access key, a web address and a domain name. These were canaries: digital tripwires whose later use would reveal that the prompt’s contents had reached another system or person.
The AWS keys were real, but connected to empty accounts with full logging. Dragos controlled the systems receiving the attempted connections. That let researchers match activity to the exact prompt, model, router and privacy setting used in the original request.
The route matters. AI routers offer one access point that forwards requests to models at different hosting providers. A request can pass through several routers before reaching the server running the model, and every intermediary can see the full prompt and response.
Late October: stolen keys expose an intrusion
Dragos dates the earliest provider incident to mid-July. None of the three hosting providers knew about the breaches before researchers notified them, Moore wrote. The largest incident involved a couple hundred AWS keys submitted during several short windows in late October.
Each key appeared in exactly one request to that provider. Dragos says every test request sent during those windows was stolen. Some keys came from requests rejected before a model ran—evidence, in Moore’s assessment, that the attacker was copying incoming requests rather than taking data from a later processing stage.
Attackers tested some keys within hours. They searched cloud storage and services holding secrets, then tried to enable hosted AI models and create users to preserve access. Some stolen requests in the other two breaches carried Zero Data Retention flags, which specify that prompts and responses should not be stored.
Other leaks needed no break-in
Marketplace sellers could obtain prompts without exploiting a software flaw, Dragos says. They listed models at low prices, enabled ordinary request logging and waited. In one case, two secret links sent through one seller were later opened, followed by attempts to guess usernames. Prompts sent to the same model through another seller were untouched.
Another person tried passwords from prompts against a remote-login service, then tested several AWS keys. One of those keys had gone through a different router. Dragos traced that overlap to an undisclosed connection: the router was forwarding some traffic into the same marketplace.
Separate privacy failures involved automated visits rather than credential use. Systems belonging to an unnamed Chinese AI lab fetched secret links from more than half the test prompts sent to its models between July and October. That included roughly two-thirds of requests marked no-data-collection or Zero Data Retention, according to Dragos.
Requests to several U.S. hosts reselling that lab’s models triggered the same scraper. The behavior was not limited to Chinese organizations: one U.S. provider kept visiting a secret link more than two months after a single request to its search service.
In another case, Dragos says a link from a no-retention request appeared in a Feishu company chat 24 days later. It was opened from devices in Beijing and Shanghai. Whether privacy-setting failures resulted from deliberate choices or processing bugs remains unresolved.
Reader comments
Newest comments first. Replies stay oldest first.