Researchers find gaps in AI agents’ identity and human-approval records

Catching an AI agent’s unauthorized actions can depend on knowing which agent acted and which human approved its work. A study of four widely used agent frameworks found gaps in those basic records, according to findings described by Partnership on AI researchers in an October 29 opinion article in MIT Technology Review.

Authors Madhulika Srikumar and Claire Leibowicz explain that agents can plan, select tools, create sub-agents and take actions over longer tasks. Those changing paths make pre-release testing insufficient to anticipate every risk, strengthening the case for monitoring agents while they work.

The missing trail from action to approval

The missing records included persistent agent IDs—identifiers that remain consistent over time—and records showing which humans approved tasks. The authors say these gaps made it harder to detect policy violations, unauthorized activity and failures of human oversight.

Their proposed remedy starts with observability: collecting system activity and turning it into signals people or automated oversight systems can inspect. They distinguish two categories:

• What happened: identify the agent while preserving privacy, record human interventions, and track changes to its permissions or memory.
• How it decided: capture records of the agent’s reasoning and its confidence in the actions it took.

In the authors’ hypothetical mortgage rejection, a bank would need to establish which agent recommended denial, which loan officer approved it and how long approval took. Examining whether the decision used prohibited neighborhood information would require a different record: how the agent reached its recommendation.

Shared logs cannot settle questions about reasoning

For action records, the authors point to OpenTelemetry, an open industry standard for collecting software activity. A working group’s shared agent-tracing formats are not final. They urge it to include the missing records before systems scale, while addressing privacy risks when agent identities link to real people.

The step-by-step explanation a model displays, known as chain of thought, may not reflect how it actually reached a decision. Reliable measurement of agent confidence is also unresolved. The authors argue these questions need answers before standards determine how decision records should be used.

Someone still has to define the rules

Banks, hospitals and other regulated buyers can press developers to supply missing records before standards exist, the authors say. Buyers must also define policies, permissions and roles: activity logs alone cannot determine whether an agent broke a rule.

They propose that regulators specify required records for high-stakes deployments while leaving technical implementation to industry. They identify NIST’s agent standards initiative, which is working on agent identity, as an initial step.