TestMu’s New Agent Test Shows What It Can’t Verify

TestMu AI has launched Agent Assurance, a testing product for AI agents that can change files, call tools, access APIs and open pull requests. Rather than grade an agent only on its final response, the product is designed to check the effects left behind in the systems it touched.

The Aug. 18 launch comes from TestMu AI, formerly LambdaTest. Agent Assurance spans conversational agents and autonomous agents: the first category covers chat, voice, phone, video and image interactions, while the second targets software that takes actions across other systems.

From an agent’s answer to evidence of its actions

The intended mechanism is to run the agent and judge criteria against observable evidence: changed files, generated artifacts and verified tool calls. TestMu AI says the tool checks calls against the agent’s declared tool surface, shifting the evaluation toward what the agent did rather than what it says it did.

The test suite TestMu says it can generate

• Functional and non-functional checks derived from the codebase.
• Adversarial cases for prompt injection, tool misuse and instruction overrides.
• CI runs ranging from commit-level smoke tests to pre-release checks, with exit codes meant to distinguish agent failures from harness limitations.

A release gate with a visible blind spot

The company says teams can reduce the gap by making their agents more observable. It also positions the product for CI release gating, with run-over-run results intended to separate newly failing, newly fixed and flaky tests—categories that call for different release decisions.

Conversational testing is available; autonomous testing is early access

For video agents, TestMu AI says a simulated person can join a live video session and evaluate a conversation against team-defined criteria, with verdicts tied to moments in the recording. The company says any moment the evaluator cannot verify from that recording is counted as not met.

Proof of action is not yet proof of production safety

That leaves the next move with engineering teams: point the harness at agents operating across real corporate systems and see whether the evidence layer catches consequential errors before a release. Agent Assurance makes uncertainty visible in its reports; the harder question is how reliably that visibility changes outcomes.