Rescale says AI triage cut deployment-blocking analysis time by 20%

Rescale says it has reduced the time test-failure analysis blocks production deployments by 20%, using an AI agent to handle the first pass. An October 22 AWS case study details the production system: it investigates nightly failures and prepares recommendations, but engineers validate its work before any action is taken.

The delay was not simply an inconvenience for the on-call engineer. Rescale requires its end-to-end tests, which check how services work together, to pass before weekly production releases. A failure could draw several teams into hours of coordination. Before debugging began, engineers had to classify the problem, identify its likely owner and collect evidence from logs, past failures and source repositories.

The agent replaces that manual first pass, not the engineer’s judgment. Rescale says its classifications now agree with human triage 68% of the time, up from 44% at initial deployment. The team continues to refine its reference dataset and prompts. These are company-reported results in an AWS case study, not an independent evaluation.

Closer to human triage

Classification agreement with human triage

44%→68%

%

Rescale’s reported agreement rate rose from initial deployment to the figure disclosed in the case study.

From a failed test to a proposed fix

The system starts inside Rescale’s existing automated build, test and deployment pipeline. When its nightly Playwright suite of hundreds of tests produces failures, Argo Workflows triggers the agent on Amazon Elastic Kubernetes Service, or EKS. Built with the Strands software development kit, the agent can analyze failures concurrently and return structured reports in JSON and HTML.

Claude Opus 4.8, accessed through Amazon Bedrock, classifies each failure into one of four categories: infrastructure issue, code bug, flaky test or environment problem. The agent retrieves historical failure patterns from a knowledge base. It also queries runtime logs from Amazon CloudWatch through Model Context Protocol, or MCP, the connection used to retrieve that evidence.

The output goes beyond a category label. Each remediation report includes file references, function names and recommended fixes for engineer review. AWS says the workflow completes in seconds and sends its output into the team’s existing reporting pipeline. Automatic pull-request generation was explicitly deferred during scoping; the deployed workflow supplies recommendations for people to assess.

Five days to staging, with humans choosing the architecture

AWS architects and Rescale engineers built the system during a five-day, in-person workshop using the AI-Driven Development Lifecycle, or AI-DLC. The open-source methodology has AI generate plans, code and architecture proposals while humans approve decisions. Requirements and construction are worked through in focused group sessions rather than handed between people for later review.

The early work established boundaries before implementation. Engineers collectively answered more than 20 AI-generated clarification questions and separated immediate requirements from deferred features. They organized the build around four units: the historical knowledge base, the triage agent, integration with Argo Workflows and an evaluation framework. Interface contracts, data schemas and architecture also required team approval.

Two decisions changed the design. The team switched its planned agent runtime from Amazon Bedrock AgentCore to EKS, where the existing pipeline already ran. That avoided another runtime to operate and gave Argo a direct trigger path. It also replaced a third-party logging service with CloudWatch to simplify dependencies, while retaining the same MCP retrieval pattern.

By day four, the complete pipeline was running in staging. The fifth day focused on evaluation and handoff, including a PyTest-based framework with live and stubbed test modes to establish a classification baseline. AWS says the project reached production within weeks. The evaluation framework gave the team a way to measure and improve the agent after the workshop ended.