Microsoft Research Finds Copilot Retries Can Quadruple Compute Costs

Microsoft Research’s latest research roundup highlights a costly pattern in GitHub Copilot: when tools fail, coding agents may repeatedly retry with more context, raising computing costs by up to 4×.

The underlying study examined sampled Copilot traces from June 2026, covering 3.2 million users, 13 million sessions, 761 million language-model calls and 95 trillion tokens. It offers a production-scale view of a workload that differs from a chatbot conversation: a user starts a task, then an agent can carry out a sequence of model calls and software-tool actions.

A prompt can set off an autonomous loop

The researchers found that language-model calls and tool executions occur at nearly a one-to-one ratio in agentic coding sessions. That pairing makes tool behavior part of the computing workload, rather than a separate operational detail after a model generates text.

90%Within a turn

Average KV-cache hit rates reached 90% within a turn in the sampled Copilot traces.

55%Across turns

Average KV-cache hit rates fell to 55% across turn boundaries.

Up to 4×Failed-tool retries

Microsoft Research says retries with growing context after tool failures can raise computing costs by up to 4×.

Context is not carried forward reliably

The study measured average cache hit rates of 90% within a turn but 55% across turn boundaries. A KV cache stores work a model can reuse; lower hit rates mean less of that earlier work remains available. The researchers also found major invalidation after model switches or conversation-history compaction.

Three moments that weaken information reuse

• Starting a new conversation.
• Switching to another model.
• Compacting the conversation history.

The next infrastructure problem is waiting time

The paper describes a contrast between fast agent turnarounds and user idle periods that can last minutes at turn boundaries. Its lightweight idle-time predictor captured 86% to 90% of total idle time, which the researchers say could support proactive resource decisions. Microsoft Research presents the findings as a foundation for agent-native serving infrastructure, rather than systems designed around self-contained chat requests.