Superpower Daily

Superpower Daily

Your reading space

Sign in
Argonne Launches Three DOE-Funded AI Biology ProjectsManhattan DA Seizes 12 Domains Accused of Selling Celebrity Dee...OpenAI Uses Contractors to Review Real ChatGPT Prompts, 404 Med...Anthropic Tells Investors It Expects Another Quarter of Positiv...UK Rights Committee Calls for AI Law and Regulator With Enforce...MIT Researchers Publish HardFlow to Keep AI Outputs Within Hard...Argonne Launches Three DOE-Funded AI Biology ProjectsManhattan DA Seizes 12 Domains Accused of Selling Celebrity Dee...OpenAI Uses Contractors to Review Real ChatGPT Prompts, 404 Med...Anthropic Tells Investors It Expects Another Quarter of Positiv...UK Rights Committee Calls for AI Law and Regulator With Enforce...MIT Researchers Publish HardFlow to Keep AI Outputs Within Hard...
Swipe

Microsoft Research Finds Copilot Retries Can Quadruple Compute Costs

A production-scale snapshot of GitHub Copilot shows why tool reliability and context retention matter to the cost of autonomous coding work.

By 2 min read
Microsoft Research Finds Copilot Retries Can Quadruple Compute Costs
Microsoft Research Finds Copilot Retries Can Quadruple Compute Costs

Listen to this story

The audio brief

About 1:30
0:001:30
Read transcript
Tool failures can make GitHub Copilot’s computing bill jump as much as fourfold. That is the central finding from a Microsoft Research study of agentic coding in production, based on June 2026 traces covering 3.2 million users, 13 million sessions, 761 million model calls, and 95 trillion tokens. The reason is that coding agents do more than answer a prompt. In these sessions, nearly every language-model call was paired with a software-tool execution. When a tool fails, the agent may retry, often carrying a larger amount of context into the next attempt. That combination of repetition and expanding context drives up compute use. The study also shows why context management matters. The KV-cache, which stores work a model can reuse, had an average hit rate of 90 percent within a turn. Across turn boundaries, that fell to 55 percent. Starting a new conversation, switching models, and compacting the conversation history were major points where reuse broke down. There is a second infrastructure wrinkle: users can pause for minutes between turns. A lightweight predictor captured 86 to 90 percent of total idle time at those boundaries, potentially giving providers a basis for proactive resource decisions. The broader constraint is clear: serving autonomous coding agents may require infrastructure designed around retries, context loss, and waiting time—not just isolated chat requests. The key question is whether agent-native systems can manage those costs without sacrificing reliability.

Story brief

3 key points

A Microsoft Research analysis of production GitHub Copilot usage finds that agentic coding workloads create infrastructure costs conventional chat serving can miss. The study covered 3.2 million users, 13 million sessions, 761 million model calls, and 95 trillion tokens from June 2026 traces. Tool failures can trigger retries with expanding context, increasing compute costs by as much as fourfold. Cache reuse also...

  1. 01

    Copilot traces showed nearly one tool execution for every language-model call in agentic coding sessions.

  2. 02

    KV-cache hit rates averaged 90% within turns but only 55% across turn boundaries.

  3. 03

    New conversations, model switches, and history compaction were major points of cache invalidation.

Microsoft Research’s latest research roundup highlights a costly pattern in GitHub Copilot: when tools fail, coding agents may repeatedly retry with more context, raising computing costs by up to 4×.

The underlying study examined sampled Copilot traces from June 2026, covering 3.2 million users, 13 million sessions, 761 million language-model calls and 95 trillion tokens. It offers a production-scale view of a workload that differs from a chatbot conversation: a user starts a task, then an agent can carry out a sequence of model calls and software-tool actions.

A prompt can set off an autonomous loop

The researchers found that language-model calls and tool executions occur at nearly a one-to-one ratio in agentic coding sessions. That pairing makes tool behavior part of the computing workload, rather than a separate operational detail after a model generates text.

Earlier context is much easier to reuse within a turn
90%Within a turn

Average KV-cache hit rates reached 90% within a turn in the sampled Copilot traces.

55%Across turns

Average KV-cache hit rates fell to 55% across turn boundaries.

Up to 4×Failed-tool retries

Microsoft Research says retries with growing context after tool failures can raise computing costs by up to 4×.

Context is not carried forward reliably

The study measured average cache hit rates of 90% within a turn but 55% across turn boundaries. A KV cache stores work a model can reuse; lower hit rates mean less of that earlier work remains available. The researchers also found major invalidation after model switches or conversation-history compaction.

Three moments that weaken information reuse

  • Starting a new conversation.
  • Switching to another model.
  • Compacting the conversation history.

The next infrastructure problem is waiting time

The paper describes a contrast between fast agent turnarounds and user idle periods that can last minutes at turn boundaries. Its lightweight idle-time predictor captured 86% to 90% of total idle time, which the researchers say could support proactive resource decisions. Microsoft Research presents the findings as a foundation for agent-native serving infrastructure, rather than systems designed around self-contained chat requests.

Sources

  1. microsoft.comAgentic Coding in the Wild: Characterizing GitHub Copilot at Production Scale - Microsoft Research
  2. linkedin.comEvaluating coding agents at scale, lymphoma detection, AI dependence

Loading discussion...

Related coverage

Continue reading