Superpower Daily

Superpower Daily

Your reading space

Sign in
Review Separates Workflow Approval From Tool Enforcement in AI...AWS Says Iran Strikes Caused Permanent Customer Data LossOpenAI Launches Astra for Law as a Platform for Legal AI Builde...Scale AI and Korea AI Safety Institute Release Benchmark Exposi...Anthropic Opens Claude-Built Biology Tool Optimizations It Says...Scaleout Demonstrates Drone and Field AI Built for Disrupted Li...Review Separates Workflow Approval From Tool Enforcement in AI...AWS Says Iran Strikes Caused Permanent Customer Data LossOpenAI Launches Astra for Law as a Platform for Legal AI Builde...Scale AI and Korea AI Safety Institute Release Benchmark Exposi...Anthropic Opens Claude-Built Biology Tool Optimizations It Says...Scaleout Demonstrates Drone and Field AI Built for Disrupted Li...
Swipe

Oracle Publishes 1,000-Agent Cloud Benchmark, With a Clear Latency Trade-Off

The company’s reference design separates public request handling from private agent execution and gives each agent durable storage, but its own test shows that pushing concurrency higher can sharply slow the slowest responses.

By 3 min read
Oracle Publishes 1,000-Agent Cloud Benchmark, With a Clear Latency Trade-Off
Oracle Publishes 1,000-Agent Cloud Benchmark, With a Clear Latency Trade-Off

Listen to this story

The audio brief

About 1:38
0:001:38
Read transcript
Oracle has published a benchmark for running 1,000 persistent AI agents in the cloud—and its clearest result is a latency warning. In an Ashburn test, all 6,000 short, read-only requests succeeded without client retries. But the fastest overall batch did not come from the highest concurrency. With 500 requests running at once, 1,000 requests finished in 54.27 seconds, or 18.4 successful requests per second. At 1,000-way concurrency, the batch still completed, but the slowest one percent of responses took as long as 111.53 seconds. At 50 and 100 concurrent requests, that same p99 latency stayed near 10 seconds. The architecture separates a public request path from private agent execution. Oracle Kubernetes Engine, or OKE, manages the agent containers. OCI File Storage Service gives every agent a durable private workspace, while account reference files are mounted read-only and selected project artifacts can be shared. A stateless Model Context Protocol gateway receives requests and hands them to runners through in-cluster messaging, with model serving kept on a private path. That makes this a useful infrastructure baseline, not a production-capacity promise. The test used one short, read-only prompt per agent, and did not include lengthy tool use, writes, or active collaboration. Oracle’s next validation will measure nodes and Pods, file-storage and model-service latency, queue depth, and restarts. The key constraint is now clear: scaling concurrency can finish the batch, while making the slowest responses dramatically slower.

Story brief

3 key points

Oracle’s result is best read as an infrastructure baseline, not proof that 1,000-agent deployments are production-ready. Its architecture kept all 6,000 short, read-only requests successful, using durable per-agent storage, isolated workspaces, and queued handoffs. But the fastest batch favored 500-way concurrency; pushing to 1,000 raised p99 latency to 111.53 seconds. Because the test excluded lengthy tool use,...

  1. 01

    At 500 concurrent requests, the 1,000-request batch finished in 54.27 seconds, delivering 18.4 successful requests per second.

  2. 02

    At 1,000 concurrency, the batch completed, but p99 end-to-end latency reached 111.53 seconds.

  3. 03

    Concurrency levels of 50 and 100 kept p99 latency near 10 seconds.

Oracle has published a reference architecture and benchmark for operating 1,000 persistent AI agents on its cloud. In the company’s Ashburn test, all 6,000 short, read-only requests succeeded without client retries—but the results also showed a stark choice between batch speed and slower responses for the last users in line.

A public front door, private agents behind it

The design uses Oracle Kubernetes Engine, or OKE, to place and manage the containers that run individual agents. OCI File Storage Service provides durable workspaces, while a stateless Model Context Protocol gateway receives public requests and hands work to runners through in-cluster messaging. Model serving stays on a private path.

That separation addresses a practical problem with long-running agents: their work cannot safely live only inside a container that may be recreated or moved. Oracle’s storage layout gives each agent a private read-write area, makes account reference files read-only, and reserves a separate shared area for project artifacts. The runner is not given the storage root or other agents’ private workspaces.

The workspace boundaries

  • Private workspace: an individual agent’s state and task artifacts.
  • Reference files: account materials and installed skills mounted read-only.
  • Project workspace: shared artifacts available to related agents.

The test measured a narrow workload

Oracle used a fixed roster of 1,000 persistent agents across six concurrency settings, from 10 to 1,000 simultaneous requests. Each agent received one short, read-only prompt in each phase, so the benchmark tests the public request path’s ability to reach the whole roster rather than a workload involving lengthy tool use, writes, or active collaboration among agents.

The strongest throughput result came at 500 concurrent requests: 18.4 successful requests a second. Lower settings of 50 and 100 simultaneous requests held p99 end-to-end latency near 10 seconds. Oracle’s benchmark therefore shows a system that can finish the entire request set at every tested level, while exposing an operating choice: more concurrency did not mean faster service for every request.

A baseline, not a capacity promise

The reported result moves the discussion from an abstract agent swarm to a disclosed deployment pattern: clustered runners, durable workspaces, controlled access and queued handoffs. It is not yet a full account of how the design behaves under heavier or more varied work. Oracle says its next validation stage will add measurements for nodes and Pods, file-storage latency, queue depth, model-service latency and restart events.

Those measurements are intended to help Oracle evaluate paths toward larger agent populations and more demanding workloads. For now, the company has supplied a disclosed baseline with a useful warning built into the numbers: scaling an agent fleet is also a decision about which delays users can tolerate.

Sources

  1. blogs.oracle.comScaling 1,000 AI Agents on Oracle Cloud Infrastructure Kubernetes Engine and File Storage

Loading discussion...