Perplexity launches AI gateway that retries failures without switching models
Orchestrator combines model access and billing. Its optional routing preview goes further, allowing a conversation’s model—and cost—to change between turns.
Loading page…
Orchestrator combines model access and billing. Its optional routing preview goes further, allowing a conversation’s model—and cost—to change between turns.
Listen to this story
Perplexity's Orchestrator API gives developers a single interface to access proprietary and open-weight models while routing failed calls to another deployment of the same model—for example, moving a Claude request from Anthropic to AWS Bedrock without changing the answerer. Its separate `perplexity/auto` preview can select a different model for each request, potentially changing capability and cost. The service supports existing OpenAI and Anthropic request formats, but routing is stateless, model access depends on account tier, and streamed failures after output begins remain billable.
Perplexity says Claude and GPT requests had higher success rates than any individual provider it measured from October 4 to November 2, excluding client errors.
Provider errors, rate limits, or timeouts can trigger a same-model retry; invalid requests and context-window overruns return HTTP 400 without failover.
The `auto` preview supports a default model and one to five plain-language task routes, with the default used when no route fits or routing fails.
A failed AI request need not mean abandoning the chosen model. Perplexity introduced its Orchestrator API on November 17, 2026, combining proprietary and open-weight models behind one software interface and bill. Its central reliability feature retries requests through another provider of the same model; a separate preview can choose different models as a conversation changes.
The distinction starts with where a model runs. Perplexity says Orchestrator selects a provider using availability, capacity and latency. After a model is selected, traffic is distributed among its deployments, taking observed error rates into account. Customers do not choose the deployment themselves.
Provider errors, rate limits and timeouts can trigger another attempt elsewhere. A Claude request that fails at Anthropic, for example, can move to AWS Bedrock while remaining a Claude request. Unite.AI’s launch coverage describes this as same-model failover, distinct from the optional system that chooses which model should handle a task.
Perplexity reports that Claude and GPT requests through Orchestrator succeeded more often than requests to any individual provider it measured between October 4 and November 2. Daily success rates also fluctuated less during provider incidents. These are company-reported results, with client errors—including invalid requests and client rate-limit violations—excluded. A failure meant all available deployments had been exhausted or the serving layer returned an error.
Setting the requested model to perplexity/auto invokes a decision model. Developers can supply a default model and one to five task routes, each described in plain language. The decision model sees the conversation and route descriptions, but not route names or model IDs.
Routing happens independently for each request, so a conversation can change models between turns. That can change both capability and cost. The default model handles requests when no route fits, when the decision model fails or times out, or when the conversation includes images, audio, video or files.
Orchestrator is stateless: every request must carry the full conversation. Each model keeps a separate prompt cache, which stores reusable input. Stable session identifiers or cache keys offer only a best-effort connection to one deployment; they do not lock a conversation to one model. Response headers describe the routing outcome, while the response’s model field identifies which model answered.
The API accepts OpenAI Chat Completions, OpenAI Responses and Anthropic Messages requests. Developers can retain their existing OpenAI or Anthropic software libraries, changing the API key, service address and full catalog model ID. Unlike Perplexity’s Agent API, which supplies web-grounded answers with citations, Orchestrator provides direct model access using customers’ own prompts and tools.
Billing follows the model that actually answers, at its published per-token rate, regardless of which deployment serves it. Usage draws from the same prepaid credits as other services in a Perplexity API project. Failover does not mean every unsuccessful request receives another attempt, or that every interrupted response is free:
Model access also depends on account tier. Google, xAI and open-weight models are available at every usage tier, while OpenAI and Anthropic models require Tier 1 or higher, with some Claude models requiring Tier 2. A familiar client library therefore does not guarantee access to every model.
Loading discussion...
Join the conversation
Explain when that information would help, or just add noise.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.