Superpower Daily
Perplexity Search API Sweeps Artificial Analysis Test, but Medi...Anthropic Says Claude Found Fixes Across 10 Alignment Failures,...Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near...Decathlon Deploys Chronos-2, Cuts 12-Week WAPE by 11–15 PointsIndiGo Ventures Joins Sarvam’s $1.5B Series B, Linking Airline...Salesforce Says Agentforce Passed $1.5B ARR as It Pushes Consum...Perplexity Search API Sweeps Artificial Analysis Test, but Medi...Anthropic Says Claude Found Fixes Across 10 Alignment Failures,...Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near...Decathlon Deploys Chronos-2, Cuts 12-Week WAPE by 11–15 PointsIndiGo Ventures Joins Sarvam’s $1.5B Series B, Linking Airline...Salesforce Says Agentforce Passed $1.5B ARR as It Pushes Consum...
Toolspublished

Perplexity Search API Sweeps Artificial Analysis Test, but Medium Context Wins

The result gives developers a concrete retrieval-quality lead to test, while showing that sending an agent more page text is not automatically the best route to better answers.

By 3 min read
Perplexity Search API Sweeps Artificial Analysis Test, but Medium Context Wins
Sign in to save and react

Listen to this story

The audio brief

About 1:34
0:001:34
Read transcript
Perplexity’s medium Search API setting took the top spot in Artificial Analysis’s Search Index, scoring 80 out of 100. It also beat Perplexity’s own high-context setting, which scored 79, and its low-context setting, at 77. That is the interesting result: giving an AI agent more extracted page text did not produce the best answer quality. In this test, low context returned short, query-relevant passages; medium delivered a broader but still focused selection; and high supplied detailed page content. The medium payload appears to have given the agent the most useful material to reason over, without making relevant evidence harder to find. The nearest competing configurations, Parallel Search Advanced and Brave Search’s LLM Context mode, both scored 75. Perplexity charges five dollars per 1,000 Search API requests across all three settings, so the choice is mainly about what happens downstream. Artificial Analysis estimated total task costs of about 9.1 cents for medium and high, versus 10.5 cents for low. Medium also finished in 28.3 seconds, slightly faster than high at 29.1, while low took 36.6 seconds. The benchmark held the agent layer constant with GPT-5.6 Luna, 25 turns, and up to 10 results per search, across 900 research tasks. The practical next step is narrow but important: test medium on your own workload, because another model, context strategy, or task mix could change the ranking.

Story brief

3 key points

Artificial Analysis’ controlled agent benchmark found Perplexity’s medium Search API context setting delivered the best overall result, scoring 80 versus 79 for high context and 77 for low. The finding suggests retrieval quality depends on payload selection, not simply maximum extraction. Perplexity’s medium and high configurations cost about $0.091 per task and completed tasks in 28.3 and 29.1 seconds,...

  1. 01

    Perplexity’s medium setting scored 80, ahead of high at 79 and low at 77.

  2. 02

    Parallel Search advanced and Brave Search LLM context both scored 75.

  3. 03

    Perplexity charges $5 per 1,000 Search API requests across context settings.

Perplexity’s Search API occupied the top three positions in Artificial Analysis’ Search Index, with its medium-context setting scoring 80. The notable detail is not just the sweep: the medium setting outperformed both the larger high-context payload and the smaller low-context option in the same controlled agent test.

Search APIs give an AI agent web results and extracted page material to reason over. In this evaluation, the context setting determined how much content Perplexity returned from each result page: low supplied short, query-relevant passages, medium a balanced amount, and high detailed content.

That design choice produced an inverted result. Perplexity’s medium setting scored 80, versus 79 for high and 77 for low. Parallel Search’s advanced setting and Brave Search’s LLM context mode were next at 75, leaving a five-point gap between the leader and the nearest competing results.

Perplexity’s three leading settings
80Medium context

Medium context led Perplexity’s three settings in the index.

79High context

High context placed second overall.

77Low context

Low context placed third overall.

The payload is part of the product

The ranking suggests that a retrieval system’s value is not simply how much source text it can deliver. For this agent and task mix, the middle setting gave the model the strongest material to work with. More extracted content can increase downstream model usage or make the relevant passage harder for an agent to isolate.

The price structure makes that distinction operational. Perplexity charges $5 per 1,000 Search API requests regardless of context setting, but the total task bill can still change with the number of searches an agent makes and the tokens its answer model processes.

Quality came with a timing trade-off

Artificial Analysis estimated roughly $0.091 in total cost per task for Perplexity’s medium and high settings, including search charges and agent-model inference. The low setting cost about $0.105 per task. Parallel Search’s fast setting was cheaper at about $0.068 per task, but scored 73.

Perplexity did not lead on end-to-end task time. Its medium and high settings took 28.3 and 29.1 seconds per task, while low took 36.6 seconds. The metric includes both search calls and model processing, so it does not isolate the time for an individual search request.

A controlled result, not a universal verdict

Artificial Analysis held the agent layer constant by pairing every provider with GPT-5.6 Luna at medium reasoning effort in its open-source Stirrup harness. The agent had web-search and page-fetching tools, a maximum of 25 turns, and up to 10 results per search.

What the index tested

  • 900 DeepSearchQA tasks.
  • 600 held-out factual questions from AA-Omniscience.
  • A 200-question BrowseComp subset.

The setup makes the result useful for teams choosing retrieval infrastructure: it compares providers under the same model and agent rules. It does not establish that the ordering will persist with another model, another context strategy, or workloads centered on coding, commerce, or local search. The immediate decision is therefore narrower: test whether Perplexity’s medium payload improves the specific agent workload being built.

Related coverage

Continue reading