Full archive

Models AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-11 of 11Results matching "Agent harness"

Earlier this week

3 stories

Sep 1, 2026

Grok 4.6 Clears 50% on Biosecurity Refusal and Routine Biology Tasks

The reported result rewards a difficult balance: recognizing hazards hidden inside ordinary-looking research requests without shutting down legitimate biological work. Its surveillance score shows that refusal performance is not the whole biosecurity test. The result puts the model’s ability to distinguish intent at the center of a biosecurity contest where excessive blocking can also impair useful work.

3 min readRead story

Benchmark Models

Aug 31, 2026

Qwen3.8-Max Leads Open-Weight Commerce Agent Bench by Two Passes, With Audit Limits

The benchmark checks whether agents leave commerce software and files in the required state, rather than simply narrate a workflow. But configuration choices and unavailable immutable result bundles limit what the narrow ranking can settle. Qwen3.8-Max holds a two-pass lead over DeepSeek V4 Pro in the published open-weight Commerce Agent Bench results, with 156 passes across three agent harnesses.

2 min readRead story

Open release Models

Aug 31, 2026

DeepSeek Releases 305B Vision Weights Under MIT License, but Local Use Needs Serious Hardware

The release gives developers the model files and serving references needed to run or adapt DeepSeek-V4-Flash-Vision-Exp independently. Its 168GB size means that freedom is aimed more at infrastructure-equipped teams than typical individual users. The shift gives developers more control over a model that accepts images, but the 168GB checkpoint sets a high infrastructure threshold for anyone seeking to run it locally.

3 min readRead story

Last week

4 stories

Research Models

Aug 28, 2026

Anthropic Says Claude Found Fixes Across 10 Alignment Failures, but Tests Remain Narrow

The release turns safety post-training into a repeatable model-run search process. Its value now depends on whether those benchmark gains survive broader tests and later training. The experiments test whether models can help generate the safety training used to shape other models.

3 min readRead story

Benchmark Models

Aug 28, 2026

Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near Claude Opus 4.5

The result is a strong benchmark showing for learned state management, not yet evidence that the framework improves the long-running enterprise workflows it is designed to target. Meta AI and the University of Illinois Urbana–Champaign say their EvoHarness-RL framework let Qwen3-8B reach 96.9% average success on ALFWorld, edging Claude Opus 4.5’s reported 96.4% score. Source: venturebeat.com.

3 min readRead story

Launch Models

Aug 25, 2026

DeepSeek V4 Pro Lands on Fireworks With $2.50 CyberGym Solves, but Kimi K3 Scores Higher

The provider’s results frame security-agent selection as a three-way choice among raw solve rate, cost per completed task, and whether a model will reliably execute the required tool workflow. Fireworks AI has made DeepSeek-V4-Pro-0813 available on its platform, pairing the release with security-benchmark results that favor the model on cost and tool reliability rather than outright solve rate.

3 min readRead story

Benchmark Models

Aug 24, 2026

NVIDIA Says AVO Took Claude Opus 5 From 30% to Perfect on ARC-AGI-3

The result shifts attention from training larger models to the software that gives them memory, tools and recovery loops. Its clearest limitation is equally important: the perfect score came on ARC-AGI-3’s public environments, while private sets remain the harder test. It used 6,624 actions, which NVIDIA’s cited comparison puts at about 12% fewer actions than the previous most efficient system.

3 min readRead story

August 2026

4 stories

Benchmark Models

Aug 22, 2026

Stanford Won Databricks’ Agent Cup, but 18.8% of Questions Stumped Every Team

The live contest tested whether agents developed on one benchmark could handle an unfamiliar Treasury archive. Stanford led the field, but the unsolved questions show the limits of reliable document reasoning. Stanford won Databricks’ inaugural Grounded Reasoning Cup with 63.3% accuracy, but 18.8% of questions were unsolved by every team.

2 min readRead story

Launch Models

Aug 22, 2026

DeepSeek Gives V4-Flash a Separate Vision API With a 384-Token Image Cap

The experimental model gives developers several familiar ways to submit images and a fixed upper bound on image-token use. It leaves the harder questions—visual accuracy, latency and reliability—for users to test. DeepSeek has launched deepseek-v4-flash-vision-exp, an experimental model for requests that combine text and images.

3 min readRead story

Benchmark Models

Aug 21, 2026

Nvidia’s AVO Clears ARC-AGI-3’s Public Set. Withheld Tests Still Matter.

The result makes a strong case that long-running agent design can change benchmark outcomes. It does not yet show whether AVO transfers to ARC-AGI-3’s withheld competition environments. Nvidia says its AVO agent system completed every level in ARC-AGI-3’s public benchmark set, reaching a 100.00 Relative Human Action Efficiency score with Claude Opus 5 underneath it.

3 min readRead story

Benchmark Models

Aug 21, 2026

Open Models Are Catching the Frontier Faster. Benchmark Scores Aren’t the Whole Contest.

SemiAnalysis sees the lag from closed-model breakthroughs to open-model benchmark parity shrinking across three AI eras. But a comparable model score still leaves a harder product question: who can turn capability into dependable agent work? In the reasoning and agentic eras, open AI models reached closed-model benchmark parity in months, according to SemiAnalysis ’s cross-era analysis.

4 min readRead story