Full archive

Models AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-11 of 11Results matching "Agent harness"

Earlier this week

3 stories

YesterdayModels

Benchmark Models

Grok 4.6 Clears 50% on Biosecurity Refusal and Routine Biology Tasks

The reported result rewards a difficult balance: recognizing hazards hidden inside ordinary-looking research requests without shutting down legitimate biological work. Its surveillance score shows that refusal performance is not the whole biosecurity test.

3 min

Aug 31, 2026Models

Benchmark Models

Qwen3.8-Max Leads Open-Weight Commerce Agent Bench by Two Passes, With Audit Limits

The benchmark checks whether agents leave commerce software and files in the required state, rather than simply narrate a workflow. But configuration choices and unavailable immutable result bundles limit what the narrow ranking can settle.

2 min

Aug 31, 2026Models

Open release Models

DeepSeek Releases 305B Vision Weights Under MIT License, but Local Use Needs Serious Hardware

The release gives developers the model files and serving references needed to run or adapt DeepSeek-V4-Flash-Vision-Exp independently. Its 168GB size means that freedom is aimed more at infrastructure-equipped teams than typical individual users.

3 min

Last week

4 stories

Aug 28, 2026Models

Research Models

Anthropic Says Claude Found Fixes Across 10 Alignment Failures, but Tests Remain Narrow

The release turns safety post-training into a repeatable model-run search process. Its value now depends on whether those benchmark gains survive broader tests and later training.

3 min

Aug 28, 2026Models

Benchmark Models

Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near Claude Opus 4.5

The result is a strong benchmark showing for learned state management, not yet evidence that the framework improves the long-running enterprise workflows it is designed to target.

3 min

Aug 25, 2026Models

Launch Models

DeepSeek V4 Pro Lands on Fireworks With $2.50 CyberGym Solves, but Kimi K3 Scores Higher

The provider’s results frame security-agent selection as a three-way choice among raw solve rate, cost per completed task, and whether a model will reliably execute the required tool workflow.

3 min

Aug 24, 2026Models

Benchmark Models

NVIDIA Says AVO Took Claude Opus 5 From 30% to Perfect on ARC-AGI-3

The result shifts attention from training larger models to the software that gives them memory, tools and recovery loops. Its clearest limitation is equally important: the perfect score came on ARC-AGI-3’s public environments, while private sets remain the harder test.

3 min

August 2026

4 stories

Aug 22, 2026Models

Benchmark Models

Stanford Won Databricks’ Agent Cup, but 18.8% of Questions Stumped Every Team

The live contest tested whether agents developed on one benchmark could handle an unfamiliar Treasury archive. Stanford led the field, but the unsolved questions show the limits of reliable document reasoning.

2 min

Aug 22, 2026Models

Launch Models

DeepSeek Gives V4-Flash a Separate Vision API With a 384-Token Image Cap

The experimental model gives developers several familiar ways to submit images and a fixed upper bound on image-token use. It leaves the harder questions—visual accuracy, latency and reliability—for users to test.

3 min

Aug 21, 2026Models

Benchmark Models

Nvidia’s AVO Clears ARC-AGI-3’s Public Set. Withheld Tests Still Matter.

The result makes a strong case that long-running agent design can change benchmark outcomes. It does not yet show whether AVO transfers to ARC-AGI-3’s withheld competition environments.

3 min

Aug 21, 2026Models

Benchmark Models

Open Models Are Catching the Frontier Faster. Benchmark Scores Aren’t the Whole Contest.

SemiAnalysis sees the lag from closed-model breakthroughs to open-model benchmark parity shrinking across three AI eras. But a comparable model score still leaves a harder product question: who can turn capability into dependable agent work?

4 min