Together AI Adds Three Model Rollout Paths; Canary Pauses on Metric Regressions

Together AI has introduced rollout controls that move traffic between model deployments on the same inference endpoint. The new feature gives teams three ways to make a production change, but only its canary path can halt between traffic steps when live metrics show a regression.

The safety check belongs to one path

A rollout transfers requests from a source deployment to a target deployment. Together says applications can keep calling a stable endpoint while multiple deployments operate behind it, separating an infrastructure change from an application update.

The distinction is consequential. Canary releases shift traffic through operator-defined stages and can check router error rate, router latency, or in-flight requests after each stage. Blue-green and rolling releases still run health checks, but they do not have canary-style metric gates between traffic steps.

The measured route found a problem early

Together’s documented Qwen2.5-7B-to-Qwen3.5-9B canary found a 137% p95 latency regression when the target was handling 10% of traffic. The company says it canceled and reversed the rollout while live requests continued without failures. It is a company example, not evidence of typical rollout results.

The documented canary setup

• Traffic moved through 10%, 50%, and 100% stages, with 600-second intervals.
• The p95 latency measurement window was 300 seconds.

Pausing still leaves a decision to make

A failed metric gate, insufficient capacity, or missing metrics can place a rollout in SYSTEM_PAUSED. Operators can resume or cancel it. Canceling freezes the traffic split rather than automatically restoring the old deployment; returning traffic requires a new rollout with source and target swapped.