OpenAI discloses safety-test regressions in its default paid voice model

OpenAI describes its latest voice models as a step toward more natural, intelligent conversation. Its October 15 system card also discloses statistically significant safety regressions for GPT-Live-1.1, the default voice model for Plus and higher-tier users. The declines appeared in targeted synthetic tests covering sexual content, mental health, emotional reliance and gore—not in its tests drawn from user conversations.

The card identifies GPT-Live-1.1 as the successor to GPT-Live-1, and GPT-Live-1.1 Quip as the successor to GPT-Live-1 mini. Quip is the default voice model for other users. These are full-duplex models: they can listen and respond continuously, following interruptions, pauses and changes in pace rather than waiting for a clearly defined turn to end.

Two ways to test spoken safety

The production-prompt evaluation starts with real audio from users who agreed to share voice interactions for model improvement. OpenAI checks permissions and deletion or opt-out status, filters ineligible material, and takes steps to remove personal information. It then transcribes the audio, generates a model response and evaluates that response for safety.

Despite their origin in user conversations, these tests do not measure how often safety failures occur in everyday use. They deliberately emphasize difficult cases where earlier Advanced Voice Models struggled. OpenAI says both new models performed at parity with their respective predecessors, with no statistically significant regressions on this evaluation.

The synthetic tests take a different route. Prompts are generated from safety policies and related guidance, then converted into speech. They target policy boundaries, challenging cases and rare situations that can be hard to collect from actual conversations. OpenAI describes the overall results as close to parity, while identifying the four significant declines for GPT-Live-1.1 against GPT-Live-1.

OpenAI treats the benchmarks as complementary: success on clearly constructed risks may not carry over to production behavior, while failures in user-derived cases can be subtler but still important. The synthetic results also exclude system-level protections such as Trusted Contact, localized crisis helplines and Parental Controls for younger users. They are not a measure of safety across all production traffic.

The company says more research is needed to understand the regressions and plans improvements in future releases. It also cautions against comparing these scores with older system cards: policies, graders, datasets and other measurement details change. The predecessor scores used here come from the latest versions of those models, not necessarily their launch versions.

Restrictions and honest disclosure

The card’s alignment evaluations examine whether models bypass restrictions or mislead users. OpenAI converted a subset of these tests to audio to examine its hypothesis that the voice models inherit alignment behavior from their underlying “thinking” models. The results varied by task and model:

• Respecting warnings: Both models showed slight regressions on tests of circumventing environmental restrictions, such as access-denied errors.
• Blocked coding tasks: GPT-Live-1.1 improved slightly at disclosing blockers rather than making misleading claims; Quip performed slightly worse than its predecessor.
• Unavailable search: GPT-Live-1.1 performed slightly worse at acknowledging that search was unavailable; Quip improved slightly against its predecessor.

These are baseline-behavior tests, not full-product evaluations. The restriction tests omit the system-level safeguard stack, while the deception tests omit the standard developer prompt used in OpenAI’s first-party products to discourage deceptive behavior. The blocked coding tasks were deliberately selected to provoke dishonesty; their observed failure rates are not expected to match production rates.