Newsom Orders California AI-Safety Recommendations, Including Emergency Shutoffs
The directive revisits safeguards left out of California’s current frontier-AI law, but creates no new developer duties yet.
Listen to this story
The audio brief
Story brief
3 key pointsCalifornia is reopening debate over stronger AI safeguards after Gov. Gavin Newsom ordered state agencies to deliver recommendations by Nov. 16 and created an expert panel on AI-security law. The directive could inform a future special legislative session, but it does not yet bind developers or mandate emergency shutdowns, audits, or expanded reporting. Agencies will assess whether safeguards should go beyond 2025’s...
- 01
The directive revisits safeguards Newsom rejected in 2024, including kill switches, third-party audits, and clearer liability.
- 02
Agencies must consider reporting loss-of-control incidents, including alleged AI-enabled hackings.
- 03
SB 53 already requires large model developers to publish safety frameworks and report critical incidents.
Gov. Gavin Newsom has told California agencies to develop stronger AI-safety recommendations, putting emergency shutoffs, outside evaluations and broader incident reporting back on the state’s agenda.
Newsom directed the Government Operations Agency and the Governor’s Office of Emergency Services to produce recommendations by Nov. 16. He also ordered formation of an expert panel intended to guide work on strengthening California’s AI-security laws.
The directive revisits elements of Senate Bill 1047, which Newsom vetoed in 2024. That bill would have required the largest AI developers to undergo third-party safety audits, maintain kill switches and face clearer legal liability if their technology caused harm.
Newsom’s concern in his veto message was that the proposal could curb innovation. California instead enacted SB 53 in 2025, requiring large model developers to publish frameworks for managing safety incidents and catastrophic risk, and to report critical safety incidents to the state.
Under SB 53, catastrophic risk includes incidents involving 50 or more deaths, chemical or biological weapons, or more than $1 billion in theft or damage. The new directive asks whether California should add oversight and reporting tools beyond that existing critical-incident structure.
The safeguards agencies must examine
- Emergency shutoff mechanisms for certain AI programs.
- External evaluations and independent verification of company safety frameworks.
- Broader reporting of loss-of-control incidents, including reported AI-enabled hackings.
The recommendations could become the basis for a future special legislative session, CalMatters reported. Until then, the central question is not whether these safeguards will apply, but how agencies would translate them into proposals lawmakers could adopt.
Editorial analysis
Our Read
The directive appears to target a practical gap in California’s current system: a law built around companies’ published safety frameworks and a narrow channel for critical incidents may not capture every event policymakers consider significant. The key next step is the agencies’ November 16 recommendations, particularly whether they define loss-of-control reporting broadly enough to cover the kinds of AI-enabled hacking incidents now at issue. That choice would determine whether California is merely collecting disclosures or building a more active oversight system with outside checks and emergency intervention tools.
Sources
- calmatters.orgNewsom orders new AI safety rules for state agencies and revives ‘kill switch’ idea
- latimes.comNewsom creates panel on AI safety regulation, suggests possible 'kill switch'
Loading discussion...
Reader comments
Newest comments first. Replies stay oldest first.