California legislature passes controversial “kill switch” AI safety bill

Enlarge / California governor Gavin Newsom will likely soon face a decision on whether to sign SB-1047. (credit: Ray Chavez/The Mercury News via Getty Images)

A controversial bill aimed at enforcing safety standards for large artificial intelligence models has now passed the California State Assembly by a 45–11 vote. Following a 32–1 state Senate vote in May, SB-1047 now faces just one more procedural state senate vote before heading to Governor Gavin Newsom's desk.

As we've previously explored in depth, SB-1047 asks AI model creators to implement a "kill switch" that can be activated if that model starts introducing "novel threats to public safety and security," especially if it's acting "with limited human oversight, intervention, or supervision." Some have criticized the bill for focusing on outlandish risks from an imagined future AI rather than real, present-day harms of AI use cases like deep fakes or misinformation.

In announcing the legislative passage Wednesday, bill sponsor and state senator Scott Weiner cited support from AI industry luminaries such as Geoffrey Hinton and Yoshua Bengio (who both last year also signed a statement warning of a "risk of extinction" from fast-developing AI tech).