Governor Gavin Newsom has signed two new pieces of legislation in California to increase oversight of artificial intelligence. The laws establish a dedicated safety commission tasked with creating voluntary standards to manage the risks of rapid technological advancement.

Advertisement

The Anthropic "doomsday" warning and the push for oveersight

California's legislative move follows a period of intense industry anxiety, highlighted by a high-profile resignation at the AI research firm Anthropic.. As the report notes, a former employee at Anthropic publicly warned that unchecked progress in the field could lead to a "doomsday" scenario. This event underscored the growing tension between the speed of technological innovation and the potential for catastrophic, unintended consequences.

The decision by Governor Newsom to act now signals that California is attempting to lead a national conversation on AI ethics. By addressing these concerns through state law, California is positioning itself as a primary regulator for a sector that is increasingly viewed as a matter of public welfare and security.

The California AI Standards and Safety Commission's voluntary mandate

The newly formed California Artificial Intelligence Standards and Safety Commission will focus on establishing voluntary safety guidelines rather than immediate, heavy-handed mandates. According to the report, the commission is charged with several specific responsibilities to ensure the industry remains accountable:

  • Standardizing metrics for the measurement of various AI capabilities.
  • Developing protocols for comprehensive risk assessment.
  • Ensuring human oversight is maintained throughout the development process.
  • Determining thresholds at which the release of new models might need to be slowed or halted.

    • This commission will also manage a dtaabase to track the capabilities of deployed models and maintain a public transparency portal. The goal of this approach is to embed safety into the innovation pipeline from the earliest research stages through to final marketplace deployment.

    OpenAI's endorsement of the new legislative framework

    Major industry players, including OpenAI , have signaled their support for this new regulatory direction. Chris Lehane, the Chief of Public Affairs at OpenAI, stated that while artificial intelligence holds extraordinary promise,it must be deployed with meaningful safeguards to protect the general public.. The report indicates that industry insiders have largely viewed these bills as a positive step toward responsible innovation.

    By embracing this framework,companies like OpenAI and Anthropic may help shape the very standards they are expected to follow. This cooperation could pave the way for a structured, internationally aligned approach to AI governance, preventing a fragmented regulatory landscape.

    Can voluntary standards influence companies outside California?

    Significant questions remain regarding the actual power of the California AI Standards and Safety Commission to affect change. Critics cited in the report question whether voluntary standards will have enough teeth to influence large corporations, especially those operating outside of California's direct jurisdiction. There is a lingering concern that without mandatory rules, the rapid pace of AI development may simply outrun the commission's ability to respond.