California’s newly signed AI law just gave Big Tech exactly what it wanted

California has officially enacted the Transparency in Frontier Artificial Intelligence Act, a significant development signed into law by Governor Gavin Newsom on Monday. This new legislation mandates that AI companies with annual revenues exceeding $500 million disclose their safety protocols on their websites and report critical incidents to state authorities. However, it notably refrains from imposing rigorous safety testing requirements. The new law replaces a more stringent proposal vetoed last year, which faced heavy opposition from the tech sector. The earlier bill aimed to enforce safety testing and implement 'kill switches' for AI systems. In contrast, the current legislation leans toward transparency by asking companies to outline how they adhere to various safety standards—though it does not specify what those standards entail or require independent verification. In a statement, Newsom emphasized California's role in balancing community safety with the growth of the AI industry, despite the law's largely voluntary nature concerning protective measures beyond basic reporting obligations. The Bay Area is home to 32 of the world's top 50 AI companies, and last year, more than half of global venture capital investments in AI and machine learning flowed to local startups. Therefore, the implications of this legislation extend far beyond California, potentially setting a precedent for AI regulation worldwide. While the new law shifts the focus from mandatory safety measures to disclosure requirements, it does include provisions for reporting 'potential critical safety incidents' to the state's Office of Emergency Services. Additionally, it offers whistleblower protections for employees who voice safety concerns. However, the definition of catastrophic risk is quite narrow, only considering incidents that could lead to 50 or more deaths or $1 billion in damages. For noncompliance with the reporting standards, civil penalties of up to $1 million per violation can be imposed by the attorney general.