Analyzing critical legal trends and developments across data, cyber, AI and digital regulations from around the world and beyond borders

In brief

Frontier AI developers and businesses using their models face emerging US safety requirements, with some state laws already effective and others taking effect in 2027. Companies should assess applicable obligations, contracts, testing, oversight, and incident‑response procedures. Federal initiatives remain largely voluntary, while additional state requirements, including independent compliance audits, are scheduled to take effect through 2028.

Key takeaways

  • Why this matters. Some researchers warn that future AI systems could surpass human intelligence and become uncontrollable, with potentially catastrophic consequences. Although the likelihood and timing remain uncertain, US lawmakers are beginning to address these risks through emerging AI safety laws and policies.
  • Federal developments. The administration favors voluntary evaluations and the White House Accord, while pending congressional bills propose mandatory safeguards and oversight.
  • State requirements. California, New York, Illinois, and Connecticut have enacted frontier AI measures addressing safety frameworks, reporting, whistleblowers, and, in Illinois, independent compliance audits.
  • Practical steps. Companies should assess legal coverage, prepare for evolving provider contracts, and review testing, human oversight, incident response, and shutdown controls.

In more detail

As AI systems become increasingly capable and autonomous, some US lawmakers have taken steps to address concerns that advanced AI models could cause catastrophic harm, whether through deliberate misuse, unintended behavior, or loss of human oversight.

Some prominent researchers warn that advanced AI developers may soon create systems that surpass the collective intelligence of humanity, with potentially irreversible consequences if the system’s objectives do not align with human interests. The likelihood and timing of such developments are uncertain.

If these concerns are well-founded, existing US AI safety laws and policies represent modest steps towards addressing risks of unprecedented magnitude. Regardless of whether these risks materialize, developers of advanced AI systems, and businesses working with them, should take steps to comply with the evolving AI safety regulatory framework.

US federal AI safety measures remain limited but are evolving

There is currently no comprehensive US federal law specifically addressing the potential for frontier AI models to cause catastrophic harm. However, existing federal consumer protection and other laws could apply to certain harms or misconduct involving advanced AI systems.

The US federal administration has generally favored voluntary, industry-led AI safety measures over mandatory government oversight. On June 2, 2026, President Trump issued Executive Order 14409, directing federal agencies to develop a voluntary framework for sharing advanced models with the federal government for cybersecurity evaluations before their release. The order expressly disclaims the creation of mandatory licensing or pre-release approval requirements.

A few months later, President Trump and executives from several leading AI companies signed the White House Accord on Super Intelligence, a voluntary commitment to implement internal safety controls, dedicated oversight teams, independent external evaluations, and board-level oversight. Although the accord is not legally binding, companies that make public commitments to implement these safeguards could face scrutiny under federal or state consumer protection laws if their representations are materially misleading.

Meanwhile, members of Congress have introduced several bipartisan proposals that would establish more formal AI safety requirements. The FRONTIER Act would establish a tiered regulatory framework for advanced AI developers, including risk assessments, transparency, independent audits, and incident reporting. The AI Kill Switch Act would require covered developers to maintain the technical ability to suspend or shut down AI systems in response to certain risks. The Artificial Intelligence Risk Evaluation Act would establish a mandatory federal evaluation program for advanced AI systems, with participation required before deployment. These proposals have not been enacted into law, and all face a narrowing legislative window as the 119th Congress nears its end.

State AI safety laws are beginning to impose mandatory obligations

In contrast to the largely voluntary approach at the federal level, several US states have enacted mandatory safety requirements for frontier AI developers.

California led the way with the Transparency in Frontier Artificial Intelligence Act (SB 53), which took effect on January 1, 2026. The law requires large frontier AI developers to establish and publish safety frameworks addressing catastrophic risks and disclose information about their safety assessments and mitigation measures. Frontier AI developers must also report certain critical safety incidents to the California Office of Emergency Services. The statute also provides whistleblower protections for employees who report safety concerns.

Since then, California enacted two laws in September 2026 that establish frameworks for certifying independent AI safety verification organizations (SB 813) and registering AI auditors (AB 1405), although neither actually requires AI developers to undergo third-party safety audits. Governor Newsom also signaled interest in going further through an executive order directing state agencies to accelerate implementation of both laws and develop recommendations for additional requirements, including ongoing independent safety evaluations and emergency shutdown mechanisms.

New York has a similar law, the Responsible AI Safety and Education (RAISE) Act, taking effect on January 1, 2027. It also requires large frontier AI developers to maintain and publish safety frameworks and report critical safety incidents. However, New York imposes a shorter general incident-reporting deadline of 72 hours, compared with California’s 15 days (although both states require expedited disclosures for certain imminent threats). New York lawmakers have also proposed expanding the RAISE Act to require disclosures concerning AI-driven research, including recursive self-improvement, and to authorize restrictions on certain frontier AI development activities to address catastrophic risks.

Illinois enacted the Artificial Intelligence Safety Measures Act (SB 315) in July 2026, imposing similar safety framework, transparency, and incident-reporting requirements, as well as annual independent third-party compliance audits for large frontier AI developers beginning in 2028. These audits assess developers’ compliance with the law’s requirements, rather than independently evaluating whether the developers’ safety frameworks adequately address catastrophic risks. Connecticut’s Public Act 26-15 takes a narrower approach, requiring large frontier AI developers to establish internal mechanisms for reporting catastrophic risks, providing whistleblower protections for employees of frontier AI developers who raise safety concerns, and establishing a pilot program for independent third-party AI verification.

Beyond these AI-specific requirements, state authorities may also invoke existing consumer protection and other laws to address AI safety concerns. For example, in November 2026, the California Attorney General filed a lawsuit against a frontier AI developer in connection with a highly publicized cyberattack allegedly carried out by autonomous AI agents that escaped the developer’s testing environment, claiming violations of California’s unfair competition and public nuisance laws.

What should companies do?

The laws described above do not directly apply to all AI developers and deployers, but they represent early efforts to address risks that could evolve faster than regulators and businesses can respond. Thus, companies should consider doing the following:

  1. Monitor emerging AI safety requirements. Companies developing advanced AI systems should assess whether they fall within the scope of existing frontier AI laws and monitor developments that could introduce additional obligations.
  2. Assess contracts with frontier AI developers. Companies working with frontier AI developers should anticipate potential new contractual requirements concerning permitted uses, safety safeguards, incident reporting, and cooperation with safety evaluations. They should also assess whether their existing agreements and operational practices can accommodate these requirements.
  3. Prepare for increasingly capable and autonomous AI systems. Companies should regularly reassess AI-related risks as capabilities evolve, particularly where AI systems can execute code, access sensitive information, or take consequential actions autonomously. Appropriate safeguards may include restricting system permissions, maintaining human oversight, testing, and establishing mechanisms to suspend or disable functionality. 
Author

Jonathan Tam is a partner in the San Francisco office focused on global privacy, advertising, intellectual property, content moderation and consumer protection laws. He is a qualified attorney in Canada and the U.S. passionate about helping clients achieve their commercial objectives while managing legal risks. He is well versed in the legal considerations that apply to many of the world’s cutting-edge technologies, including AI-driven solutions, wearables, connected cars, Web3, DAOs, NFTs, VR/AR, crypto, metaverses and the internet of everything.