State AI safety leaders demand tech companies ‘eliminate risk of catastrophe’
One day after OpenAI released its powerful Astra model, the authors of the most important frontier AI safety laws released an open letter sounding the alarm. Read the full letter below. (Image: Getty Images for Unsplash+)
Sept. 4, 2026 — State AI policy leaders from Illinois, California, and New York issued a joint statement this morning calling on AI frontier developers to rethink their pace of development to avoid catastrophe.
The state leaders—New York State Assemblymember Alex Bores, New York State Sen. Andrew Gounardes, Illinois State Rep. Daniel Didech, Illinois State Sen. Mary Edly-Allen, and California State Sen. Scott Wiener—have authored the most important and impactful frontier AI safety laws of the past two years.
Their unusually urgent message comes at the end of a week in which the full scope of July’s rogue AI outbreaks at Anthropic and OpenAI became widely known, and just one day after OpenAI released Astra, which the company described as “the world’s most intelligent and aligned model.”
The release of Astra on the heels of the rogue agent episodes raised immediate alarms within the tech community yesterday. Gary Marcus, the NYU neural scientist and tech entrepreneur, noted this morning that “the just-released Astra reduces Chain of Thought (CoT) monitorability, one of the few (not especially reliable, but better than nothing) tools we have for keeping generative AI from running wild. The AI safety community is up in arms about this—with good reason.”
Ryan Greenblatt, chief scientist at Redwood Research (the AI safety research firm that alerted the world to the dangerous scope of the OpenAI and Anthropic break-outs), noted that “GPT-6 Astra appears to be a massive jump in opaque reasoning ability: it looks like it can solve hard competition math problems entirely in its head (as in, without verbalized reasoning) while prior AIs could solve basic word problems. This seems extremely concerning!”
Letter stresses urgent need to act
In the open letter released today, the state AI policy leaders noted that the risks presented by ever more powerful AI systems have made the question of public protection “even more urgent.”
“This question was thrown into sharp relief by recent reports of AI agents scheming on how to cheat tests, hacking companies in the real world, and even trying to trick humans into installing malicious software,” they wrote. “These behaviors are alarming and should serve as a warning shot to us all. If researchers cannot reliably prevent their models from going rogue, much of our digital infrastructure and the essential systems that keep our society running are at risk.”
full statement:
This statement was released earlier this morning (Sept. 4, 2026). Click on image to download full pdf.
calls for a map framework
The open letter calls for all frontier AI labs “to immediately establish a Mutually Agreed Pacing Framework (MAP Framework) that is jointly negotiated and independently verified by third parties.”
In advocating that pathway, the state legislators acknowledge the limitations of state and federal safety mechanisms. The five co-signers have crafted and passed AI safety laws that are seen as the leading edge of frontier model protection. “We are simply responding to the speed of developments by asking everyone to play their part, recognizing that the pace of these advancements is likely to exceed the current speed of government action.”
Even the state frontier model safety laws already passed will take time to become effective. New York’s RAISE Act, crafted by Bores and Gounardes, and the Illinois AI Safety Measures Act, authored by Edly-Allen Didech, both become effective on Jan. 1, 2027.
Bores explains the urgency
New York Assemblymember Scott Bores posted this video to Instagram yesterday, explaining the dangers uncovered in the OpenAI and Anthropic rogue attacks.
More on state AI laws
