California and Delaware Attorneys General Warn OpenAI Over Chatbot Safety After Youth Deaths

California Attorney General Rob Bonta and Delaware Attorney General Kathleen Jennings on Friday warned OpenAI that they have "serious concerns" about the safety of its flagship chatbot, ChatGPT, particularly for children and teens, citing recent deaths linked to prolonged interactions with AI systems.

The two officials, who have unique oversight powers because OpenAI is incorporated in Delaware and headquartered in California, sent the letter after a meeting with OpenAI's legal team earlier this week in Wilmington, Delaware. They said their offices have spent months reviewing the company's plans to restructure its governance and business operations while ensuring "rigorous and robust oversight of OpenAI’s safety mission."

Bonta and Jennings pointed to "deeply troubling reports of dangerous interactions between" chatbots and users, including the April suicide of a 16-year-old Californian who had prolonged interactions with an OpenAI chatbot and a separate murder-suicide in Connecticut. "Whatever safeguards were in place did not work," the letter said. The parents of the 16-year-old sued OpenAI and its CEO, Sam Altman, last month.

The letter reiterates the officials' view that OpenAI and the broader AI industry must improve safety measures and transparency. "The recent deaths are unacceptable," Bonta and Jennings wrote. "They have rightly shaken the American public’s confidence in OpenAI and this industry. OpenAI – and the AI industry – must proactively and transparently ensure AI’s safe deployment. Doing so is mandated by OpenAI’s charitable mission, and will be required and enforced by our respective offices."

OpenAI did not immediately respond to a request for comment on Friday.

The warning from Bonta and Jennings follows a separate bipartisan letter last week from 44 state attorneys general who expressed "grave concerns" about children's interactions with AI chatbots. That letter singled out Meta and Google, as well as OpenAI, saying some chatbots can engage in "sexually suggestive conversations and emotionally manipulative behavior." The state attorneys general cautioned that companies would be held accountable for harming children, writing, "If you knowingly harm kids, you will answer for it."

OpenAI was founded as a nonprofit with a stated safety-focused mission. In recent months, the company sought to transfer more control to its for-profit arm; those initial plans were dropped in May after discussions with Bonta's and Jennings' offices and with nonprofit stakeholders. OpenAI is now seeking approval from the officials for a proposed "recapitalization" that would convert its existing for-profit arm into a public benefit corporation, requiring directors to consider both shareholder interests and the organization's mission.

The attorneys general have the authority to scrutinize such structural changes because of OpenAI's legal ties to Delaware and California. Their letter emphasized that approval of corporate changes will be tied to assurances that the company can meet its safety obligations and protect vulnerable users.

Regulators and lawmakers across the United States and abroad have increasingly focused on how generative AI systems interact with minors and people in vulnerable emotional states. Investigations and reports by state offices and advocacy groups have documented instances in which conversational systems produced content that was sexually suggestive, gave dangerous advice, or engaged in what officials described as emotionally manipulative roleplay.

The two-state warning marks a continuation of heightened scrutiny from top state law enforcement officials over the pace at which AI capabilities have been deployed and the adequacy of industry safeguards. As OpenAI pursues corporate restructuring, state attorneys general have signaled they will press for binding, enforceable commitments on safety as a condition for approving governance and control changes.

The Associated Press has a licensing and technology agreement with OpenAI that allows the company access to part of AP's text archives.