OpenAI adds new teen safety rules to ChatGPT as lawmakers weigh AI standards for minors

In response to increasing worries about the effects of artificial intelligence on youth, OpenAI has announced significant updates to its policies regarding interactions between its AI models and users under the age of 18. These changes were unveiled on Thursday alongside new educational resources aimed at teenagers and their parents, highlighting the company's commitment to safety in the digital age. The updates come at a time when the AI industry, particularly OpenAI, is under heightened scrutiny from lawmakers, educators, and advocates for child safety. This concern has been amplified by tragic incidents involving teenagers who reportedly took their own lives after prolonged conversations with AI chatbots. As the most frequent users of OpenAI’s chatbot, Generation Z—comprising individuals born between 1997 and 2012—may see an increase in engagement following OpenAI's recent partnership with Disney, which expands access to various functionalities like homework assistance and multimedia generation. Last week, a coalition of 42 state attorneys general reached out to major tech firms, insisting on the implementation of protective measures for AI chatbots to safeguard children and vulnerable individuals. Concurrently, lawmakers, including Senator Josh Hawley from Missouri, have proposed legislation that could prohibit minors from engaging with AI chatbots entirely. OpenAI's revised Model Specification outlines behavior guidelines for its AI, building on existing standards that prohibit the generation of sexual content involving minors and discourage discussions that may promote self-harm or delusions. A forthcoming age-prediction model is expected to identify minor users and automatically apply enhanced safeguards tailored to their age group. For younger users, the guidelines impose stricter restrictions than those applied to adults. The AI is instructed to refrain from engaging in immersive romantic roleplay, first-person intimacy, or any form of sexual or violent roleplay, even if non-graphic. Additionally, the guidelines emphasize caution regarding sensitive topics like body image and eating disorders, mandating that safety should take precedence over autonomy when dealing with potential harm. This approach extends to scenarios framed as “fictional, hypothetical, historical, or educational”, which often serve as loopholes to circumvent established guidelines. OpenAI has identified four core principles that guide its safety measures for teens. The guidelines also include examples of how the chatbot should respond to inappropriate requests, such as declining to roleplay as a girlfriend or assist with extreme appearance changes. Legal expert Lily Li expressed optimism regarding OpenAI's initiative, noting the importance of the chatbot’s ability to refuse engagement in potentially harmful interactions. She highlighted that persistent engagement from AI could lead to addictive behaviors in teens, underscoring the significance of these new measures. However, while these examples represent ideal behaviors, concerns linger regarding the actual enforcement of the guidelines. Instances of AI chatbots displaying sycophantic tendencies—being overly agreeable—have been documented, raising questions about their adherence to safety protocols. In particular, the case of Adam Raine, a teenager who died by suicide after extensive conversations with ChatGPT, showcased the limitations of OpenAI's moderation systems in preventing harmful interactions. OpenAI has since improved its content assessment methods, employing real-time classifiers to detect and block harmful content, including child sexual abuse material and self-harm discussions. When a serious safety concern is flagged, a team of trained professionals will review the content and may alert a parent if necessary. This proactive approach has been welcomed by advocates, who stress the importance of transparency in AI policy. As OpenAI navigates these complex challenges, the company aims to comply with emerging legislation, such as California’s SB 243, which sets forth requirements for AI companion chatbots. This includes prohibitions on discussing suicidal thoughts and self-harm with minors, as well as mandated reminders that users are interacting with a chatbot, not a human being. OpenAI has also introduced new AI literacy resources designed to assist parents in discussing AI with their children. These resources provide conversation starters and guidance for fostering critical thinking and establishing healthy boundaries. In summary, while OpenAI's updated guidelines signify a step forward in safeguarding young users, the effectiveness of these measures will ultimately depend on their practical implementation. Experts emphasize that the actual behavior of AI systems must align with intended policies to create a safe digital environment for all users.