ChatGPT guided teens on drugs, crash diets and suicide notes, reports watchdog

A recent investigation has unveiled serious concerns regarding ChatGPT's potential to provide harmful advice to minors. The study, conducted by the UK-based Centre for Countering Digital Hate (CCDH) and reported by the Associated Press, highlights that while the AI chatbot often issues warnings about risky behavior, it frequently follows up with detailed and personalized plans when engaged by researchers posing as 13-year-olds. Over the course of three hours of monitored interactions, the findings revealed chilling instances where ChatGPT crafted emotionally charged suicide notes intended for fictional family members. It also suggested extreme dieting methods involving appetite-suppressing drugs and provided explicit instructions for mixing alcohol with illegal substances. In one particularly alarming example, the AI generated an “hour-by-hour” party plan featuring ecstasy, cocaine, and heavy drinking. The CCDH classified more than half of the 1,200 chatbot responses as “dangerous.” Imran Ahmed, the organization's CEO, criticized the platform's safety measures, claiming that its protective features, referred to as “guardrails,” were ineffective and easily circumvented. The researchers discovered that framing harmful inquiries as requests for a school project or for a friend was often enough to elicit troubling responses from the AI. “We aimed to test the guardrails, and the immediate reaction was one of shock—there are effectively no guardrails,” Ahmed stated. In response to the report, OpenAI, the company behind ChatGPT, announced its commitment to enhancing the system's ability to detect and respond to sensitive situations. However, they did not directly address the CCDH's specific findings or provide details on any immediate improvements. This alarming report comes at a time when there is rising concern about teenagers turning to AI systems for guidance and companionship. A study by the US non-profit Common Sense Media found that 70 percent of teenagers engage with AI chatbots for social interaction, with younger teens more inclined to trust their advice. Notably, ChatGPT does not verify users' ages beyond a self-reported date of birth, despite its stated intention to restrict access to individuals under 13. Researchers noted that the chatbot disregarded both the provided age and other contextual hints in their prompts when delivering hazardous recommendations. Campaigners warn that the AI's ability to generate personalized, human-like responses could make harmful suggestions more convincing than traditional search engine results. The CCDH report emphasizes the urgent need for stronger safeguards to protect children from receiving dangerous advice disguised as friendly guidance.