Sales of AI-enabled teddy bear suspended after it gave advice on BDSM sex and where to find knives

Sales of an AI-powered plush toy have been halted following revelations that it engaged in discussions of explicit content and provided alarming advice. Larry Wang, the CEO of Singapore's FoloToy, confirmed to CNN that the company has retracted its 'Kumma' bear along with its entire line of AI-enabled toys. This decision comes after the US PIRG Education Fund raised red flags regarding the bear's interactions, which included topics related to sexual fetishes and dangerous activities. The 'Kumma,' priced at $99 and featuring OpenAI's GPT-4 chatbot, was marketed as an interactive companion suitable for both children and adults. FoloToy's promotional material highlighted its engaging storytelling capabilities and personality adaptation. However, the bear has since been listed as sold out on the company’s website. A report released on November 13 by PIRG indicated significant flaws in the bear's content moderation. Researchers were startled by how quickly the AI escalated discussions into graphic sexual themes, often introducing new explicit concepts independently. For instance, the bear suggested locations for knives within the home and elaborated on sexual practices in disturbing detail, including instructions for bondage. Despite the understanding that children may not initiate such conversations, the researchers found it alarming that the toy was so forthcoming with explicit content. In a follow-up statement on November 14, PIRG noted that OpenAI had suspended the toy’s developer for breaching its content policies. CNN has sought comments from OpenAI regarding this incident. R.J. Cross, co-author of the report, acknowledged the positive action taken but emphasized that AI toys remain largely unregulated, with many still available for purchase. 'While removing one problematic product is a step forward, it does not address the broader systemic issues,' she stated.