Global movement to protect kids online fuels a wave of AI safety tech

A significant global initiative focused on online safety is driving the development of artificial intelligence tools aimed at protecting children from harmful online content. In the U.K., the recently enacted Online Safety Act mandates that technology companies have a duty to shield minors from inappropriate material, hate speech, bullying, fraud, and child sexual abuse material (CSAM). Companies that fail to comply with this legislation could face fines amounting to 10% of their global annual revenues for violations. Meanwhile, similar protective measures are gaining traction in the U.S. Congress, with the Kids Online Safety Act proposing that social media platforms be held accountable for preventing their services from endangering children, akin to the U.K.'s Online Safety Act. This regulatory push has prompted major tech firms to rethink their safety protocols. Notably, Pornhub and other adult content sites are now requiring users to verify their ages before accessing their platforms. This shift isn't limited to adult websites; other major platforms like Spotify, Reddit, and X have implemented age verification systems to safeguard minors from inappropriate content. However, these measures have sparked debate within the tech industry, with concerns raised about potential privacy infringements. Central to the implementation of age verification technologies is Yoti, a company specializing in AI-driven identity verification. Yoti's technology analyzes selfies to determine a person's age based on facial features. The firm claims that its AI can accurately estimate ages of individuals between 13 and 24 within a two-year margin of error. Yoti has previously collaborated with the U.K. Post Office and is poised to benefit from the increasing demand for government-issued digital ID cards. Despite Yoti's prominence, it faces competition from other identity verification firms like Entrust, Persona, and iProov. "There is a competitive rush in the child safety technology sector to gain trust and recognition," noted Pete Kenyon, a partner at law firm Cripps. He emphasized that the new regulations have opened up a fresh marketplace, prompting service providers to establish their presence. Yet, the surge in digital identification technology raises significant privacy concerns. Kenyon pointed out that substantial privacy issues can arise from these systems, highlighting the necessity for stringent governance and technical measures to protect personal data. Rani Govender, policy manager for child safety at NSPCC, asserted that viable technologies already exist that can authenticate users without compromising their privacy. She urged tech companies to make ethical choices that prioritize child safety without infringing on user privacy. In addition to software advancements, new hardware solutions are also emerging. This month, Finnish company HMD Global introduced the Fusion X1 smartphone, which employs AI to prevent children from recording or sharing nude images or accessing sexually explicit content. This technology is powered by SafeToNet, a U.K.-based cybersecurity firm dedicated to child safety. James Robinson, HMD's vice president for family verticals, emphasized that the idea for these child-friendly devices was conceived before the Online Safety Act was enacted, but he welcomed the government's proactive steps in this direction. As the NSPCC's Govender noted, ensuring child safety will increasingly become a priority for major tech companies like Google and Meta, especially in light of growing concerns about the impact of online bullying and social media addiction on youth mental health. "The era of neglect must end," she concluded, calling for greater accountability from tech giants.