Reddit blocks Internet Archive to end sneaky AI scraping

In a significant move, Reddit has implemented restrictions on the Internet Archive (IA), halting its ability to index popular Reddit threads. This decision comes after the discovery that certain AI companies, which are prohibited from scraping Reddit, have been circumventing these rules by extracting data from IA's archived materials. Previously, IA's Wayback Machine served as a reliable resource for archiving Reddit pages, profiles, and comments, aligning with its mission to preserve the Internet. However, going forward, the archive will only capture screenshots of the Reddit homepage. As highlighted by The Verge, this change limits the archive's utility, transforming it into a mere snapshot of trending posts and headlines rather than a comprehensive repository of deleted content or insights into various Reddit communities. While Reddit has not disclosed the identities of the AI companies involved in this data scraping, spokesperson Tim Rathschmidt confirmed to Ars that the platform has observed violations of its policies involving AI firms. He hinted at potential measures that the Internet Archive could adopt to enhance its defenses against such scraping activities. This development coincides with Reddit's broader initiative to tackle longstanding privacy issues, as the platform argues that the Wayback Machine's practice of archiving deleted content poses significant concerns. As a result, Reddit plans to intensify its restrictions on scraping activities starting today.