Inside the old church where one trillion webpages are being saved

Nestled close to San Francisco’s iconic Presidio, a striking white building with eight gothic columns now serves as a crucial repository for Internet history — the Internet Archive. This non-profit library, managed by a dedicated team of software engineers and librarians, has been diligently preserving the web for nearly three decades, capturing it page by page. Inside this former church, the tranquility of sermons has given way to the hum of servers. The Internet Archive’s renowned Wayback Machine, a tool utilized by millions daily, plays an essential role for researchers and journalists seeking historical content that may have vanished or altered online. Last month, this remarkable service celebrated the preservation of its trillionth webpage, highlighting the increasing significance of archiving in today’s rapidly changing digital landscape. With recent government directives resulting in the removal of extensive online content, and the rise of artificial intelligence complicating access to information, the challenge of preserving digital history has never been more daunting. Brewster Kahle, the founder of the Internet Archive, emphasizes the importance of documenting the past to help shape a better future. He founded the archive in 1996 when data storage was minimal, and now the organization saves approximately 150 terabytes of data daily, a feat that reflects the exponential growth of the Internet. Kahle’s vision for the archive is inspired by the ancient Library of Alexandria, aiming to collect and preserve all human knowledge. The Wayback Machine not only captures web pages but also retains their underlying code, ensuring that users can access the original structure of sites even if the original servers become obsolete. As artificial intelligence and chatbots become prevalent, the Internet Archive is adapting its methods. It now includes AI-generated content, documenting interactions with chatbots and preserving the evolving nature of online information consumption. The team is committed to maintaining a diverse record of digital news, actively capturing various queries and their outcomes. In the face of political pressures, Kahle and his team focus on ensuring that multiple perspectives remain accessible. They have witnessed significant shifts in online content due to various administrations, with the archive serving as a critical resource for understanding these changes. Kahle notes that while some sections of the web may disappear, the archive stands as a testament to the importance of maintaining a comprehensive record of online history. The Internet Archive’s main servers are housed in a large facility outside San Francisco, while a symbolic set of servers resides in the main hall of the former church, a visual reminder of their mission. Inside, a dedicated staff of around 200 engages in various forms of media preservation, from digitizing books to streaming their work live on platforms like YouTube. The atmosphere at the Internet Archive is vibrant and community-oriented, reflecting the passion of its staff. Kahle describes the organization as a resource for others to build upon, rather than a gatekeeper of truth. Their goal is clear: to ensure that the vast history of the web remains accessible for future generations, empowering individuals to create and innovate based on the wealth of knowledge preserved within their digital sanctuary.