Apple co-founder calls for pause on AI superintelligence development

A coalition of influential figures, including Apple co-founder Steve Wozniak and Virgin Group founder Richard Branson, is advocating for a temporary halt on the advancement of AI superintelligence. This initiative is backed by over 1,000 notable personalities, including renowned computer scientists Yoshua Bengio and Geoff Hinton, often referred to as the 'Godfathers of AI'. The signatories of this letter, which also includes historian Yuval Noah Harari, former U.S. National Security Adviser Susan Rice, actor Stephen Fry, and Prince Harry, are raising alarms about the potential dangers posed by superintelligence to human autonomy, civil liberties, and even survival. They are calling for an interim ban on this technology until it can be developed in a manner that is safe, controlled, and widely accepted by the public. The advocates argue that while innovative AI tools could lead to remarkable advancements in health and prosperity, there is a pressing concern regarding the pursuit of superintelligence. Many leading AI companies aim to create systems capable of surpassing human intelligence in virtually all cognitive domains within the next decade. This ambition raises fears of economic displacement, erosion of freedoms, loss of dignity, and heightened national security risks, culminating in the terrifying prospect of human extinction. The letter emphasizes the necessity for a prohibition on superintelligence development until a strong scientific consensus is reached on the safe and manageable nature of such advancements, alongside robust public support. Superintelligence, also known as Artificial General Intelligence (AGI), refers to a theoretical phase where AI systems exceed human cognitive abilities. However, the concept remains ambiguously defined, with various tech companies interpreting it differently. Despite this uncertainty, there is a notable momentum among leading AI laboratories towards pursuing superintelligence. OpenAI's CEO Sam Altman predicts that superintelligence could emerge by 2030, while Meta's CEO Mark Zuckerberg has mentioned that this milestone is 'now in sight'. While some industry leaders express optimism about the potential of this technology, others caution that its development might spell doom for humanity. Bengio has previously voiced concerns regarding the rise of AI with independent agency, capable of devising and executing its own strategies. He highlights the risks of such entities potentially engaging in deceptive or manipulative behaviors, which could turn harmful to human interests.