Researchers find what makes AI chatbots politically persuasive

In a groundbreaking investigation, researchers from the UK AI Security Institute, along with teams from MIT, Stanford, and Carnegie Mellon, sought to understand the persuasive power of conversational AI systems. This extensive study, which involved nearly 80,000 participants across the UK, aimed to determine whether large language models (LLMs) could effectively sway public opinion on political matters. Two years ago, Sam Altman, a prominent figure in AI development, predicted that AI would achieve superhuman levels of persuasion prior to reaching general intelligence. This forecast raised significant concerns regarding AI's potential impact on democratic processes. However, the recent findings reveal that political AI chatbots do not possess the persuasive abilities once anticipated. The research delved into the capabilities of 19 different LLMs, including notable versions of ChatGPT and xAI's Grok-3 beta, among others. These AI systems were tasked with advocating for or against various positions on 707 selected political issues. Participants, who were recruited via a crowdsourcing platform, engaged in brief conversations with the AIs and then rated their agreement with specific stances on a scale of 1 to 100 before and after the interaction. While the study found that AI chatbots fell short of the expected superhuman persuasion, it opened up discussions about the nuances of human-AI interactions. With access to vast amounts of information and potentially personal data from numerous online interactions, the prospect of conversing with a powerful AI can be intimidating. This research aims to dissect the perceived dangers and clarify the actual influence these systems hold in political discourse.