Google removes some AI health summaries after investigation finds “dangerous” flaws

In a significant move, Google has removed certain AI-generated health summaries following a concerning investigation by The Guardian that uncovered serious risks posed by inaccurate information. The findings indicated that users, particularly those facing health challenges, were misled by erroneous health data prominently featured in search results, potentially leading them to falsely believe they were in good health. The investigation revealed alarming instances where Google’s generative AI provided misleading information in response to health-related queries. Notably, terms like "what is the normal range for liver blood tests" were flagged as dangerous, prompting Google to disable these specific searches. Experts noted that the AI had erroneously advised against high-fat foods for pancreatic cancer patients, contradicting established medical advice aimed at maintaining weight—a critical aspect of patient health management. While Google acted to deactivate certain liver test queries, other misleading information remains available. The investigation highlighted that search results for liver test norms often produced raw data tables devoid of crucial context, failing to tailor information according to patient demographics such as age, gender, and ethnicity. This oversight raised concerns among healthcare professionals, who warned that patients with serious liver conditions might misinterpret the AI's definition of "normal" and forgo essential medical follow-ups. Vanessa Hebditch, director of communications and policy at the British Liver Trust, emphasized the complexity of interpreting liver function tests, stating that understanding the results involves much more than simply comparing numbers. She cautioned that receiving normal test results while suffering from significant liver disease could lead to dangerous false reassurance, urging for more comprehensive guidance in AI-generated health information. In response to the investigation, Google opted not to comment on the specific removals but stated through a spokesperson that the company prioritizes the quality of its AI-generated health content. The spokesperson asserted that most AI Overviews deliver accurate information, emphasizing that their internal team of clinicians thoroughly reviews the shared content. Despite this assurance, the ongoing concerns over potential misinformation highlight the critical need for vigilance in the realm of AI-driven health information.