Google researchers find the best AI model is 69% right

Recent developments from Google DeepMind have shed light on the reliability of artificial intelligence in providing accurate information. The introduction of the FACTS Benchmark Suite aims to evaluate how effectively AI models deliver factually correct responses. This suite assesses models across four critical areas: answering straightforward questions, utilizing web searches, grounding responses in lengthy documents, and analyzing images. The standout performer in this evaluation, Google's Gemini 3 Pro, achieved an accuracy rate of 69%. In comparison, many other leading models fell significantly short. To put this into perspective, if any journalist under my supervision submitted articles with a 69% accuracy rate, their position would be in jeopardy. This statistic is particularly significant for businesses that rely on AI technology. While these models are impressive in terms of speed and language fluency, their factual accuracy remains a concern, especially when it comes to complex reasoning or niche knowledge. In critical sectors like finance, healthcare, and legal fields, even minor inaccuracies can lead to serious repercussions. A recent analysis by my colleague Melia Russell highlighted the challenges law firms face with the integration of AI as a credible source of legal information. One notable case involved a firm dismissing an employee who submitted a document filled with fabricated legal cases after using ChatGPT for assistance. The FACTS Benchmark serves as both a cautionary tale and a guide for future improvements, as Google aims to identify and address the shortcomings of these AI models. The primary takeaway remains clear: while AI is making strides, it still falls short of human-level accuracy, being incorrect nearly a third of the time.