Great news for xAI: Grok is now pretty good at answering questions about Baldur’s Gate

In the competitive landscape of AI development, different labs prioritize distinct areas of expertise. While OpenAI has primarily catered to consumer needs, Anthropic has focused on enterprise solutions. Meanwhile, Elon Musk's xAI has recently turned its attention to enhancing video game walkthrough capabilities. A report from Business Insider's Grace Kay highlighted the inner workings of xAI, which was acquired by SpaceX, along with some challenges faced by its workforce. A notable incident involved a delay in a model release last year, attributed to Musk's dissatisfaction with Grok's responses to intricate questions about the popular video game, "Baldur's Gate." This led to high-level engineers being reassigned from other projects to refine the AI's answers before the launch. Such a scenario could understandably frustrate seasoned engineers who expect to engage with core issues of knowledge and machine intelligence, only to find themselves supporting a senior figure's gaming endeavors. This raises an intriguing question: Did Musk achieve the gaming insights he sought? To investigate, our RPG expert, Ram Iyer, formulated five key questions about "Baldur's Gate" and put Grok to the test alongside three major AI models, which we dubbed the BaldurBench. In the spirit of transparency, we have made the chat transcripts accessible for public viewing, showcasing Grok, ChatGPT, Claude, and Gemini. The initial findings are promising for Grok, which provided competent responses. While its language was occasionally laden with gaming terminology—using phrases like "save-scumming" and "DPS"—the information was both relevant and insightful for those familiar with the jargon. Notably, Grok displayed a penchant for structured information, favoring tables and theoretical concepts. As for stylistic differences, ChatGPT leaned towards bullet points and concise phrases, while Gemini emphasized key terms in bold. Claude, surprisingly, was particularly cautious about revealing potential spoilers, advising players to enjoy the game without stressing too much about party compositions. It's crucial to note that xAI's efforts to align Grok with its competitors in gaming knowledge were intentional. Thus, while Grok's performance may mirror that of other models, it demonstrates that xAI can rise to the occasion when it puts its efforts into developing its AI capabilities.