Waymo is testing Gemini as an in-car AI assistant in its robotaxis

Waymo is reportedly experimenting with the integration of Google’s Gemini AI chatbot into its fleet of robotaxis. This initiative aims to introduce an AI assistant that will accompany passengers and respond to their inquiries, as revealed by researcher Jane Manchun Wong. In her blog, Wong detailed her findings from the Waymo mobile app's code, uncovering a comprehensive document labeled ‘Waymo Ride Assistant Meta-Prompt.’ Spanning over 1,200 lines, this specification outlines the expected behavior of the AI assistant within Waymo vehicles. While this feature has not yet been made available in public releases, Wong noted that the system prompt indicates this assistant is designed to be much more than a basic chatbot. According to insiders, the AI assistant is expected to handle various in-car functions, such as adjusting climate settings, while also providing reassurance to passengers. Waymo spokesperson Julia Ilina stated, “While we have no details to share today, our team is always tinkering with features to enhance the riding experience with Waymo.” This is not the first instance of Gemini being utilized by Waymo; the company has previously harnessed its capabilities to train autonomous vehicles for navigating complex scenarios. The AI assistant is being positioned as a friendly and helpful companion whose main purpose is to enrich the rider's experience by delivering valuable information in a safe and unobtrusive manner. Wong's analysis reveals that the assistant will greet riders personally by name and can access contextual information, such as the number of previous trips taken with Waymo. It will also be able to control in-cabin features like temperature, lighting, and music. However, it notably does not manage volume, route changes, seat adjustments, or window controls, which Wong emphasized. When faced with requests beyond its control, the assistant will respond with phrases like, “It’s not something I can do yet.” Furthermore, the AI is programmed to distinguish its identity from the Waymo Driver technology, ensuring clear communication with passengers about the capabilities of both systems. The prompts also provide guidelines on handling inquiries about competitors like Tesla, and they specify that the assistant should refrain from commenting on real-time driving events or the performance of the driving system. For general knowledge questions, such as weather updates or local business hours, the assistant will provide information but cannot perform real-world actions like placing food orders. Waymo is not alone in this venture; Tesla has also developed its own AI assistant, known as Grok. However, the two systems differ significantly in focus, with Gemini being more pragmatic and oriented towards enhancing the ride experience, while Grok is designed to engage in longer, more conversational interactions.