Gemini Robotics On-Device enables smarter robots without the cloud

Google DeepMind has unveiled a groundbreaking artificial intelligence model designed specifically for robotics. Dubbed Gemini Robotics On-Device, this innovative system functions entirely on local devices, eliminating the need for a constant internet connection. This advanced model empowers bi-arm robots to execute intricate tasks in real-world settings by integrating voice, language, and action (VLA) processing. Carolina Parada, Senior Director and Head of Robotics at Google DeepMind, shared insights into the model's low-latency performance and adaptability in a recent blog post. Its independence from cloud services makes it ideal for environments where immediate responsiveness is critical. Currently, access to Gemini Robotics On-Device is limited to participants in Google’s trusted tester program. Developers are encouraged to explore the capabilities of this AI through the Gemini Robotics software development kit (SDK) and the MuJoCo physics simulator. Although Google has not disclosed detailed information about the architecture or training methods of this model, it has emphasized its powerful features. Designed for bi-arm robotic systems, Gemini Robotics On-Device is remarkably efficient, requiring minimal computing power. It can learn new tasks with just 50 to 100 demonstrations, greatly expediting its deployment across various applications. In internal evaluations, the model has proven its ability to understand natural language commands and complete a diverse range of complex tasks, including folding laundry, unzipping bags, and manipulating unfamiliar objects. Its precision extends to tasks typical in industrial settings, demonstrating impressive dexterity. Initially trained on ALOHA robotic systems, Gemini Robotics On-Device has also been successfully adapted for use with other bi-arm robots, such as Franka Emika’s FR3 and Apptronik’s Apollo humanoid robot. According to the tech giant, the model maintained consistent performance across different platforms, even when faced with unfamiliar tasks or multi-step instructions.