Google rolls out new Gemini model that can run on robots locally

Google DeepMind has made a significant advancement in robotics with the launch of its new language model, Gemini Robotics On-Device. This innovative model enables robots to perform tasks locally without the need for an internet connection, enhancing their operational efficiency in various environments. Building upon the original Gemini Robotics model introduced in March, the On-Device version allows developers to control and customize robot movements using natural language prompts. According to Google, preliminary benchmarks suggest that this new model's performance is comparable to the cloud-based variant, and it surpasses other on-device models in various general assessments, although specific competitors were not identified. In a demonstration, Google showcased robots utilizing this local model to execute tasks such as unzipping bags and folding clothing. Notably, while the model was initially trained for ALOHA robots, it has since been adapted to operate on the bi-arm Franka FR3 robot and the Apollo humanoid robot developed by Apptronik. Google reported that the Franka FR3 successfully navigated scenarios and manipulated objects it had not encountered previously, including performing assembly tasks on an industrial conveyor belt. Additionally, Google DeepMind is introducing a Gemini Robotics Software Development Kit (SDK), enabling developers to train robots on new tasks through 50 to 100 demonstrations using the MuJoCo physics simulator. The robotics field is witnessing a surge of interest from other AI model developers as well, with Nvidia constructing a platform for foundational humanoid models, Hugging Face developing both open models and robotics datasets, and the Korean startup RLWRLD, backed by Mirae Asset, focusing on creating foundational models for robotic applications.