Caterpillar taps Nvidia to bring AI to its construction equipment

Caterpillar is making significant strides in integrating artificial intelligence and automation into its range of construction machinery through a collaboration with Nvidia, a leading semiconductor company. The partnership is currently showcasing an AI-assisted system in its mid-size Cat 306 CR Mini Excavator, known as 'Cat AI.' This innovative system, developed on Nvidia’s Jetson Thor AI platform, is being presented at CES this week. Brandon Hootman, Caterpillar's Vice President of Data and AI, explained to TechCrunch that Cat AI is designed to function as a fleet of AI agents. It can assist machine operators by answering queries, providing access to resources, delivering safety advice, and facilitating service scheduling. The introduction of this technology aims to transform how operators interact with their machines, particularly by enabling them to receive critical insights without needing to step away from the job site. "Our customers don’t live in front of a laptop day in and day out; they live in the dirt," Hootman stated, highlighting the importance of on-site data utilization. Additionally, Caterpillar is experimenting with digital twin technology for construction sites, utilizing Nvidia’s Omniverse simulation resources to optimize scheduling and project material requirements. The data collected from Caterpillar’s equipment, which generates around 2,000 messages back to the company every second, will play a crucial role in enhancing these simulations. Already a leader with fully autonomous vehicles in the mining sector, Caterpillar views these pilot programs as a significant advancement toward increased automation in its operations. Hootman noted, "The reason that we started here was it was a real challenge of our customers today that needed to be addressed, and also something that we had some real momentum on and we felt like we could bring to market pretty quickly." This collaboration aligns perfectly with Nvidia's strategy in the realm of physical AI. Bill Dally, Nvidia’s Chief Scientist, shared insights on the company's commitment to exploring this next frontier with its powerful GPUs. During its CES keynote, Nvidia outlined its vision for a comprehensive ecosystem supporting physical AI, which includes open AI models, simulation tools, and developer kits. While some may perceive physical AI as primarily beneficial for robotics, Deepu Talla, Vice President of Robotics and Edge AI at Nvidia, emphasized the broader implications of this technology, stating that it encompasses a wide range of applications, not limited to robotics alone. "Physical AI is the next wave of AI," Talla asserted, reflecting Nvidia's goal to lead in this innovative field by offering the necessary tools to train, simulate, and deploy AI models in various sectors, including construction and autonomous vehicles.