How a once-tiny research lab helped Nvidia become a $4 trillion-dollar company

In 2009, when Bill Dally joined Nvidia’s research lab, it was a modest team of just a dozen individuals focused primarily on ray tracing, a technique crucial for rendering graphics. Fast forward to today, that same lab has expanded to over 400 employees, playing a pivotal role in transforming Nvidia from a fledgling graphics card company into a powerhouse valued at $4 trillion, driving the artificial intelligence revolution. The lab's ambitions have now shifted toward pioneering technologies essential for robotics and AI. Recently, Nvidia introduced a suite of AI models, libraries, and infrastructure tailored for robotics developers, showcasing the practical applications of their research. Dally, who serves as Nvidia’s chief scientist, began his journey with the company in 2003 while working at Stanford University. Initially contemplating a sabbatical after stepping down as chair of Stanford’s computer science department, he caught the attention of Nvidia’s then-research lab head, David Kirk, and CEO Jensen Huang. Their persuasive efforts eventually led him to commit to Nvidia. Upon taking the helm of the research lab, Dally prioritized expansion, encouraging researchers to explore beyond ray tracing into areas like circuit design and very large-scale integration (VLSI). This proactive approach has fueled continuous growth within the lab. "We strive to identify what will make the most positive impact for the company," Dally remarked, reflecting on the lab's commitment to innovation. One significant focus has been on enhancing GPUs for artificial intelligence. Nvidia was ahead of the curve, venturing into AI GPU development as early as 2010, well before the current AI surge. Dally recalled, "We recognized the transformative potential of AI. Jensen believed in our vision, and together we began to tailor our GPUs and develop extensive supporting software." Now, as Nvidia solidifies its dominance in the AI GPU sector, the company is actively exploring new opportunities in physical AI and robotics. Dally emphasized the future potential of robotics, stating, "We aim to craft the brains of future robots, necessitating the development of key technologies." Sanja Fidler, vice president of AI research at Nvidia, joined the team in 2018, bringing her expertise from MIT, where she was developing simulation models for robotics. Fidler was drawn to Nvidia by Huang’s invitation to collaborate closely, emphasizing a cultural fit and a shared vision. She spearheaded the creation of the Omniverse research lab in Toronto, focusing on simulations for physical AI. One of their primary challenges was acquiring the necessary 3D data to create realistic simulations. Fidler explained their investment in differentiable rendering technology, allowing for a seamless transition from images to 3D models. The lab's first major milestone was the release of GANverse3D in 2021, which converts images into 3D models, followed by advancements in video processing for similar purposes. The technologies developed form the backbone of Nvidia’s Cosmos family of world AI models, unveiled at CES in January. Currently, the lab is dedicated to enhancing the speed of these models, aiming for real-time responsiveness in simulations and faster reaction times for robots. Fidler noted, "Robots can process their environment at speeds far exceeding real-time, enabling them to be extremely effective in various applications." Recently, Nvidia showcased a new array of world AI models at the SIGGRAPH conference, designed to generate synthetic data for training robots, alongside new resources for robotics developers. Despite the advancements and excitement surrounding robotics, both Dally and Fidler remain grounded, acknowledging that the industry is still years away from seeing humanoid robots in homes, likening it to the initial hype surrounding autonomous vehicles. "We’re making significant strides," Dally commented. "AI is a crucial enabler, particularly in visual perception and generative tasks. As we tackle these challenges and expand our training data, the capabilities of these robots will continue to evolve."