OpenAI cofounder says scaling compute is not enough to advance AI: 'It's back to the age of research again'

Ilya Sutskever, a co-founder of OpenAI and a leading figure in the artificial intelligence sector, has indicated that the industry must pivot back to a focus on foundational research. In a recent episode of the 'Dwarkesh Podcast,' Sutskever challenged the prevailing notion that simply increasing computational power is the key to advancing AI technologies. For years, technology companies have invested heavily in acquiring GPUs and expanding data centers, operating under the assumption that enhancing their AI capabilities—whether through large language models or image generation—could be achieved by scaling up resources. This approach has yielded significant outcomes over the past five years, offering a low-risk investment strategy compared to the unpredictable nature of research. However, Sutskever, who now leads Safe Superintelligence Inc., argues that this method may be reaching its limits. With finite data and an already vast amount of computational power at their disposal, he questions the belief that simply multiplying resources would lead to revolutionary advancements in AI. 'Is the belief really: 'Oh, it's so big, but if you had 100x more, everything would be so different?' he asked. 'I don't think that's true.' While he acknowledges the ongoing necessity for computational resources, Sutskever emphasizes that meaningful research will be essential to maximize the potential of these resources. One critical area needing attention is enhancing the ability of AI models to generalize from limited data, akin to human learning. 'These models somehow just generalize dramatically worse than people,' he observed. 'That seems like a very fundamental thing.'