Alibaba leads $290 million investment for building a new kind of AI model as LLM limits emerge

In a significant move within the artificial intelligence landscape, Alibaba Cloud has announced a substantial investment aimed at developing a novel type of AI that aspires to better mirror the complexities of the real world. This initiative marks a departure from conventional chatbots like OpenAI's ChatGPT, which heavily rely on large language models predominantly trained on textual data. Recognizing the limitations of these language models, Alibaba is redirecting its focus toward 'world models' that leverage video and real-world scenarios for training. The tech giant spearheaded a 2 billion yuan ($290 million) investment in ShengShu, the innovative startup behind the AI video generation tool, Vidu. This funding round also saw participation from notable players such as TAL Education and Baidu Ventures. This investment comes on the heels of ShengShu's previous funding of 600 million yuan from Qiming Venture Partners and others just two months prior. Although the startup did not disclose its valuation, it expressed that the new funding will facilitate the development of a 'general world model.' This model aims to bridge two currently distinct realms: the digital environments of games and AI-generated videos, alongside the tangible world of autonomous vehicles and robotics. ShengShu asserts that a general world model, constructed from multimodal data encompassing vision, audio, and tactile feedback, can more effectively represent the dynamics of the physical world than traditional large language models. Founder Zhu Jun emphasized the goal of connecting perception with action, enabling AI systems to consistently model and predict real-world behavior. The Vidu Q3 Pro, ShengShu's latest model released in January, has already gained recognition, ranking among the top 10 AI models for converting text and images into videos, according to Artificial Analysis. The startup had previously launched Vidu globally, just before OpenAI's now-discontinued Sora tool became widely available. In a competitive landscape, Chinese short-video platforms Kuaishou and ByteDance have also introduced similar AI video generation tools. Alibaba has been proactive in expanding its portfolio of investments in related startups, including a recent $50 million stake in Tripo AI, which specializes in rapidly generating digital 3D models from photographs using AI. Tripo AI is also transitioning away from language model techniques, opting for AI tools that are embedded in physical space and are working on their own world model. Furthermore, Alibaba led a $60 million investment in PixVerse, which has released an AI world model that allows users to dictate the flow of video content during its generation. Alibaba, originally rooted in e-commerce, has also made strides in releasing open-source AI models for video generation and robots. ShengShu mentioned its strategic partnerships with companies developing embodied AI systems, such as humanoid robots, that engage with the physical environment in various sectors, including industrial, commercial, and home applications. Experts, including Wired co-founder Kevin Kelly, have highlighted the importance of world models in robotics, suggesting that the technology requires more than just large language models to function effectively. For AI to truly emulate human intelligence, it needs reasoning capabilities, an understanding of the physical world, and the ability to learn continuously. While the latter aspect is still under development, language model-driven chatbots have initiated the knowledge component, making advancements in world models a crucial area for future breakthroughs.