Indian AI lab Sarvam’s new models are a major bet on the viability of open-source AI

At the India AI Impact Summit held in New Delhi, Indian AI lab Sarvam introduced an innovative suite of large language models aimed at challenging the dominance of pricier systems from major U.S. and Chinese competitors. This strategic launch reflects India's initiative to lessen dependence on foreign AI platforms while catering to local languages and specific use cases. The newly unveiled models include impressive options with 30 billion and 105 billion parameters, along with advanced text-to-speech, speech-to-text, and document analysis capabilities. These models signify a significant upgrade from Sarvam's earlier 2-billion-parameter model released in October 2024. Employing a mixture-of-experts architecture, the 30-billion and 105-billion-parameter models activate only a select number of parameters, which greatly reduces computational costs. The 30B model features a 32,000-token context window tailored for real-time conversations, while the 105B model extends this to a 128,000-token window for intricate, multi-step reasoning tasks. Crucially, Sarvam developed these models from the ground up, training them on vast datasets—16 trillion tokens for the 30B model and trillions more across several Indian languages for the 105B variant. This extensive training positions the models for real-time applications, such as voice assistants and chat systems supporting Indian languages. The project received substantial backing from India's government-supported IndiaAI Mission, with additional infrastructure assistance from Yotta and technical expertise from Nvidia. Company co-founder Pratyush Kumar emphasized a careful approach to scaling the models, focusing on practical applications rather than sheer size. "We want to be mindful in how we do the scaling," he stated, highlighting the importance of understanding significant tasks before expanding. Looking ahead, Sarvam plans to open-source the 30B and 105B models, although details regarding the availability of training data and complete training code remain unspecified. Additionally, the company aims to develop specialized AI systems, including coding-focused tools and enterprise solutions through its Sarvam for Work initiative, alongside a conversational AI agent platform named Samvaad. Founded in 2023, Sarvam has successfully secured over $50 million in funding from notable investors such as Lightspeed Venture Partners, Khosla Ventures, and Peak XV Partners (formerly Sequoia Capital India).