Hot French startup ZML releases free product to speed inference across lots of AI chips

ZML, a burgeoning AI startup from France and supported by Turing Award laureate Yann LeCun, is shaking up the tech landscape with the launch of its innovative inference-performance software, ZML/LLMD. This new tool enables the execution of various open-source large language models across a wide array of hardware, including chips from Nvidia, AMD, Google’s TPU, Apple Metal, and Intel Arc. The startup aims to dismantle the barriers that have traditionally confined AI processing to specific hardware, allowing for optimal performance across different platforms. Founder Steeve Morin expressed to TechCrunch that as AI becomes increasingly woven into daily work and life, the importance of optimizing inference—the processing of user prompts—has surged, often overshadowing model training in significance. This optimization has frequently been hindered by software and architectural constraints that contribute to vendor lock-in. ZML's breakthrough could revolutionize the market, especially in light of rising concerns regarding the costs associated with AI technologies. The company hopes to empower businesses and cloud services to leverage a diverse mix of chips, some potentially being more economical or energy-efficient. "Our goal is to return the control back to the users, enabling them to design their systems and achieve genuine efficiency improvements that promote the widespread adoption of AI," Morin stated. The software could also be advantageous for emerging AI chip manufacturers, particularly those in Europe, such as Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA. Morin emphasized the importance of collaboration with these innovators on pioneering projects that have not been attempted elsewhere. Despite the competitive landscape, which includes companies like Baseten, valued at $13 billion, and RadixArk, Morin maintains a positive outlook on Nvidia. He highlighted ZML's strong partnership with the established AI chip leader, which is preparing for the growing demand for inference capabilities, often referred to as the "inference gold rush." ZML’s agile team of 20 has facilitated rapid development, with additional releases on the horizon. Supported by a robust funding round totaling $20 million from notable venture firms, Morin’s history as the VP of engineering at Zenly, acquired by Snapchat, has played a crucial role in attracting investment. Unlike its earlier open-source projects, ZML/LLMD will initially be offered as a free product to gather user insights, though there are plans to monetize it in the future. The startup's potential has caught the attention of industry leaders, including Dagger and Docker founder Solomon Hykes and Hugging Face co-founders Clément Delangue and Julien Chaumond, reinforcing the notion that Europe’s AI startups are capable of thriving locally. "I believe ZML could only have been established in Paris," Morin concluded.