Co-founders behind Reface and Prisma join hands to improve on-device model inference with Mirai

In the rapidly evolving world of artificial intelligence, the focus has predominantly been on expanding cloud capabilities and massive data centers. However, a small yet ambitious team of 14 at London-based Mirai is shifting the spotlight toward enhancing on-device AI performance on smartphones and laptops. Founded last year by Dima Shevts and Alexey Moiseenkov, Mirai has successfully secured a $10 million seed investment led by Uncork Capital. Shevts, known for his role in creating the popular face-swapping app Reface, and Moiseenkov, co-founder of the viral AI filters application Prisma, bring a wealth of experience in developing scalable consumer applications. Recognizing a gap in the market, the duo has been contemplating the potential of on-device AI long before generative AI became a buzzword. Shevts shared his thoughts in a recent conversation, highlighting that amidst the excitement surrounding generative AI, there is an overlooked need for effective on-device solutions. Their vision for Mirai is to leverage AI to facilitate complex tasks directly on mobile devices, ultimately enhancing user experience. Many developers have expressed a desire for improved cost management and efficiency, prompting Mirai to create a framework that optimizes model performance on devices. The company has developed a specialized inference engine tailored for Apple Silicon, which significantly boosts on-device processing capabilities. Mirai’s forthcoming SDK promises seamless integration for developers, allowing them to incorporate the engine into their applications with minimal effort. "Our goal was to provide an integration experience akin to Stripe, where developers can easily start utilizing our platform for tasks such as summarization and classification with just a few lines of code,” Shevts explained. The engine, built using Rust, reportedly enhances model generation speeds by up to 37% without compromising output quality. Currently, Mirai focuses on improving text and voice functionalities, with plans to expand into visual applications in the future. The team is collaborating with leading model providers to optimize their offerings for edge computing and is actively engaging with various chip manufacturers. Future expansions will also see Mirai's technology supporting Android devices. Additionally, Mirai intends to release benchmarks for on-device performance, allowing model developers to assess efficiency. While Shevts acknowledges that not all AI tasks can be managed on-device, the team is working on a hybrid system that will enable requests that exceed device capabilities to be processed in the cloud. Although Mirai isn't currently partnering directly with applications, their technology could potentially empower on-device assistants, transcription services, translation tools, and chat applications. Uncork Capital’s managing partner, Andy McLoughlin, reflects on his previous investments in edge machine learning, noting the changing dynamics of the industry. He believes that as the costs of cloud inference continue to rise, the demand for efficient on-device solutions will only grow, positioning Mirai advantageously in the market.