Liquid AI wants to give smartphones small, fast AI that can see with new LFM2-VL model

Liquid AI has launched its latest innovation, the LFM2-VL, a cutting-edge vision-language foundation model engineered for efficient operation across a diverse range of hardware, including smartphones, laptops, wearables, and embedded systems. This new model promises remarkable low-latency performance, exceptional accuracy, and adaptability for real-world applications. Building on the existing LFM2 architecture, LFM2-VL enhances multimodal processing capabilities, allowing it to handle both text and image inputs at varying resolutions. Liquid AI claims that this new model can deliver GPU inference speeds that are up to twice as fast as similar vision-language models while still performing competitively on major benchmarks. Ramin Hasani, co-founder and CEO of Liquid AI, emphasized the importance of efficiency in their latest offering, stating, "Efficiency is our product." The LFM2-VL models feature open weights with sizes of 440M and 1.6B, processing images at native resolutions of up to 512x512 pixels to eliminate distortion. For larger images, the models utilize non-overlapping patching techniques, ensuring that both intricate details and broader context are captured effectively. Founded by former MIT CSAIL researchers, Liquid AI aims to create AI architectures that transcend traditional transformer models. Their flagship Liquid Foundation Models (LFMs) leverage principles from dynamical systems, signal processing, and numerical linear algebra, resulting in versatile AI models capable of processing various data types, including text, video, and audio. Liquid's approach allows for superior performance while utilizing fewer computational resources, making it ideal for real-time applications and resource-constrained environments. Earlier this year, the company expanded its platform with the Liquid Edge AI Platform (LEAP), a versatile SDK that enables developers to seamlessly run small language models on mobile and embedded devices. LEAP supports both iOS and Android, integrates with Liquid’s models and other open-source SLMs, and features a library of models as compact as 300MB—perfect for modern smartphones. Additionally, the Apollo app allows developers to test models offline, aligning with Liquid AI’s commitment to privacy and low-latency AI. The LFM2-VL model employs a modular architecture that combines a language model backbone with a vision encoder and a multimodal projector. This innovative design includes a two-layer MLP connector that optimizes image token processing, allowing users to customize parameters for speed and quality based on their specific deployment needs. The training of LFM2-VL involved around 100 billion multimodal tokens from both open datasets and proprietary synthetic data, resulting in competitive performance across various vision-language evaluations. Notably, the LFM2-VL-1.6B model achieved impressive scores in Real World QA and Info VQA assessments. Currently, LFM2-VL models are available on Hugging Face, along with example fine-tuning code in Colab. They are compatible with Hugging Face transformers and TRL, and are released under a custom license inspired by Apache 2.0 principles. Liquid AI has indicated that commercial use will be allowed under specific terms, depending on the company's annual revenue. With the introduction of LFM2-VL, Liquid AI is poised to enhance the accessibility of high-performance multimodal AI for on-device applications, ensuring robust capabilities without compromising efficiency.