Microsoft launches Mu AI model for smart local tasks on Windows PCs

Microsoft has introduced Mu, an innovative AI model designed to operate locally on Copilot+ Windows PCs. This model aims to provide users with swift and precise assistance by leveraging the device’s Neural Processing Unit, eliminating the need for cloud server dependency. Currently, Mu is compatible with the Windows Settings app for users enrolled in the Windows Insider Dev Channel. Users can interact with the AI using natural language commands such as "turn on night light," to which Mu responds instantly. With 330 million parameters, Mu employs a Transformer encoder-decoder architecture, enabling it to process input and output independently, resulting in enhanced speed and reduced latency. According to Microsoft, Mu can generate over 100 tokens per second, demonstrating significantly lower latency compared to similar models. The training of Mu utilized Azure A100 GPUs, and it was refined using advanced techniques, including grouped-query attention and rotary positional embeddings, allowing it to function effectively even on devices with limited hardware capabilities. To optimize Mu’s performance across various PCs, Microsoft collaborated with Intel, AMD, and Qualcomm. The model underwent post-training quantization, enabling it to operate with 8-bit and 16-bit integer formats. For instance, on devices like the Surface Laptop 7, Mu achieves impressive output speeds exceeding 200 tokens per second. Initially, Microsoft experimented with a more substantial model named Phi LoRA, which, despite its accuracy, was slower in processing. After fine-tuning, Mu emerged as a faster alternative without sacrificing accuracy. The development team enhanced Mu’s dataset to include 3.6 million training samples and expanded its support to hundreds of Windows system settings. Additional features like prompt tuning and noise injection have further improved its capacity to comprehend real-world queries. Mu is part of Microsoft’s broader initiative to integrate AI functionalities directly into user devices. This model builds upon previous research from other AI models like Phi and Phi Silica and is expected to be pivotal in shaping future AI experiences on Windows PCs.