Microsoft launches Mu, an ultra-fast on-device AI model for Copilot+ PCs

Microsoft has rolled out Mu, an innovative AI language model crafted to function solely on a PC's Neural Processing Unit (NPU). This groundbreaking model emphasizes both speed and privacy, allowing users to execute natural language tasks on Copilot+ PCs without needing to connect to the cloud. Unlike traditional cloud-dependent models, Mu processes user requests locally, resulting in quicker interactions and enhanced privacy. Microsoft has integrated this model with the Settings agent found in Windows Insider builds, enabling users to effortlessly adjust system settings, such as brightness and power modes, simply by using everyday language. "Mu features a compact design with just 330 million parameters, yet its capabilities rival those of significantly larger models," stated a Microsoft representative. The model employs a transformer-based encoder-decoder architecture, which efficiently separates input processing from output generation, thereby improving overall performance. Testing conducted on Qualcomm’s Hexagon NPU revealed a remarkable 47 percent reduction in the time required to generate the first token, with decoding speeds nearly five times faster than comparable decoder-only models. On optimized devices like the Surface Laptop 7, Mu can produce over 200 tokens per second in real-time, ensuring swift responses and a seamless user experience. In addition to its speed, Mu places a high priority on user privacy by conducting all computations locally on the device. This approach ensures that personal data remains securely on the machine, a critical consideration for both individual users and enterprises. Furthermore, Microsoft highlighted that Mu has been meticulously fine-tuned to manage a wide range of system-level commands. From adjusting screen settings to configuring power preferences, users can control their systems effortlessly with simple verbal instructions. This announcement follows Microsoft's recent initiative to upgrade its European cloud infrastructure, reinforcing its commitment to data sovereignty and ensuring that customer data is kept within the legal frameworks of Europe.