Microsoft gives Windows 11 a voice, vision and action with new AI features

In a significant upgrade as it phases out Windows 10, Microsoft is enhancing Windows 11 with powerful AI capabilities. The tech giant is integrating its Copilot AI assistant more deeply into the operating system, asserting that every Windows 11 device will effectively become an AI-enabled machine. Copilot is set to become more interactive, mirroring the functionality of popular voice assistants such as Google Assistant, Alexa, and Siri. Users can activate Copilot with the phrase "Hey Copilot," making it easier to engage with the assistant just like other voice-operated technologies. This new voice capability requires users to opt-in through the Copilot app's settings. Once activated, a microphone icon will appear on the screen, accompanied by a chime to indicate that Copilot is ready to listen. A distinctive feature is the use of the phrase “Goodbye” to conclude a conversation, although users can also simply tap an ‘X’ icon, or Copilot will automatically end the interaction after a period of silence. Microsoft reports that users who engage with Copilot via voice show twice the interaction compared to text-based communication, leading to more meaningful exchanges with the AI assistant. In addition to voice enhancements, Microsoft is introducing 'Copilot Vision,' a feature that allows users to share their desktops or applications for real-time analysis and insight generation. This new capability enables users to ask Copilot not only questions about their current tasks but also to demonstrate how to complete specific actions, such as offering tips while gaming or reviewing travel plans. When sharing files like Word, Excel, or PowerPoint, Copilot can analyze entire documents without needing to sift through each slide, providing quick insights and assistance. For Windows Insiders, Microsoft plans to roll out a text-in, text-out feature enabling users to communicate with Copilot Vision through text. Additionally, a new "Ask Copilot" option will be accessible on the taskbar for Windows Insiders, offering users a seamless way to locate apps, files, and settings while providing quick access to Copilot's voice and vision capabilities. Yusuf Mehdi, Microsoft's Executive Vice President and Consumer Chief Marketing Officer, emphasized that this functionality utilizes existing Windows APIs to ensure user privacy, stating that Copilot does not gain access to personal content. Furthermore, Microsoft is launching Copilot Actions, empowering users to describe tasks in natural language. The AI will attempt to complete these tasks by interacting with various applications on the desktop and web, allowing users to focus on other jobs while monitoring progress. This feature can assist with tasks such as resizing images, extracting data from PDFs, and bulk editing documents.