Microsoft saved its most powerful AI tools for paying customers in the first phase of its AI evolution. Now, the company has announced a series of Copilot features coming to all Windows 11 PCs, including Voice, Copilot Vision, and Copilot Actions.
Alongside the update, Microsoft is launching an ad campaign to expose people to these new features. The timing isn’t an accident. Windows 10 support ended on October 14, and we’re about to see a wave of people upgrade to Windows 11; Microsoft seems intent on putting advanced Copilot features at the fingertips of as many people as possible—and convincing them they’re worth using.
It’s All About Voice
Courtesy of Microsoft
“Hey, Copilot.” That’s the wake word for the AI assistant in voice mode, now coming to all Windows 11 computers. Whether you like it or not, Microsoft wants you to start talking to your computer. The company is introducing new Copilot features across all Windows 11 devices, featuring direct voice interaction with Copilot in apps and throughout the operating system. This capability has been available on smartphones for many years with varying degrees of success, as well as Microsoft’s own Copilot+ line of laptops.
“It’s been almost four decades since the PC changed the way you interact with it, which is primarily mouse and keyboard,” says Yusuf Mehdi, consumer chief marketing officer at Microsoft. “But what if you can go beyond that and talk to it? In our minds, voice would become the third input mechanism to use with your PC.”
The push to speak directly to your PC has some data behind it. Mehdi cited Microsoft’s own observations of how people interact with its PCs via voice, such as through dictation, note-taking, voice typing, and transcriptions. Mehdi also says people use the Copilot app twice as much when using voice instead of typing, due to ease of use. “People talk through their computer, they don’t talk to their computer or with their computer,” he says. “But we think that’s what’s actually going to change a lot.”
Courtesy of Microsoft
With voice-powered Copilot arriving on all Windows 11 computers, Microsoft hopes people will begin using its AI features more. The point isn’t to replace mouse and keyboard altogether, but instead to make voice “additive,” as Mehdi puts it.
Mehdi seemingly forgot to mention “touch” as an input mechanism from his history lesson. Microsoft famously invested quite a lot in trying to transform the PC into a touch-first operating system with Windows 8 and its push into Windows tablets (and Windows Phone), which was ultimately abandoned outside of specific devices like the Surface Pro. Maybe things will be different with voice.
Copilot Vision and Actions
Courtesy of Microsoft
The more prompting and context you give AI, the more it can do. That’s not exactly intuitive with voice, but that’s where Copilot Vision comes into play. Copilot Vision allows AI to view your screen and offer assistance based on this context. It’s been a feature of Copilot+ PCs since June, but now it’ll be available broadly across all Windows 11 computers, even those without an NPU (neural processing Unit). Google recently added a similar capability to its Chrome browser.
One of the most prominent use cases for Copilot Vision has been for tutorials in applications. If you’re looking for a certain action or menu in a complicated piece of software, Copilot Vision can quickly point you in the right direction. It’s a floating toolbar that can follow you around to any app or part of Windows, given the context of what’s on your screen, whether that’s the whole desktop or just a particular window.
That means less prompting, and combined with voice control, it’s more akin to having a knowledgeable friend standing over your shoulder. Microsoft calls it “Highlights,” which can be triggered just by asking Copilot to “show me how,” whether that’s while editing a photo, checking your calendar, or writing a grocery list. In some apps, like Word, Excel, and PowerPoint, Copilot Vision can even “see” what’s beyond your actual screen, such as slides in a presentation or pages in a Word doc that aren’t in full view.
Mehdi also talked about Gaming Copilot, which brings Copilot Vision into the world of gaming, whether on a PC or a device like the new ROG Xbox Ally. I saw a demo of it in action, with the assistant offering directions to a player about what missions to take next in an open-world adventure.
It’s when Copilot Vision is combined with Copilot Actions that things get interesting. This is Microsoft’s take on AI agents. Copilot Actions can do things on your behalf locally with an app or operating system, just by following natural language instructions. So, rather than have Copilot Vision show you where to find that obscure setting in Adobe Photoshop, Copilot Actions can just go turn it on for you. You can have Copilot Actions make the same edit on a folder full of photos or extract information from a large PDF.
Courtesy of Microsoft
Microsoft also showed a preview of where Copilot Actions are heading, which highlights the vision of “rewriting” Windows 11 for AI. A feature called Connectors, available now for Windows Insiders, lets you take actions locally on your PC directly from the Copilot app, such as opening files from OneDrive, opening emails, or even creating a PowerPoint presentation. Microsoft says third-party applications like Google Calendar or Gmail will work within the Copilot app.
For example, you can ask Copilot to find details of a dentist appointment or find a specific file by describing it. So long as you’ve permitted Copilot to have access to those apps, Copilot can not only locate those apps and files, but it can also take action on them or in them. While this replicates some of the functionality of Recall, the security-troubled AI feature exclusive to Copilot+ PCs, Microsoft says they will live side by side. Other features specific to Copilot+, like Click to Do, will remain exclusive to devices with more powerful on-device NPUs for faster AI processing.
Copilot Actions have been a part of Microsoft’s 365 Copilot enterprise tools since late 2024, but this is the first time they’ll be rolled out widely to average Windows users. Microsoft says it’ll be available “in the coming months,” and this time you’ll be able to use Copilot Actions with text prompts instead of just voice.
The idea of taking action on apps from within a search is not dissimilar to what Apple is doing with Spotlight Actions and shortcuts in macOS Tahoe. But Microsoft’s ambitions seem much more grand. For now, Copilot is still something you have to call up, whether with voice or by clicking the app (built into the Taskbar search bar). But it’s not hard to imagine Copilot growing into the primary interface of Windows.
Will any of this actually get people to use Copilot more? That’s hard to say. For a myriad of reasons, the Windows crowd tends to change behavior slowly, and moving from Windows 10 to Windows 11 has already proven to be a challenge.
