Microsoft wants to bring generative AI to the forefront of Windows \u2014 and the PCs running it.<\/p>\n
At a pair of keynotes during its annual Build developer conference this week, the company unveiled a new lineup of Windows machines it\u2019s calling Copilot+ PCs, plus generative AI-powered features like Recall, which helps users find apps, files and other content they\u2019ve viewed in the past. Copilot, Microsoft\u2019s brand of generative AI, will soon be far more deeply integrated into the Windows 11 experience. And new Microsoft Surface devices are on the way.<\/p>\n
We\u2019ve rounded up all the major announcements from Monday and Tuesday here.<\/p>\n
Volumetric Apps<\/h2>\n
Microsoft is bringing Windows Volumetric Apps<\/a> \u2014 basically spatially aware, interactive VR apps \u2014 to Meta Quest headsets. Through a partnership with Meta, Microsoft says that it\u2019ll deliver Windows 365 and local PC connectivity to Quest headsets, enabling developers to extend their apps into 3D space.<\/p>\n During Tuesday\u2019s keynote, Microsoft showed off a digital exploded 3D view of an Xbox controller from the perspective of a Meta Quest 3 headset \u2014 a digital object the wearer could manipulate with their hands. \u201cWe\u2019re deepening our partnership with Meta to make Windows a first-class experience on Quest devices,\u201d Pavan Davuluri, CVP of Windows and devices at Microsoft, said during the demo.<\/p>\n Developers can\u00a0sign up for a preview<\/a>\u00a0Tuesday to receive access to Microsoft\u2019s new volumetric API.<\/p>\n Copilot+ PCs are Microsoft\u2019s vision of AI-first, flagship Windows hardware. All include dedicated chips called NPUs to power AI experiences like Recall. And they ship with 16GB of RAM minimum, paired with SSD storage. <\/p>\n The first Copilot+ PCs will pack Qualcomm\u2019s Snapdragon X Elite and Plus chips, which Microsoft claims delivers up to 15 hours of web browsing and 20 hours of video battery life. Chipmakers Intel and AMD are also committed to building processors for Copilot+ devices in partnership with a range of manufacturers, including Acer, Asus, Dell, HP, Lenovo and Samsung.<\/p>\n Copilot+ PCs start at $999, and some are available for preorder today.<\/p>\n Microsoft\u2019s newly unveiled Surface devices, the Surface Laptop and Surface Pro, focus on performance and battery. <\/p>\n The latest Surface Laptop \u2014 available with a 13.8- or 15-inch display \u2014 has been redesigned with \u201cmodern lines\u201d and thinner screen bezels. It lasts up to 22 hours on a charge and is up to 86% faster than the Surface Laptop 5, the company says. It also supports Wi-Fi 7 and has a haptic feedback touchpad.<\/p>\n As for the new Surface Pro, Microsoft says it\u2019s up to 90% faster than the previous-gen Surface Pro (the Surface Pro 9), and it sports a new OLED with HDR display, Wi-Fi 7 (and optional 5G) and an upgraded ultrawide front-facing camera. Also, its detachable keyboard \u2014 which has been reinforced with additional carbon fiber \u2014 now has haptic feedback.<\/p>\n Windows 11\u2019s forthcoming Recall feature can \u201cremember\u201d apps and content a user accessed on their PC weeks or even months ago, for example helping them to find a Discord chat where they were discussing clothes they were considering buying. Users can use Recall\u2019s timeline to \u201cscroll back\u201d to see what they were working on in the recent past and drill down in files like PowerPoint presentations to surface info potentially relevant to their searches. <\/p>\n Microsoft says that Recall can create associations between colors, images and more to let users search for practically anything on their PCs in natural language (not dissimilar to startup Rewind\u2019s<\/a> tech); developers will be able to improve Recall by adding contextual information to their apps. And Microsoft claims all user data associated with Recall is kept private and on-device \u2014 and not used to train AI models, importantly.<\/p>\n Here\u2019s more<\/a> from Microsoft: \u201cYour snapshots are yours; they stay locally on your PC. You can delete individual snapshots, adjust and delete ranges of time in Settings, or pause at any point right from the icon in the System Tray on your Taskbar. You can also filter apps and websites from ever being saved.\u201d<\/p>\n There\u2019s now more AI in Windows than ever and some of it exclusively on the new Copilot+ PCs.<\/p>\n A new feature called Super Resolution can restore old photos by automatically upscaling them. And Copilot can now analyze images to give users ideas for creative compositions. Through a feature called Cocreator, users can generate images and also ask the AI model to follow what they\u2019re drawing to change or restyle the image. <\/p>\n Elsewhere, Live Captions with live translations translates any audio that passes through a PC \u2014 whether from YouTube or a local file \u2014 into the language of the user\u2019s choosing. Live translations will initially support around 40 languages including English, Spanish, Mandarin and Russian.<\/p>\n A separate but related new feature in Microsoft Edge offers real-time video translation on sites like LinkedIn, YouTube, Coursera, Reuters, CNBC, Bloomberg and more. Set to become available in the near future, the feature \u2014 which supports the translation of Spanish into English and English to German, Hindi, Italian, Russian and Spanish \u2014 translates spoken content through both dubbing and subtitles live.<\/p>\n Team Copilot<\/a> is the latest expansion of Microsoft\u2019s growing Copilot suite of generative AI tech. It integrates with Teams, the company\u2019s videoconferencing app, to help manage meeting agendas and take notes that anyone in a meeting can co-author. And it extends to Loop and Planner, Microsoft\u2019s collaboration and planning platforms, to create and assign tasks, track deadlines and notify team members when their input\u2019s needed. <\/p>\n In somewhat related Copilot news, Microsoft has launched (in private preview) Copilot Extensions<\/a>, which allow developers to extend GitHub\u2019s code-generating tool GitHub Copilot with third-party apps and skills. Launch partners include DataStax, Docker and LambdaTest; extensions will live in the GitHub Marketplace, but developers will also be able to create their own private extensions to integrate with their internal systems and APIs.<\/p>\n Powering capabilities such as Recall and Super Resolution is the Windows Copilot Runtime, a collection of ~40 generative AI models make up what Microsoft describes as \u201ca new layer\u201d of Windows. In tandem with the semantic index, a vector-based system local to an individual Copilot+ PC, the Windows Copilot Runtime allows generative AI-powered apps \u2014 including third-party apps \u2014 to run without necessarily needing an internet connection.<\/p>\n \u201c[The runtime] consists of ready-to-use AI APIs like Studio Effects, Live Captions translations, OCR, Recall with user activity and [more], which will be available to developers in June,\u201d Davuluri said on Tuesday. <\/p>\n Microsoft says that CapCut, the popular video editor from TikTok owner ByteDance, will use the Windows Copilot Runtime and accompanying new Windows Copilot Library, a set of APIs and AI dev tools, to speed up its AI features. And Meta will add the aforementioned Studio Effects to WhatsApp to deliver features such as background blur and eye contact during video calls.<\/p>\n Azure AI Studio, the toolset within Microsoft\u2019s\u00a0Azure OpenAI Service<\/a>\u00a0that lets customers combine an AI model and build an app that \u201creasons over\u201d that data, will soon allow developers to create apps using pay-as-you-go inference APIs<\/a> \u2014 the APIs through which developers can access and fine-tune generative AI models hosted on Azure infrastructure. Microsoft calls this \u201cmodel-as-a-service,\u201d and it\u2019s launching with models from Nixtla and Core42 to start.<\/p>\n In the adjacent Copilot Studio product suite, Microsoft is launching Copilot agents, which the company describes as AI bots that can \u201cindependently orchestrate tasks tailored to specific roles and functions.\u201d (Copilot Studio provides tools to connect\u00a0Copilot for Microsoft 365<\/a>, the AI-powered \u201ccopilot\u201d in apps like Excel and Word, to third-party data.) Leveraging memory and knowledge of context, Copilot agents can navigate various types of business workflows, learning from user feedback and asking for help when they encounter situations they don\u2019t know how to handle.<\/p>\n There\u2019s a new dev kit from Qualcomm aimed at developers building apps for Arm-chip-packing Copilot+ PCs<\/p>\n The $899.99 Snapdragon Dev Kit for Windows \u2014 which measures around the same width, height and length as Apple\u2019s Mac Mini, incidentally \u2014 houses Qualcomm\u2019s Snapdragon X Elite chip paired with 32GB of RAM, 512GB of storage and lots of I\/O. The Dev Kit supports Wi-Fi 7 and Bluetooth 5.4, and, via its various USB-C and HDMI ports, it can drive up to three 4K monitors at once.<\/p>\n Microsoft announced a new addition to its generative AI model family Phi, Phi-3-vision, that can do general visual analysis and reasoning tasks, such as answering questions about charts and images. The model can read both text and images and is efficient enough to run on a mobile device.<\/p>\n Phi-3-vision is available in preview, while the model\u2019s previously-announced text-only counterparts \u2014 Phi-3-mini, Phi-3-small and Phi-3-medium\u00a0\u2014 are now generally available.<\/p>\n Microsoft is teaming up with Khan Academy to donate access to cloud compute infrastructure, allowing Khan Academy to offer educators in the U.S. free access to Khan Academy\u2019s AI-powered tools. The two companies will also collaborate to explore opportunities to improve AI apps for math tutoring through generative AI, Microsoft said on Tuesday.<\/p>\n We\u2019re launching an AI newsletter! Sign up\u00a0<\/em>here<\/a><\/em>\u00a0to start receiving it in your inboxes on June 5.<\/em><\/p>\n<\/div>\nCopilot+ PCs<\/h2>\n
![\"Microsoft](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/Copilot-PC-Hero-1536x865-1.jpg?w=680\")
Surface Pro and Surface Laptop<\/h2>\n
![\"Microsoft](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/Surface-Laptop-for-Enterprise1920.jpg?w=680\")
![\"Microsoft](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/Surface-Pro-for-Enterprise1920.jpg?w=680\")
Recall<\/h2>\n
Image editing and live translations<\/h2>\n
![\"Live](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/Restyle-Image-1024x640-transformed.jpeg?w=680\")
Team Copilot and extensions <\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/copilot-extension.jpg?w=680\")
Windows Copilot Runtime<\/h2>\n
![\"Microsoft](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/upscaler-Windows_Copilot_Runtime_image_1_1024x533-2x.jpg?w=680\")
Upgraded bot builders<\/h2>\n
Snapdragon Dev Kit<\/h2>\n
![\"Qualcomm](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/Restyle-Image-1024x640-transformed-1.jpeg?w=680\")
Phi-3<\/h2>\n
Partnership with Khan Academy<\/h2>\n