It\u2019s that moment you\u2019ve been waiting for all year: Google I\/O keynote day! Google kicked off its developer conference each year with a rapid-fire stream of announcements, including many unveilings of recent things it\u2019s been working on. Brian<\/a> already kicked us off by sharing what we are expecting<\/a>.<\/p>\n Since you might not have had time to watch the whole two-hour presentation<\/a> Tuesday, we took that on and delivered quick hits of the biggest news from the keynote as they were announced, all in an easy-to-digest, easy-to-skim list.<\/p>\n Google showcased a demo of a call scam detection feature during I\/O, which it says will be added to a future version of Android. The feature scans voice calls as they\u2019re happening with AI, which effectively is client-side scanning, which has already sparked such a backlash on iOS that Apple abandoned its plans to adopt it in 2021. And as expected, a number of privacy advocates and experts voiced concerns<\/a> over Google\u2019s use of the technology, which they say could swiftly expand beyond applying just to scams and be used in more malicious ways. Read more<\/strong><\/a><\/p>\n On Wednesday, Google announced it is adding new security and privacy protections to Android, including on-device live threat detection to catch malicious apps, new safeguards for screen sharing, and better security against cell site simulators.<\/p>\n The company said it is increasing the on-device capability of its Google Play Protect system to detect fraudulent apps trying to breach sensitive permissions. It also uses AI to detect if apps are trying to interact with other services and apps in an unauthorized manner.<\/p>\n Google said if the system is certain about malicious behavior, it disables the app automatically. Otherwise, it alerts the company for a review and then alerts users. Read more<\/a><\/strong><\/p>\n And to protect devices in the real world, Google also announced Theft Detection Lock<\/a>, an AI-powered addition that identifies motion commonly associated with theft, like a swift movement in an opposite direction. Once detected, the phone screen automatically locks, preventing future usage of the device without clearing whatever safeguards you\u2019ve put in place. Read more<\/strong><\/a><\/p>\n Google worked its Gemini into its Google TV smart TV operating system so it can generate descriptions for movies and TV shows. When a description is missing on the home screen, the AI will fill it in automatically to ensure that viewers never have to wonder what a title is about. It\u2019ll also translate descriptions into the viewer\u2019s native language, making the content more discoverable to a wider audience. The best part? The AI-generated descriptions are also personalized based on a viewer\u2019s genre and actor preferences. Read more<\/a><\/strong><\/p>\n Now here\u2019s a fun one. Private Space is a new Android feature that lets users silo a portion of the operating system for sensitive information. It\u2019s a bit like Incognito mode for the mobile operating system, sectioning designated apps into a \u201ccontainer.\u201d<\/p>\n The space is available from the launcher and can be locked as a second layer of authentication. Apps in Private Space will be hidden from notifications, settings and recents. Users can still access the apps through a system sharesheet and photo picker in the main space, so long as the private space has been unlocked.<\/p>\n Developers can play around with it now, but there is a caveat \u2014 there is a bug. Google says it expects to address the bug in the coming days. Read more<\/a><\/strong><\/p>\n Google Maps users will soon have a new layer of content on their phones \u2014 they will have access to geospatial augmented reality content. The feature will first appear in Singapore and Paris as part of a pilot program launching later this year.<\/p>\n Users will be able to access the AR content by first searching for a location in Google Maps. If the location has AR content and the user is near the place, they will have to tap on the image that says \u201cAR Experience\u201d and then lift their phone.\u00a0<\/p>\n If someone is exploring a place remotely, they can see the same AR experience in Street View. After exploring the AR content, users can share the experience through a deep link URL or QR code on social media. Read more<\/a><\/strong><\/p>\n Google gave a developer preview of the new version of its smartwatch operating system, Wear OS 5. The latest release focuses on improved battery life and other performance improvements, like more efficient workout tracking. Developers are also getting updated tools for creating watch faces, as well as new versions of Wear OS tiles and Jetpack Compose for building watch apps. Read more<\/a><\/strong><\/p>\n As we note all over this post, the Google I\/O developer conference came with\u00a0a big dose of AI. See how Anthony Ha summed it up Wednesday. Read more<\/a><\/strong><\/p>\n We\u2019ve launched a new \u201cWeb\u201d filter that shows only text-based links, just like you might filter to show other types of results, such as images or videos. The filter appears on the top of the results page alongside other filters or as part of the \u201cMore\u201d option, rolling out today\u2026 pic.twitter.com\/tIUy9LNCy5<\/a><\/p>\n \u2014 Google SearchLiaison (@searchliaison) May 14, 2024<\/a><\/p><\/blockquote>\n<\/div>\n<\/figure>\n Google introduced a new way to filter for just text-based links. The new \u201cWeb\u201d filter appears at the top of the results page and enables users to filter for text links the way they can today filter for images, video, news or shopping.<\/p>\n As Sarah Perez reports, the launch is an admission that sometimes people will want to just surface text-based links to web pages, aka the classic blue links, that today are often of secondary importance as Google either answers the question in its informational Knowledge Panels or, now, through AI experiments. Read more<\/a><\/strong><\/p>\n There\u2019s a new addition to the Firebase platform, called Firebase Genkit, that aims to make it easier for developers to build AI-powered applications in JavaScript\/TypeScript, with Go support coming soon. It\u2019s an open source framework, using the Apache 2.0 license, that enables developers to quickly build AI into new and existing applications.<\/p>\n Some of the use cases for Genkit the company is highlighting Tuesday include many of the standard GenAI use cases: content generation and summarization, text translation and generating images. Read more<\/strong><\/a><\/p>\n Tuesday\u2019s Google I\/O ran for 110 minutes, but Google managed to reference AI a whopping 121 times during (by its own count) the event. CEO Sundar Pichai referenced the figure to wrap up the presentation, cheekily stating that the company was doing the \u201chard work\u201d of counting for us. Again, it was no surprise \u2014 we were ready for it. Read more<\/a><\/strong><\/p>\n Also today, Google unveiled LearnLM, a new family of generative AI models \u201cfine-tuned\u201d for learning. It\u2019s a collaboration between Google\u2019s DeepMind AI research division and Google Research. LearnLM models are designed to \u201cconversationally\u201d tutor students on a range of subjects, Google says. <\/p>\n Though it is already available on several of Google\u2019s platforms, the company is taking LearnLM through a pilot program in Google Classroom. It is also working with educators to see how LearnLM might simplify and improve the process of lesson planning. LearnLM could help teachers discover new ideas, content and activities, Google says, or find materials tailored to the needs of specific student cohorts.\u00a0Read more<\/a><\/strong><\/p>\n Speaking of education, new to YouTube are AI-generated quizzes. This new conversational AI tool allows users to figuratively \u201craise their hand\u201d when watching educational videos. Viewers can ask clarifying questions, get helpful explanations or take a quiz on the subject matter.\u00a0<\/p>\n This is going to be some relief for those who have to watch longer educational videos, such as lectures or seminars, due to the Gemini model\u2019s long-context capabilities. These new features are rolling out to select Android users in the U.S. Read more<\/a><\/strong><\/p>\n One of the top requests Google heard from developers is for a bigger Gemma model, so Google will be adding a new 27-billion-parameter model to Gemma 2. This next generation of Google\u2019s Gemma models will launch in June. This size is optimized by Nvidia to run on next-generation GPU and can run efficiently on a single TPU host and vertex AI, Google said. Read more<\/strong><\/a><\/p>\n Google Play is getting some attention with a new discovery feature for apps, new ways to acquire users, updates to Play Points and other enhancements to developer-facing tools like the Google Play SDK Console and Play Integrity API, among other things.<\/p>\n Of particular interest to developers is something called the Engage SDK, which will introduce a way for app makers to showcase their content to users in a full-screen, immersive experience that\u2019s personalized to the individual user. Google says this isn\u2019t a surface that users can see at this time, however. Read more<\/a><\/strong><\/p>\n Tuesday, Google previewed a feature it believes will alert users to potential scams during the call.\u00a0<\/p>\n The feature, which will be built into a future version of Android,\u00a0utilizes Gemini Nano, the smallest version of Google\u2019s generative AI offering, which can be run entirely on-device. The system effectively listens for \u201cconversation patterns commonly associated with scams\u201d in real time.\u00a0<\/p>\n Google gives the example of someone pretending to be a \u201cbank representative.\u201d Common scammer tactics like password requests and gift cards will also trigger the system. These are all pretty well understood to be ways of extracting your money from you, but plenty of people in the world are still vulnerable to these sorts of scams. Once set off, it will pop up a notification that the user may be falling prey to unsavory characters.\u00a0Read more<\/a><\/strong><\/p>\n Google Photos is getting an AI infusion with the launch of an experimental feature, Ask Photos, powered by Google\u2019s Gemini AI model. The new addition, which rolls out later this summer, will allow users to search across their Google Photos collection using natural language queries that leverage an AI\u2019s understanding of their photo\u2019s content and other metadata.<\/p>\n While before users could search for specific people, places, or things in their photos, thanks to natural language processing, the AI upgrade will make finding the right content more intuitive and less of a manual search process.<\/p>\n And the example was cute, too. Who doesn\u2019t love a tiger stuffed animal\/Golden Retriever band duo called \u201cGolden Stripes?\u201d Read more<\/strong><\/a><\/p>\n Gmail users will be able to search, summarize, and draft their emails using its Gemini AI technology. It will also be able to take action on emails for more complex tasks, like helping you process an e-commerce return by searching your inbox, finding the receipt and filling out an online form. Read more<\/a><\/strong><\/p>\n Another upgrade to the generative AI is that Gemini can now analyze longer documents, codebases, videos and audio recordings than before.<\/p>\n In a private preview of a new version of Gemini 1.5 Pro, the company\u2019s current flagship model, it was revealed that it can take in up to 2 million tokens. That\u2019s double the previous maximum amount. With that level,\u00a0the new version of Gemini 1.5 Pro supports the largest input of any commercially available model. Read more<\/strong><\/a><\/p>\n The company previewed a new experience in Gemini called Gemini Live, which lets users have \u201cin-depth\u201d voice chats with Gemini on their smartphones. Users can interrupt Gemini while the chatbot\u2019s speaking to ask clarifying questions, and it\u2019ll adapt to their speech patterns in real time. And Gemini can see and respond to users\u2019 surroundings, either via photos or video captured by their smartphones\u2019 cameras.<\/p>\n At first glance, Live doesn\u2019t seem like a drastic upgrade over existing tech. But Google claims it taps newer techniques from the generative AI field to deliver superior, less error-prone image analysis \u2014 and combines these techniques with an enhanced speech engine for more consistent, emotionally expressive and realistic multi-turn dialogue. Read more<\/strong><\/a><\/p>\n Now for a tiny announcement. Google is also building Gemini Nano, the smallest of its AI models, directly into the Chrome desktop client, starting with Chrome 126. This, the company says, will enable developers to use the on-device model to power their own AI features. Google plans to use this new capability to power features like the existing \u201chelp me write\u201d tool from Workspace Lab in Gmail, for example. Read more<\/strong><\/a><\/p>\n Google\u2019s Gemini on Android, its AI replacement for Google Assistant, will soon be taking advantage of its ability to deeply integrate with Android\u2019s mobile operating system and Google\u2019s apps. Users will be able to drag and drop AI-generated images directly into their Gmail, Google Messages and other apps. Meanwhile, YouTube users will be able to tap \u201cAsk this video\u201d to find specific information from within that YouTube video, Google says. Read more<\/strong><\/a><\/p>\n Gemini model capabilities are coming to the Google Maps platform for developers, starting with the Places API. Developers can show generative AI summaries of places and areas in their own apps and websites. The summaries are created based on Gemini\u2019s analysis of insights from Google Maps\u2019 community of more than 300 million contributors. What\u2019s better? Developers will no longer have to write their own custom descriptions of places. Read more<\/strong><\/a><\/p>\n Google unveiled its next generation \u2014 the sixth, to be exact \u2014 of its Tensor Processing Units (TPU) AI chips. Dubbed Trillium, they will launch later this year. If you recall, announcing the next generation of TPUs is something of a tradition at I\/O, even as the chips only roll out later in the year.\u00a0<\/p>\n These new TPUs will feature a 4.7x performance boost in compute performance per chip when compared to the fifth generation. What\u2019s maybe even more important, though, is that Trillium features the third generation of SparseCore, which Google describes as \u201ca specialized accelerator for processing ultra-large embeddings common in advanced ranking and recommendation workloads.\u201d Read more<\/strong><\/a><\/p>\n Google is adding more AI to its search, assuaging doubts that the company is losing market share to competitors like ChatGPT and Perplexity. It is rolling out AI-powered overviews to users in the U.S. Additionally, the company is also looking to use Gemini as an agent for things like trip planning. Read more<\/strong><\/a><\/p>\n Google plans to use generative AI to organize the entire search results page for some search results. That\u2019s in addition to the existing AI Overview feature, which creates a short snippet with aggregate information about a topic you were searching for. The AI Overview feature becomes generally available Tuesday, after a stint in Google\u2019s AI Labs program. Read more<\/strong><\/a><\/p>\n Google announced Imagen 3, the latest in the tech giant\u2019s Imagen generative AI model family.<\/p>\n Demis Hassabis, CEO of DeepMind, Google\u2019s AI research division, said that Imagen 3 more accurately understands the text prompts that it translates into images versus its predecessor, Imagen 2, and is more \u201ccreative and detailed\u201d in its generations. In addition, the model produces fewer \u201cdistracting artifacts\u201d and errors, he said.<\/p>\n \u201cThis is [also] our best model yet for rendering text, which has been a challenge for image generation models,\u201d Hassabis added. Read more<\/strong><\/a><\/p>\n Project IDX, the company\u2019s next-gen, AI-centric browser-based development environment, is now in open beta. With this update comes an integration with the Google Maps Platform into the IDE, helping add geolocation features to its apps, as well as integrations with the Chrome Dev Tools and Lighthouse to help debug applications. Soon, Google will also enable deploying apps to Cloud Run, Google Cloud\u2019s serverless platform for running front- and back-end services. Read more<\/strong><\/a><\/p>\n Google\u2019s gunning for OpenAI\u2019s Sora with Veo, an AI model that can create 1080p video clips around a minute long given a text prompt. Veo can capture different visual and cinematic styles, including shots of landscapes and time lapses, and make edits and adjustments to already-generated footage.<\/p>\n It also builds on Google\u2019s preliminary commercial work in video generation, previewed in April, which tapped the company\u2019s Imagen 2 family of image-generating models to create looping video clips. Read more<\/strong><\/a><\/p>\n The AI-powered Circle to Search feature, which allows Android users to get instant answers using gestures like circling, will now be able to solve more complex problems across psychics and math word problems. It\u2019s designed to make it more natural to engage with Google Search from anywhere on the phone by taking some action \u2014 like circling, highlighting, scribbling or tapping.\u00a0Oh, and it\u2019s also better to help kids with their homework directly from supported Android phones and tablets. Read more<\/strong><\/a><\/p>\n Google couldn\u2019t wait until I\/O to show off the latest addition to the Pixel line and announced the new Pixel 8a last week. The handset starts at $499 and ships Tuesday. The updates, too, are what we\u2019ve come to expect from these refreshes. At the top of the list is the addition of the Tensor G3 chip. Read more<\/strong><\/a><\/p>\n Google\u2019s Pixel Tablet, called Slate, is now available. If you recall, Brian reviewed the Pixel Tablet<\/a> around this time last year, and all he talked about was the base. Interestingly enough, the tablet is available without it. Read more<\/strong><\/a><\/p>\n We\u2019ll be updating this post throughout the day \u2026<\/p>\n We\u2019re launching an AI newsletter! Sign up here<\/a> to start receiving it in your inboxes on June 5.<\/em><\/p>\nPrivacy concerns over AI voice call scans<\/strong><\/h2>\n
Updated security features<\/strong><\/h2>\n
Google TV<\/strong><\/h2>\n
Private Space feature<\/strong><\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/Picture1.gif?w=244\")
Google Maps gets geospatial AR<\/strong><\/h2>\n
Wear OS 5<\/strong><\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/wearos-watch-faces.png?w=680\")
TechCrunch Minute<\/strong><\/h2>\n
Even Elon Musk took note<\/strong><\/h2>\n
\u201cWeb\u201d search filter<\/strong><\/h2>\n
\n
Firebase Genkit<\/strong><\/h2>\n
AI ad nauseam<\/strong><\/h2>\n
Generative AI for learning<\/strong><\/h2>\n
![\"Google](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/ezgif-6-0a7708963f_356e73.gif?w=680\")
Quiz master<\/strong><\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/youtube-ai-quizzes.png?w=680\")
Gemma 2 updates<\/strong><\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/google-gemma2.png?w=680\")
Google Play<\/strong><\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2020\/07\/GettyImages-1214086455.jpg?w=680\")
Detecting scams during calls<\/strong><\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/4.-Scam-detection.jpeg?w=680\")
Ask Photos<\/strong><\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/Sundar-Ask-Photos-Gemini-IO-2024.png?w=680\")
All About Gemini<\/strong><\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/IMG_9483_4b691b.jpg?w=680\")
Gemini in Gmail<\/strong><\/h3>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/Gemini-15-2M-Context-Tokens-IO-2024.png?w=680\")
Gemini 1.5 Pro<\/strong><\/h3>\n
Gemini Live<\/strong><\/h3>\n
Gemini Nano<\/strong><\/h3>\n
Gemini on Android<\/strong><\/h3>\n
![\"Google](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/03\/Google-Maps-AI-highlights-.gif?w=335\")
Gemini on Google Maps<\/strong><\/h3>\n
Tensor Processing Units get a performance boost<\/strong><\/h2>\n
AI in search<\/strong><\/h2>\n
Generative AI upgrades<\/strong><\/h2>\n
Project IDX<\/strong><\/h2>\n
Veo<\/strong><\/h2>\n
Circle to Search<\/strong><\/h2>\n
Pixel 8a<\/strong><\/h2>\n
![\"Pixel](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/03\/Pixel-8-Call-Screen-Update.jpeg?w=680\"){kind=tracking}
Pixel Slate<\/strong><\/h2>\n
![\"\"](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2023\/06\/CMC_6728.jpg?w=680\")
![\"\"](\"https:\/\/lh7-us.googleusercontent.com\/jk2bDnLyD8qijuGpkRU-ttO2WJLdOrCsy2TlnpeyJfmYxWYBV8IhOW0ZepXP--wsvs2XRoJVx7EOZpRuWqmKfL0sx0xFREu83tfnSRhr8E96HYPky2sorCAK0Zoim5jqVE_pvkXukGybrOPNClyLc4c\")
![\"\"](\"https:\/\/lh7-us.googleusercontent.com\/3P9fIxoXrnrXyBhnMRZzAInuy14P3Rh9Nky_1mQPg3KgZOkn9R24_4UJU6eb1tfItcDjbDf3Vg3-VSENrNCHOsmRwEth8zg9Mk2kRANR91QphV_do_doQjJ5UJM3T66Ww_dCUWBu5i-rKEFy8OnRgxo\")
![\"Google](\"https:\/\/lh7-us.googleusercontent.com\/ldlbBR9mjoO70GhVuU6n_CJBxPckfUe6xew-BLTeB5XINkCYNgVNPUuxfcmKk_oLQbI-VTMwyhAFQiH8rBhQrvuCSJ4KP1bp32YrZkKk1X7C_meeCcLKi0PfNaGV3BL-A43JqjIGJbLNstn9V2hc7-c\")
![\"person](\"https:\/\/lh7-us.googleusercontent.com\/Ie9N6Q0zsnAKE4jLZnDaslRGioK87nW2da3HUoFI0Jix0zEvbTJI3yGZEjKoxC8pRs_Z6l-XVX-Bl7GQ8jLFGNBxgNKzG6e23zhs9p6BG62fwMveMETBlvDLWd9HZEiPJuoYJOM_XnUvhhtj8vy1NIg\")
![\"Read](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2024\/05\/google-io-2024-banner.jpg\")
<\/a><\/p>\n<\/div>\n