\n<\/p>\n
We\u2019ve all pulled up Street View on Google Maps to show a friend what our childhood home looked like, or dropped that little person icon onto the streets of Paris to see if we booked a hotel in a cool neighborhood. Imagine being able to do that, but in a more immersive, interactive way that allows you to really simulate the street and its environs, and even do things like adjust the weather or see what it would look like in a \u201cDay After Tomorrow\u201d scenario.<\/p>\n
That\u2019s one of the goals of Google\u2019s latest integration. Starting today, Google DeepMind is connecting Street View to Project Genie,<\/a> the company\u2019s general-purpose world model that can generate diverse, interactive environments. The new feature launched during the Google I\/O developer conference.\u00a0<\/p>\n \u201cIt\u2019s really powerful for both the agent [and robotics] use case and for humans to play with, and that\u2019s always been the thesis of Genie,\u201d Jack Parker-Holder, a research scientist on DeepMind\u2019s open-endedness team, told TechCrunch.<\/p>\n He gave the example of a new robot being deployed in London, which rarely sees the sun. Genie could, Parker-Holder says, simulate those scarce occasions when the sun glints off the Victorian housing, so the rays don\u2019t shock the robot when it happens.<\/p>\n \u201cSimultaneously, you might say, \u2018I\u2019m going to New York City, but not this time of year,\u2019\u201d he continued. \u201c\u2018It\u2019s going to be snowy. I want to see what that block looks like in the snow.\u2019\u201d\u00a0<\/p>\n Google has been collecting Street View data for 20 years via cars with cameras and individuals strapped with \u201ctracker backpacks.\u201d The tech giant has collected north of 280 billion images across 110 countries and seven continents.\u00a0<\/p>\n \u201cWith Street View, we have imagery from a large quantity of the world,\u201d Jack said. \u201cYou can imagine how potentially powerful it is to combine this rich source of real-world information and data with an ability to simulate worlds.\u201d<\/p>\n Google released its latest world model Genie 3 for research preview<\/a> last August and opened up access to the tool to Google AI Ultra subscribers in the U.S. in January, allowing customers to create interactive game worlds from text prompts or images. The goal is to use Genie for educational experiences, gaming, and robotics training.\u00a0<\/p>\n Genie 3 is already helping to power one of Waymo\u2019s simulators<\/a> to train its self-driving cars on \u201cexceedingly rare events\u201d like tornadoes or casual elephant encounters. Adding Street View data to that could help Waymo prepare to launch in more cities around the globe.<\/p>\n Waymo has its own simulator that it relied on to scale to 11 U.S. cities and test its AI driver in several more. The difference with Genie, says Parker-Holder, is that those are all from the car\u2019s point of view. Street View allows for not only simulating a world anchored to a real place, but also shifting the point of view to other types of agents, like a human or a robot.\u00a0<\/p>\n Google is launching Street View in Genie to some Ultra users in the United States starting today, with access rolling out at scale over time. Global Ultra users will gain access over the next few weeks, per the company.<\/p>\n The researchers\u2019 goal is to put this new capability into as many hands as possible, per Diego Rivas, a product manager at DeepMind. He cautioned that Street View in particular and Genie in general is still an experiment, so there\u2019s much to improve upon in terms of accuracy.<\/p>\n In the samples the Google team showed me \u2014 including an underwater simulation of a neighborhood I used to live in \u2014 the results are impressive and recognizable, but still video game quality rather than photorealistic. The models are also not yet physics-aware, meaning they don\u2019t yet understand cause and effect. For example, in a simulation of a woman running through a snowy Joshua Tree, she ran right through cacti and bushes.<\/p>\n Compare that to, say, Google\u2019s image generator Nano Banana \u2014 which can now generate perfect text in infographics \u2014 or its video generator Veo \u2014 which understands that paper boats drift on water currents, smoke disperses into the air, and fabric drapes over forms.\u00a0<\/p>\n Physics isn\u2019t hard-coded into these models; they learn it intuitively over time through passive observation, as a living being would.\u00a0<\/p>\n \u201cI think for this kind of model, it\u2019s maybe six to 12 months behind video in terms of the accuracy and quality, so I think it\u2019s something we will solve,\u201d Parker-Holder said.\u00a0<\/p>\n Jonathan Herbert, director of Google Maps who started on the Street View team as an intern 12 years ago, said that Genie can\u2019t yet create a faithful reconstruction of a street. He thinks the real breakthrough is the AI\u2019s spatial continuity. If you turn 360 degrees, the AI correctly remembers and simulates the environment behind you. From that point on, the model can build a new environment on top of that.<\/p>\n \u201cWe have long thought about how we can build out the best and richest model of the world on top of Street View data,\u201d Herbert said. \u201cIt\u2019s definitely been an idea of ours to use Maps Data in new ways and for new kinds of AI research for a pretty long time.\u201d<\/p>\n<\/div>\n When you purchase through links in our articles, we may earn a small commission<\/a>. This doesn\u2019t affect our editorial independence.<\/em><\/p>\n