Social media is abuzz with examples of Google\u2019s new AI Overview product saying weird stuff, from telling users to put glue on their pizza<\/a> to suggesting they eat rocks<\/a>. The messy rollout means Google is racing to manually disable AI Overviews for specific searches as various memes get posted, which is why users are seeing so many of them disappear<\/a> shortly after being posted to social networks.<\/p>\n<\/div>\n It\u2019s an odd situation, since Google has been testing AI Overviews for a year now \u2014 the feature launched in beta in May 2023 as the Search Generative Experience<\/a> \u2014 and CEO Sundar Pichai has said the company served over a billion queries in that time.<\/p>\n<\/div>\n But Pichai has also said that Google\u2019s brought the cost of delivering AI answers down by 80 percent over that same time, \u201cdriven by hardware, engineering and technical breakthroughs.\u201d It appears that kind of optimization might have happened too early, before the tech was ready.<\/p>\n<\/div>\n \u201cA company once known for being at the cutting edge and shipping high-quality stuff is now known for low-quality output that\u2019s getting meme\u2019d,\u201d one AI founder, who wished to remain anonymous, told The Verge<\/em>.<\/p>\n<\/div>\n Google continues to say that its AI Overview product largely outputs \u201chigh quality information\u201d to users. \u201cMany of the examples we\u2019ve seen have been uncommon queries, and we\u2019ve also seen examples that were doctored or that we couldn\u2019t reproduce,\u201d Google spokesperson Meghann Farnsworth said in an email to The Verge<\/em>. Farnsworth also confirmed that the company is \u201ctaking swift action\u201d to remove AI Overviews on certain queries \u201cwhere appropriate under our content policies, and using these examples to develop broader improvements to our systems, some of which have already started to roll out.\u201d<\/p>\n<\/div>\n Gary Marcus, an AI expert and an emeritus professor of neural science at New York University, told The Verge<\/em> that a lot of AI companies are \u201cselling dreams\u201d that this tech will go from 80 percent correct to 100 percent. Achieving the initial 80 percent is relatively straightforward since it involves approximating a large amount of human data, Marcus said, but the final 20 percent is extremely challenging. In fact, Marcus thinks that last 20 percent might be the hardest thing of all.<\/p>\n<\/div>\n \u201cYou actually need to do some reasoning to decide: is this thing plausible? Is this source legitimate? You have to do things like a human fact checker might do, that actually might require artificial general intelligence,\u201d Marcus said. And Marcus and Meta\u2019s AI chief Yann LeCun both agree<\/a> that the large language models that power current AI systems like Google\u2019s Gemini and OpenAI\u2019s GPT-4 will not be what creates AGI. <\/p>\n<\/div>\n Look, it\u2019s a tough spot for Google to be in. Bing went big on AI before Google did with Satya Nadella\u2019s famous \u201cwe made them dance<\/a>\u201d quote, OpenAI is reportedly<\/a> working on its own search engine, a fresh AI search startup is already worth $1 billion<\/a>, and a younger generation of users who just want the best experience are switching to TikTok. <\/em>The company is clearly feeling the pressure to compete, and pressure is what makes for messy AI releases. Marcus points out that in 2022, Meta released an AI system called Galactica that had to be taken down shortly after its launch because, among other things, it told people to eat glass<\/a>. Sounds familiar.<\/p>\n<\/div>\n Google has grand plans for AI Overviews \u2014 the feature as it exists today is just a tiny slice of what the company announced last week. Multistep reasoning for complex queries, the ability to generate an AI-organized results page, video search in Google Lens \u2014 there\u2019s a lot of ambition here. But right now, the company\u2019s reputation hinges on just getting the basics right, and it\u2019s not looking great.<\/p>\n<\/div>\n