{"id":267283,"date":"2026-10-06T19:56:50","date_gmt":"2026-10-06T19:56:50","guid":{"rendered":"https:\/\/entertainment.runfyers.com\/index.php\/2026\/10\/06\/the-next-hurdle-for-ai-agents-getting-websites-to-let-them-in-techcrunch\/"},"modified":"2026-10-06T19:56:50","modified_gmt":"2026-10-06T19:56:50","slug":"the-next-hurdle-for-ai-agents-getting-websites-to-let-them-in-techcrunch","status":"publish","type":"post","link":"https:\/\/entertainment.runfyers.com\/index.php\/2026\/10\/06\/the-next-hurdle-for-ai-agents-getting-websites-to-let-them-in-techcrunch\/","title":{"rendered":"The next hurdle for AI agents: getting websites to let them in | TechCrunch"},"content":{"rendered":"<p> <br \/>\n<\/p>\n<div>\n<p id=\"speakable-summary\" class=\"wp-block-paragraph\">Personal AI agents, like Meta\u2019s Muse, Instinct, ChatGPT\u2019s Dots, and others, are kicking off a new wave of consumer AI that involves more than just responding to queries. These AI agents can actually get things done on users\u2019 behalf, like booking flights, making dinner reservations, ordering groceries, and more, all without having to be technical enough to set up OpenClaw or some other agent on your own computer.<\/p>\n<p class=\"wp-block-paragraph\">But consumers adopting these agents are often running into issues when the website on the other end of their request blocks the AI from completing the job.<\/p>\n<p class=\"wp-block-paragraph\">Most recently, <a href=\"https:\/\/www.bloomberg.com\/news\/articles\/2026-09-21\/amazon-blocks-meta-s-muse-ai-agent-from-its-retail-site\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Amazon began blocking Meta\u2019s Muse AI agent<\/a> from its retail site, meaning that the agent could no longer browse or make purchases from its product catalog.<\/p>\n<p class=\"wp-block-paragraph\">However, the problem is not limited to Amazon or Muse. User complaints across social media indicate their agents are often running into similar blocks. Some of these are intentional, like Amazon\u2019s, while others are the result of websites\u2019 traditional anti-bot measures designed to protect against spam and malicious activity.<\/p>\n<p class=\"wp-block-paragraph\">What\u2019s not clear to the end user is whether or not the block is accidental on the website owner\u2019s part, which leaves them frustrated with both the service provider or retailer and the agent trying to facilitate the transaction.<\/p>\n<p class=\"wp-block-paragraph\">For instance, TechCrunch has seen a <a rel=\"nofollow\" href=\"https:\/\/x.com\/BohuangMars\/status\/2107188842969362649\" target=\"_blank\">han<\/a><a href=\"https:\/\/x.com\/BohuangMars\/status\/2107188842969362649\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">d<\/a><a rel=\"nofollow\" href=\"https:\/\/x.com\/BohuangMars\/status\/2107188842969362649\" target=\"_blank\">ful<\/a> of <a href=\"https:\/\/x.com\/buzaza\/status\/2107143801504731436\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">complaints<\/a> on social media claiming that Muse couldn\u2019t complete a purchase on Walmart\u2019s retail website. (<a href=\"https:\/\/www.nbcnews.com\/tech\/tech-news\/ai-agent-muse-shop-e-commerce-amazon-wal-mart-wayfair-buy-rcna599069\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">NBC also reported<\/a> that this was a problem Walmart had faced in its tests of the AI agent.)<\/p>\n<p class=\"wp-block-paragraph\">But a spokesperson for Walmart told us these weren\u2019t intentional. In fact, Walmart is partnered with Muse, as was <a href=\"https:\/\/techcrunch.com\/2026\/09\/23\/everything-new-coming-to-metas-ai-agent-muse\/\" target=\"_blank\" rel=\"noopener\">announced at Meta\u2019s Connect developer conference<\/a> in September. The company said it wants to be where its customers are, and that now includes being available through AI agent experiences.<\/p>\n<p class=\"wp-block-paragraph\">The issue that Walmart customers often appear to be running into (see below) is that the website presents a button that has to be clicked to verify the visitor is human. If that experience is interrupted, the check could fail, and the agent gets booted out. <\/p>\n<p class=\"wp-block-paragraph\">It\u2019s an indication that the current technology we use to verify humans and keep websites safe from spam and bots may not be the best fit for the agent-first era of the web we\u2019re now moving into. <\/p>\n<p class=\"wp-block-paragraph\">To address this problem, industry partners including Meta, Walmart, Stripe, Sierra, Genesys, Rocket, NiCE, and Decagon have begun <a href=\"https:\/\/www.facebook.com\/business\/news\/a-new-way-for-businesses-and-personal-agents-to-work-together\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">working on an open standard<\/a> that would dictate how AI agents can communicate with businesses, helping to separate the good bots acting on behalf of the user and the bad. This protocol will be specifically focused on agent-to-agent communication for the purposes of online commerce, Meta said in a <a href=\"https:\/\/www.facebook.com\/business\/news\/a-new-way-for-businesses-and-personal-agents-to-work-together\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">blog post.<\/a><\/p>\n<figure class=\"wp-block-embed is-type-rich is-provider-x wp-block-embed-x\">\n<div class=\"wp-block-embed__wrapper\">\n<blockquote class=\"twitter-tweet\" data-width=\"500\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">This is frustrating.  <a rel=\"nofollow\" href=\"https:\/\/x.com\/Walmart?ref_src=twsrc%5Etfw\" target=\"_blank\">@Walmart<\/a> in their attempt to stop agent from making purchases has gone too far and is essentially blocking actual human from making purchase on their website.  I tried to purchase something directly and I can&#8217;t even sign in. <a rel=\"nofollow\" href=\"https:\/\/t.co\/CRAgkrCCa0\" target=\"_blank\">pic.twitter.com\/CRAgkrCCa0<\/a><\/p>\n<p>\u2014 Kevin Wang (@buzaza) <a rel=\"nofollow\" href=\"https:\/\/x.com\/buzaza\/status\/2107143801504731436?ref_src=twsrc%5Etfw\" target=\"_blank\">October 5, 2026<\/a><\/p><\/blockquote>\n<\/div>\n<\/figure>\n<p class=\"wp-block-paragraph\">In the meantime, consumer confusion abounds. <\/p>\n<p class=\"wp-block-paragraph\">Although Walmart is at least trying to be accessible via AI agents, other companies aren\u2019t so keen on the idea or haven\u2019t fully embraced the technology.<\/p>\n<p class=\"wp-block-paragraph\">For example, flight booking is currently one of the main use cases being positioned for agents, but many people have reported issues with airlines rejecting their agents\u2019 requests. <\/p>\n<p class=\"wp-block-paragraph\">Delta, for instance, told us that the company is taking steps to protect customers from unauthorized automated activity, and that if it were to open up to AI agents, it would have to be done with \u201csecurity and the customer experience in mind.\u201d<\/p>\n<p class=\"wp-block-paragraph\">\u201cWhile Delta does not currently have a partnership or integration that enables a third-party AI agent to shop for or book Delta flights on a customer\u2019s behalf on our digital platforms, we\u2019re continuing to evaluate how new technology, including external AI agents, could enhance the way customers engage with Delta,\u201d said Delta\u2019s general manager of Global Communications, Heena Chavda, in response to our questions about whether or not it was blocking agents, as some users complained.<\/p>\n<p class=\"wp-block-paragraph\">United was not quite as direct as Delta, but a spokesperson pointed us to the part of its Terms of Use policy, which says that the user agrees \u201cnot to use any robot, spider, other automatic device, or manual process to monitor or copy this website or the content contained therein or for any other unauthorized purpose without United\u2019s prior written permission.\u201d (United didn\u2019t respond to a follow-up question about whether it was blocking specific personal AI agents, like Muse.) <\/p>\n<p class=\"wp-block-paragraph\">Early adopters\u2019 <a href=\"https:\/\/x.com\/rjchint\/status\/2104405141130789271\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">complaints<\/a> <a href=\"https:\/\/x.com\/timothyfcook\/status\/2107174688459784435\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">across<\/a> <a href=\"https:\/\/x.com\/maulik_sh\/status\/2107156908356329501\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">social<\/a> <a href=\"https:\/\/x.com\/jbillow\/status\/2104244311256887697\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">media<\/a> have referenced a number of brands that are reportedly <a href=\"https:\/\/x.com\/ryanfoxeth\/status\/2102023108974506154\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">blocking<\/a> AI agents, <a href=\"https:\/\/x.com\/NanjuanS\/status\/2104323028058513756\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">including<\/a> <a href=\"https:\/\/x.com\/RameshUtta83629\/status\/2105820595963531657\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Yelp<\/a>, <a href=\"https:\/\/x.com\/kallasmaa\/status\/2107071034486849541\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">eBay<\/a>, <a href=\"https:\/\/x.com\/Aerocles\/status\/2106904960768291204\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Zillow<\/a>, <a href=\"https:\/\/x.com\/JoshExile82\/status\/2106949262252941461\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Pizza Hut<\/a>, <a href=\"https:\/\/x.com\/kesslerIO\/status\/2104397316673462339\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Adidas<\/a>, <a rel=\"nofollow\" href=\"https:\/\/x.com\/michelleoconnor\/status\/2107446501605838977\" target=\"_blank\">and<\/a> <a href=\"https:\/\/x.com\/akash_bhatia\/status\/2106527240737480719\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">various<\/a> <a href=\"https:\/\/x.com\/DVallota\/status\/2107487367984758846\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">airlines<\/a>. Some believe that the blocking activity seems to have recently <a href=\"https:\/\/x.com\/Jessicalessin\/status\/2106844211773595663\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">increased<\/a>.<\/p>\n<p class=\"wp-block-paragraph\">A couple of people even said eBay <a href=\"https:\/\/x.com\/kallasmaa\/status\/2107071034486849541\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">suspended<\/a> their <a href=\"https:\/\/x.com\/kallasmaa\/status\/2107071034486849541\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">accounts<\/a> because of their use of agentic AI. Another complained that <a href=\"https:\/\/x.com\/jyothiwrites\/status\/2107493883584421973\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Google kicked out Instinct<\/a> while it was cleaning up their Gmail inbox for them.<\/p>\n<figure class=\"wp-block-embed is-type-rich is-provider-x wp-block-embed-x\"\/>\n<p class=\"wp-block-paragraph\">Despite the push to have agents book tables at restaurants and other tedious tasks, Yelp told us that it doesn\u2019t permit non-human traffic on its platform unless the agent has paid to access Yelp\u2019s content and data through its <a href=\"https:\/\/business.yelp.com\/data\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">data licensing program<\/a>. That means agents trying to use its site to get a quote, add a user to a waitlist, or book them a table will likely fail if they don\u2019t have a formal partnership with Yelp.<\/p>\n<p class=\"wp-block-paragraph\">eBay told TechCrunch that its approach is not to prohibit all third-party shopping agents from completing purchases. Instead, its policies restrict unauthorized agents and actions, like automated scraping and model training. <\/p>\n<p class=\"wp-block-paragraph\">Adidas, Zillow, and Pizza Hut either didn\u2019t respond or declined to provide a comment by the time of publication. <\/p>\n<p class=\"wp-block-paragraph\">One major company involved in this space is Cloudflare, which offers sites protection from AI agents that could be training on its data and managing other automated activity. Some <a href=\"https:\/\/www.threads.com\/@karelbaloun\/post\/DdjwBeeGvR4\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">suspect<\/a> <a href=\"https:\/\/x.com\/TheJordude\/status\/2106417869764354260\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">that<\/a> Cloudflare and other content delivery networks (CDNs) like it are currently disrupting Muse traffic. This is <a href=\"https:\/\/blog.cloudflare.com\/accountable-mixed-use-ai-crawlers\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">attributed<\/a> to a change to crawler defaults on September 15, which allows site owners to block AI training while allowing other types of bots. Existing sites that were previously set to block AI bots for security reasons were then shifted to block AI agents on any pages that had ads.<\/p>\n<p class=\"wp-block-paragraph\">However, when we asked if Cloudflare had any insight into what was happening with the blocking of these personal AI agents, like Muse and others, the company said it didn\u2019t have any specific data to share at this time. It pointed us instead to its free public data hub, <a href=\"https:\/\/radar.cloudflare.com\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Cloudflare Radar<\/a>. The site shows the increase in bot activity (which Cloudflare has also commented on previously), but it doesn\u2019t help differentiate between the good bots and the bad ones.<\/p>\n<p class=\"wp-block-paragraph\">Cloudflare also has a vested interest in mitigating AI traffic to websites, as it <a href=\"https:\/\/techcrunch.com\/2025\/07\/01\/cloudflare-launches-a-marketplace-that-lets-websites-charge-ai-bots-for-scraping\/\" target=\"_blank\" rel=\"noopener\">now runs a marketplace<\/a> where AI bots have to pay for the data they access. It recently began testing <a href=\"https:\/\/techcrunch.com\/2026\/08\/07\/cloudflare-launches-kitesurf-a-browser-built-for-ai-agents\/\" target=\"_blank\" rel=\"noopener\">a new web browser built specifically for AI agents.<\/a><\/p>\n<p class=\"wp-block-paragraph\">Given the rapidly evolving space, Meta\u2019s decision to form specific brand partnerships for Muse access seems to be the right way to go, as is its decision to focus on new standards. In doing so, it assures its customers that its agent, at least, is welcome at these partners\u2019 sites \u2014 so any issues with access would be reportable bugs.<\/p>\n<p class=\"wp-block-paragraph\">Meta also maintains a list of \u201cconnectors\u201d (a.k.a. partners) inside the Muse app, which continues to grow. Many of Meta\u2019s <a href=\"https:\/\/techcrunch.com\/2026\/09\/23\/everything-new-coming-to-metas-ai-agent-muse\/\" target=\"_blank\" rel=\"noopener\">newly announced partners<\/a> haven\u2019t yet shown up in this list. But given Muse\u2019s rapid <a href=\"https:\/\/www.cnbc.com\/2026\/10\/03\/meta-muse-shopping-ai-agent.html\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">expansion<\/a>, which now includes <a href=\"https:\/\/about.fb.com\/news\/2026\/09\/introducing-muse-small-business\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">partnered tools for small businesses<\/a>, a bigger reorganization of Muse\u2019s connector list is likely soon to be underway.<\/p>\n<p class=\"wp-block-paragraph\">\u201cTurning away a personal agent means turning away the customer behind it. We\u2019d rather work with businesses to address the underlying concerns than see them lose that customer,\u201d Meta said in its announcement about the newly announced plans to develop a standard. \u201cThere is a path that is good for users and good for businesses. It starts with clear norms, gives businesses more predictable control, and evolves toward a more efficient way for agents to work together.\u201d <\/p>\n<\/div>\n<p><em>When you purchase through links in our articles, <a href=\"https:\/\/techcrunch.com\/techcrunch-affiliate-monetization-standards\/\" target=\"_blank\" rel=\"noopener\">we may earn a small commission<\/a>. This doesn\u2019t affect our editorial independence.<\/em><\/p>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><br \/>\n<br \/><br \/>\n<br \/><a href=\"https:\/\/techcrunch.com\/2026\/10\/06\/the-next-hurdle-for-ai-agents-getting-websites-to-let-them-in\/\" target=\"_blank\" rel=\"noopener\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Personal AI agents, like Meta\u2019s Muse, Instinct, ChatGPT\u2019s Dots, and others, are kicking off a new wave of consumer AI that involves more than just responding to queries. These AI agents can actually get things done on users\u2019 behalf, like booking flights, making dinner reservations, ordering groceries, and more, all without having to be technical [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":267284,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[14],"tags":[],"class_list":{"0":"post-267283","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-tech"},"_links":{"self":[{"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/posts\/267283","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/comments?post=267283"}],"version-history":[{"count":0,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/posts\/267283\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/media\/267284"}],"wp:attachment":[{"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/media?parent=267283"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/categories?post=267283"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/tags?post=267283"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}