\n<\/p>\n
Anthropic said its models exploited websites on the internet, including some run by U.S. government agencies, and it will turn off live internet access for all of its internal evaluations until the frontier lab is sure it can monitor and control its AI agents.<\/p>\n
The incidents, disclosed in a blog post<\/a>, involved AI agents tasked to solve problems<\/a> seeking resources on the internet. In the process, they exploited software flaws, avoided paywalls and anti-bot restrictions, used URL shortening services to smuggle information pass restrictions, and even submitted<\/a> a false murder tip to the Philadelphia police.<\/p>\n Anthropic said it discovered these new issues in a review of its model\u2019s activities that began in July, underscoring the lab\u2019s lack of awareness of its software\u2019s behavior.<\/p>\n Notably, the company said that alignment training was not yet sufficient for skills like search and computer use that are central to its pitch that AI agents will be used by any professional who relies on digital tools.<\/p>\n The behaviors Anthropic disclosed are similar to incidents involving OpenAI agents that collaborated<\/a> to break into various websites in search of information, including some run by the Australian government.<\/p>\n Anthropic previously disclosed that its models had broken into<\/a> external systems. The frontier lab said it considered today\u2019s disclosures \u201csignificantly less severe from an alignment and security perspective\u201d than those it announced before.<\/p>\n However, the lab still said it had \u201cturned off live internet access\u201d for \u201call our internal evaluations\u201d until it is certain it can monitor and control its agents.<\/p>\n It\u2019s not clear what that means, but Sydney Von Arx, the founder of Nightingale, an AI safety organization, told TechCrunch in an interview before this disclosure that developing models on a data center cut off from the open internet would be very challenging for researchers to use, and for the progress of the models, which benefit from internet access.<\/p>\n \u201cYou have to align them at some point,\u201d Von Arx said. \u201cIf the AIs are released to production and never have access to the internet, that\u2019s not a very useful tool.\u201d<\/p>\n Anthropic said the behavior was a result of flaws in the lab\u2019s training environments, which led the models to believe they would be rewarded for finding loopholes or avoiding restrictions, a behavior called \u201creward hacking.\u201d<\/p>\n The company said it would stop running some of its evaluations or move them offline, and has built tooling to detect and block this behavior. This tooling was tested against the kind of incidents disclosed today and blocked them; it\u2019s not clear what evidence will prompt Anthropic to return live internet access to its internal evaluations.<\/p>\n Anthropic also said it would migrate its internal AI agents to \u201ccentrally managed infrastructure with strong containment,\u201d and is beginning to using safety classifiers more frequently to monitor those agents.<\/p>\n<\/div>\n When you purchase through links in our articles, we may earn a small commission<\/a>. This doesn\u2019t affect our editorial independence.<\/em><\/p>\n