\n<\/p>\n
A group of independent AI researchers discovered<\/a> that internally deployed OpenAI agents began posting on an obscure German wiki forum in order to collaborate on evaluations. They appear to have worked together for over a month without OpenAI\u2019s knowledge.<\/p>\n A spokesperson for the frontier lab would not say whether these agents were indeed from OpenAI, or when the lab became aware of their actions. They noted that OpenAI had not been given a chance to review the researchers\u2019 findings before they were published today but said that the AI model maker is \u201cnow carefully reviewing its contents and will take any necessary next steps.\u201d<\/p>\n After OpenAI revealed<\/a> that agents working on an internal evaluation were able to access the open internet and exploit Hugging Face, a group of researchers \u2014 Nightingale CEO Sydney Von Arx, AI researcher Cormac Slade Byrd, Redwood Research\u2019s Spencer Kitts, and AI Futures Project\u2019s Thomas Larsen \u2014 began searching for evidence of other rogue AI agents.<\/p>\n They put themselves in the agents\u2019 shoes to figure out their needs and deployed their own LLM to identify likely places the agents might congregate. They then identified a wiki-hosting service that would be particularly vulnerable: The DseWiki is 25 years old but had just 10 edits in the last 20 years \u2014 before the agents arrived.<\/p>\n Starting on May 11, the researchers tracked agents, many with OpenAI identifiers in their names<\/a>, trying and finally succeeding in editing the German wiki site. By mid-June, the agents were actively trading tips on how to answer web search questions posed under time limits, sharing answers to pass the tests. A human moderator apparently noticed these posts and started deleting them, seeing them as spam, which the agents fought back against, trying to hide their posts from alphabetical sorting by beginning each with the string \u201cZZZ.\u201d<\/p>\n \u201cThe administrator spent the next 5 days fighting a losing battle against the agents, deleting an average of 100 pages a day while the agents created about 400 new pages per day,\u201d the researchers wrote. \u201cOn June 22, the agent edits suddenly stop, and the administrator spends each evening over the next 5 weeks deleting the remaining agent-created pages. Agents deleted the content of the front page of the wiki and replaced it with their link dumps. The moderator restored the original version. This back-and-forth happened nine times.\u201d<\/p>\n Eventually, someone at OpenAI appears to have noticed \u2014 the researchers track apparently human browsers coming from OpenAI IP addresses, and then agent activity drops to near zero, before spiking as OpenAI-affiliated visitors attempt to recover the deleted pages.<\/p>\n While OpenAI has made vague disclosures about agents gaining unauthorized access to external communication services, it had not previously disclosed this specific incident, or said how often this type of thing has happened. While no obviously illegal activity appears to have occurred during this incident, it raises more questions about whether OpenAI can monitor and control the technology it is building, at a time when there is limited public oversight or input into frontier AI labs.<\/p>\n \u201cThe lack of any real federal AI governance means that frontier companies can pick and choose when they disclose incidents like this,\u201d Representative Lori Trahan (D-MA) said. Trahan has introduced a bipartisan bill, the Frontier Act, that would require labs to disclose these incidents and host independent auditors. <\/p>\n AI safety researchers are concerned that the latest generation of powerful models, whose reasoning is increasingly opaque<\/a> to its creators, could take actions that harm people. Astra, released yesterday by OpenAI, appears to be its most capable model yet.<\/p>\n The company says Astra is also the model most likely to follow human direction, but third-party researchers who were asked to evaluate it expressed concern about its alignment. The U.K.\u2019s AI Safety Institute and Apollo Research both reported concerns that the model might be aware that it was being evaluated and potentially hide its real behavior.<\/p>\n \u201cApollo believes that, given the higher rates of eval awareness and limited evaluation window, low rates of misbehavior here do not provide substantial evidence about the model\u2019s alignment or misalignment,\u201d the researchers wrote<\/a> in their evaluation.<\/p>\n<\/div>\n When you purchase through links in our articles, we may earn a small commission<\/a>. This doesn\u2019t affect our editorial independence.<\/em><\/p>\n