\n<\/p>\n
We\u2019ve been seeing increasingly dire warnings from AI researchers about the dangers of artificial intelligence, and even comments from OpenAI CEO Sam Altman<\/a> that it may be time to \u201cpace\u201d AI development. But what would that actually look like?<\/p>\n In a new blog post<\/a>, Anthropic CEO Dario Amodei not only echoed the call to \u201cpace the frontier,\u201d but also outlined three broad strategies for doing so. And he said Anthropic is \u201cunilaterally committing\u201d to one of them.<\/p>\n The debate over AI safety and alignment<\/a> intensified this week after researcher Jacob Coxon wrote that he\u2019s resigning from Anthropic<\/a> over concerns that the leading AI companies are \u201cgambling with our lives\u201d while the people building the technology \u201cearnestly believe it could kill us all by the end of the decade,\u201d a claim repeated by others at Anthropic<\/a>.<\/p>\n Amodei\u2019s post doesn\u2019t didn\u2019t explicitly mention Coxon\u2019s resignation or his concerns, but the CEO wrote that two things convinced him it\u2019s time to take a more cautious approach to AI development: the OpenAI-HuggingFace hack<\/a>, and the fact that \u201cAI has been advancing drastically faster\u201d in recent months, particularly with its \u201cgrowing ability to build the next generation of AI.\u201d<\/p>\n \u201cWe must slow the pace at which we improve the capabilities of AI models,\u201d Amodei wrote. \u201cProgress will still seem fast, and we must make wise use of the time we gain.\u201d<\/p>\n His proposed first step would involve \u201cembedded evaluators\u201d from third-party organizations like METR<\/a> \u2014 evaluators who can verify that AI companies are actually following their pacing and safety commitments and can also ensure that safety incidents get reported. (OpenAI was recently criticized for not reporting an incident where its AI agents took over a German wiki form<\/a>.)\u00a0<\/p>\n Amodei compared these evaluators to regulators who have been embedded with bank employees, and he said that inviting them in is \u201csomething Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match).\u201d That means giving evaluators company badges, desks, and laptops, and providing access \u201cmostly comparable to what internal risk assessment teams have,\u201d with exceptions when required by law or contracts.<\/p>\n Next, Amodei called for the leading AI companies \u201cwithin democratic countries\u201d to coordinate\u00a0 \u201ccommon safety standards as well as limits on the rate of unchecked AI progress.\u201d\u00a0<\/p>\n Such coordination might seem unlikely, both due to the apparent animosity between Altman and Amodei<\/a> and also because their companies are reportedly worried that a coordinated pause could lead to antitrust scrutiny<\/a>. Amodei alluded to that concern in his post, writing that \u201cfor antitrust reasons, it\u2019s helpful for the US government to mediate or at least enable these discussions \u2014 they don\u2019t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.\u201d<\/p>\n Amodei also acknowledged the spectre of Chinese AI dominance<\/a> that\u2019s often raised an argument against slowing development. But he said that if the US government and tech companies take steps like refusing to sell powerful chips or semiconductor manufacturing equipment to Chinese companies, as well as cracking down on model distillation<\/a>, they could \u201cslow China\u2019s progress enough to widen America\u2019s lead significantly over the next 3\u20135 years.\u201d<\/p>\n Lastly, Amodei called for \u201cglobal coordination,\u201d where the United States and its allies \u201cattempt to coordinate with authoritarian governments, to the extent this is possible.\u201d Amodei said this would mean \u201ccooperation with China,\u201d and he admitted that there are \u201cstark limits on what can be achieved,\u201d but he still suggested there might be opportunities for agreement, even if it\u2019s just \u201cprohibiting certain narrow and obviously dangerous uses of AI, such as using AI for the production of biological weapons or allowing users to do so.\u201d<\/p>\n With Amodei\u2019s past willingness to acknowledge AI\u2019s potential dangers, and with the company\u2019s relative openness to certain forms of regulation, some AI boosters have already criticized him as a doomer whose comments have fed the current AI backlash. In response, Amodei said he\u2019s tried to offer a \u201cbalanced\u201d perspective\u201d and argued that the backlash is \u201cfundamentally a crisis of trust,\u201d<\/a> as people have become skeptical of tech companies, the tech industry, and the government.<\/p>\n Industry critics have also been skeptical about these apocalyptic AI warnings, suggesting that they\u2019re a distraction from the harm that the technology is already causing<\/a>.<\/p>\n Journalist Brian Merchant, for example, wrote that he has yet to see<\/a> \u201ca credible, step-by-step documentation of how exactly AI might move from self-recursively improving AI to killing every single human on the planet\u201d; he also suggested that proposals similar to Amodei\u2019s \u201cwould likely only wind up serving Anthropic and OpenAI; it\u2019s what regulatory capture looks like in action.\u201d<\/p>\n In his new post, Amodei wrote that he continues \u201cto believe that AI can enormously improve the quality of human life.\u201d<\/p>\n \u201cMy desire to achieve these benefits is undimmed,\u201d he said. \u201cBut the benefits will only be achieved if we build the technology in the right way, and \u2014 so long as we use the time we gain well \u2014 it is worth taking unusually deliberate care to get it right.\u201d<\/p>\n<\/div>\n When you purchase through links in our articles, we may earn a small commission<\/a>. This doesn\u2019t affect our editorial independence.<\/em><\/p>\n