\n<\/p>\n
By late afternoon, the spreadsheets were converging on an answer, but it was a strange one. They predicted that in most scenarios, Claude 3.7 Sonnet wouldn\u2019t meaningfully increase the risk of a catastrophic biological attack. If you took the median of all the possible outcomes, it fell well below Anthropic\u2019s threshold for triggering ASL-\u00ad3. But the mean was a different story. The spreadsheets had included a few outlier scenarios, in which Claude 3.7 Sonnet helped someone make a novel influenza strain, sparking a pandemic that spread around the world. In the most extreme, low-probability scenarios, at the very tip of the long tail, the calculated death toll was huge, in the tens of millions. And when they included those scenarios in their spreadsheets, they got a number that looked catastrophic.<\/p>\n
The Frontier Red Team worked past midnight that night, scrutinizing the threat models and trying to arrive at a more definitive answer. In the end, they decided that Claude 3.7 Sonnet didn\u2019t meet the higher risk threshold that would trigger ASL-\u00ad3, and could be released with no additional safeguards. But the company pushed back the model\u2019s release by a week, in order to run additional tests on it.<\/p>\n
This was an odd way to run a business. Most tech companies didn\u2019t calculate potential death tolls for their products, or go to such extreme lengths to anticipate what horrors they might enable. But Anthropic wasn\u2019t the only AI company looking for these risks. OpenAI and Google Deep-Mind had red teams, too, and all the labs partnered with external AI testing companies to evaluate their models for dangerous capabilities. They were all poking and prodding their models, trying to figure out what they could and couldn\u2019t do. And over the past year, the red teams had been discovering some concerning things.<\/p>\n
Some of them fell into the category of \u201cmisuse risks\u201d\u2014\u00ad the dangerous things an AI model could do in the hands of a malevolent human, like helping them build a chemical weapon or carry out a cyberattack. But other risks, known as \u201calignment risks,\u201d emerged from the behavior of the models themselves. In January 2024, Anthropic published a paper titled \u201cSleeper Agents: Training Deceptive LLMs That Persist Through Safety Training.\u201d The paper, led by a researcher named Evan Hubinger, described an experiment in which Anthropic\u2019s team had deliberately trained AI models to behave deceptively. Then they tried to fix the deception using every safety technique in their tool kit\u2014\u00adsupervised fine-\u00adtuning, reinforcement learning, adversarial training. None of it worked. The deceptive behavior persisted through all of it. Even worse, their interventions actually seemed to backfire, by teaching the models to hide their deception better.<\/p>\n<\/div>\n