\n<\/p>\n
Meta CEO Mark Zuckerberg has pledged to make artificial general intelligence (AGI) \u2014 which is roughly defined as AI that can accomplish any task a human can \u2014 openly available one day. But in a new policy document<\/a>, Meta suggests that there are certain scenarios in which it may not release a highly capable AI system it developed internally.<\/p>\n The document, which Meta is calling its Frontier AI Framework, identifies two types of AI systems the company considers too risky to release: \u201chigh risk\u201d and \u201ccritical risk\u201d systems. <\/p>\n As Meta defines them, both \u201chigh-risk\u201d and \u201ccritical-risk\u201d systems are capable of aiding in cybersecurity, chemical, and biological attacks, the difference being that \u201ccritical-risk\u201d systems could result in a \u201ccatastrophic outcome [that] cannot be mitigated in [a] proposed deployment context.\u201d High-risk systems, by contrast, might make an attack easier to carry out but not as reliably or dependably as a critical risk system.<\/p>\n Which sort of attacks are we talking about here? Meta gives a few examples, like the \u201cautomated end-to-end compromise of a best-practice-protected corporate-scale environment\u201d and the \u201cproliferation of high-impact biological weapons.\u201d The list of possible catastrophes in Meta\u2019s document is far from exhaustive, the company acknowledges, but includes those that Meta believes to be \u201cthe most urgent\u201d and plausible to arise as a direct result of releasing a powerful AI system.<\/p>\n Somewhat surprising is that, according to the document, Meta classifies system risk not based on any one empirical test but informed by the input of internal and external researchers who are subject to review by \u201csenior-level decision-makers.\u201d Why? Meta says that it doesn\u2019t believe the science of evaluation is \u201csufficiently robust as to provide definitive quantitative metrics\u201d for deciding a system\u2019s riskiness.<\/p>\n If Meta determines a system is high-risk, the company says it will limit access to the system internally and won\u2019t release it until it implements mitigations to \u201creduce risk to moderate levels.\u201d If, on the other hand, a system is deemed critical-risk, Meta says it will implement unspecified security protections to prevent the system from being exfiltrated and stop development until the system can be made less dangerous. <\/p>\n Meta\u2019s Frontier AI Framework, which the company says will evolve with the changing AI landscape, and which Meta earlier committed to publishing<\/a> ahead of the France AI Action Summit this month, appears to be a response to criticism of the company\u2019s \u201copen\u201d approach to system development. Meta has embraced a strategy of making its AI technology openly available \u2014 albeit not open source by the commonly understood definition<\/a> \u2014 in contrast to companies like OpenAI that opt to gate their systems behind an API.<\/p>\n For Meta, the open release approach has proven to be a blessing and a curse. The company\u2019s family of AI models, called Llama<\/a>, has racked up hundreds of millions of downloads. But Llama has also reportedly<\/a> been used by at least one U.S. adversary to develop a defense chatbot.<\/p>\n In publishing its Frontier AI Framework, Meta may also be aiming to contrast its open AI strategy with Chinese AI firm DeepSeek\u2019s. DeepSeek<\/a> also makes its systems openly available. But the company\u2019s AI has few safeguards and can be easily steered to generate toxic and harmful outputs<\/a>.<\/p>\n \u201c[W]e believe that by considering both benefits and risks in making decisions about how to develop and deploy advanced AI,\u201d Meta writes in the document, \u201cit is possible to deliver that technology to society in a way that preserves the benefits of that technology to society while also maintaining an appropriate level of risk.\u201d<\/p>\n TechCrunch has an AI-focused newsletter!\u00a0Sign up here<\/a>\u00a0to get it in your inbox every Wednesday.<\/em><\/p>\n<\/div>\n