\n<\/p>\n
Anthropic is launching a program<\/a> to fund the development of new types of benchmarks capable of evaluating the performance and impact of AI models, including generative models like its own Claude<\/a>.<\/p>\n Unveiled on Monday, Anthropic\u2019s program will dole out grants to third-party organizations that can, as the company puts it in a blog post, \u201ceffectively measure advanced capabilities in AI models.\u201d Those interested can submit applications to be evaluated on a rolling basis. <\/p>\n \u201cOur investment in these evaluations is intended to elevate the entire field of AI safety, providing valuable tools that benefit the whole ecosystem,\u201d Anthropic wrote on its official blog. \u201cDeveloping high-quality, safety-relevant evaluations remains challenging, and the demand is outpacing the supply.\u201d<\/p>\n As we\u2019ve highlighted<\/a> before, AI has a benchmarking problem. The most commonly cited benchmarks for AI today do a poor job of capturing how the average person actually uses the systems being tested. There are also questions as to whether some benchmarks, particularly those released before the dawn of modern generative AI, even measure what they purport to measure, given their age.<\/p>\n The very-high-level, harder-than-it-sounds solution Anthropic is proposing is creating challenging benchmarks with a focus on AI security and societal implications via new tools, infrastructure and methods.<\/p>\n The company calls specifically for tests that assess a model\u2019s ability to accomplish tasks like carrying out cyberattacks, \u201cenhance\u201d weapons of mass destruction (e.g. nuclear weapons) and manipulate or deceive people (e.g. through deepfakes or misinformation). For AI risks pertaining to national security and defense, Anthropic says it\u2019s committed to developing an \u201cearly warning system\u201d of sorts for identifying and assessing risks, although it doesn\u2019t reveal in the blog post what such a system might entail.<\/p>\n Anthropic also says it intends its new program to support research into benchmarks and \u201cend-to-end\u201d tasks that probe AI\u2019s potential for aiding in scientific study, conversing in multiple languages and mitigating ingrained biases, as well as self-censoring toxicity.<\/p>\n To achieve all this, Anthropic envisions new platforms that allow subject-matter experts to develop their own evaluations and large-scale trials of models involving \u201cthousands\u201d of users. The company says it\u2019s hired a full-time coordinator for the program and that it might purchase or expand projects it believes have the potential to scale. <\/p>\n \u201cWe offer a range of funding options tailored to the needs and stage of each project,\u201d Anthropic writes in the post, though an Anthropic spokesperson declined to provide any further details about those options. \u201cTeams will have the opportunity to interact directly with Anthropic\u2019s domain experts from the frontier red team, fine-tuning, trust and safety and other relevant teams.\u201d<\/p>\n Anthropic\u2019s effort to support new AI benchmarks is a laudable one \u2014 assuming, of course, there\u2019s sufficient cash and manpower behind it. But given the company\u2019s commercial ambitions in the AI race, it might be a tough one to completely trust. <\/p>\n In the blog post, Anthropic is rather transparent about the fact that it wants certain evaluations it funds to align with the AI safety classifications it <\/em>developed<\/a> (with some input from third parties like the nonprofit AI research org METR). That\u2019s well within the company\u2019s prerogative. But it may also force applicants to the program into accepting definitions of \u201csafe\u201d or \u201crisky\u201d AI that they might not agree completely agree with.<\/p>\n A portion of the AI community is also likely to take issue with Anthropic\u2019s references to \u201ccatastrophic\u201d and \u201cdeceptive\u201d AI risks, like nuclear weapons risks. Many experts<\/a> say there\u2019s little evidence to suggest AI as we know it will gain world-ending, human-outsmarting capabilities anytime soon, if ever. Claims of imminent \u201csuperintelligence\u201d serve only to draw attention away from the pressing AI regulatory issues of the day, like AI\u2019s hallucinatory<\/a> tendencies, these experts add.<\/p>\n In its post, Anthropic writes that it hopes its program will serve as \u201ca catalyst for progress towards a future where comprehensive AI evaluation is an industry standard.\u201d That\u2019s a mission the many open<\/a>, corporate-unaffiliated<\/a> efforts to create better AI benchmarks can identify with. But it remains to be seen whether those efforts are willing to join forces with an AI vendor whose loyalty ultimately lies with shareholders. <\/p>\n