KEYTAKEAWAYS
- Anthropic launches a program to fund third-party AI benchmark development.
- Focus on creating benchmarks for advanced AI capabilities and societal impact.
- Applications evaluated on a rolling basis; tailored funding options available.
CONTENT
Anthropic Initiates Funding Program to Elevate AI Safety through Comprehensive Benchmarking
In a move aimed at enhancing AI safety and evaluation, Anthropic has unveiled a new program dedicated to funding the development of comprehensive AI benchmarks. Announced on Monday, the initiative seeks to address the growing need for robust AI performance evaluations, particularly for generative models like Anthropic’s own Claude.
Anthropic’s program will provide financial support to third-party organizations capable of creating effective measures for advanced AI capabilities. Applications for funding will be accepted on a rolling basis, ensuring a continuous influx of innovative benchmark proposals.
“Our investment in these evaluations is intended to elevate the entire field of AI safety, providing valuable tools that benefit the whole ecosystem,” Anthropic stated on its official blog. The company acknowledges the challenges in developing high-quality, safety-relevant evaluations and aims to bridge the gap between demand and supply in the AI safety domain.
Existing AI benchmarks often fail to capture real-world usage accurately, raising questions about their relevance, especially those predating modern generative AI. Anthropic’s solution involves creating challenging benchmarks with a focus on AI security and societal implications through new tools, infrastructure, and methods.
Anthropic calls for tests assessing a model’s ability to perform tasks such as cyberattacks, enhancing weapons of mass destruction, and manipulating or deceiving individuals through deepfakes or misinformation. For national security and defense, the company aims to develop an “early warning system” to identify and assess AI risks, though details of this system remain unspecified.
In addition to security-related benchmarks, Anthropic’s program will support research into AI’s potential for scientific study, multilingual communication, bias mitigation, and self-censorship of toxicity. New platforms enabling subject-matter experts to develop evaluations and large-scale model trials involving thousands of users are also envisioned.
The company has hired a full-time coordinator for the program and may purchase or expand projects with scalable potential. While specific funding options were not disclosed, Anthropic promises tailored support to meet the needs of each project.
Despite Anthropic’s commercial interests in the AI race, its effort to support new benchmarks is commendable. The company acknowledges that certain evaluations will align with its AI safety classifications, developed with input from third parties like the nonprofit AI research organization METR.
However, some in the AI community may question Anthropic’s focus on catastrophic and deceptive AI risks, such as nuclear weapons, given the lack of evidence for imminent superintelligent AI threats. Critics argue that such claims divert attention from pressing regulatory issues like AI’s hallucinations.
Anthropic hopes its program will catalyze progress towards making comprehensive AI evaluation an industry standard, aligning with the mission of various open, corporate-unaffiliated efforts to improve AI benchmarks. It remains to be seen whether these efforts will collaborate with an AI vendor driven by shareholder interests.
Buy Bitcoin at Binance
Enjoy up to 20% off on trading fees! Sign up Now!
