The company has produced an extensive collection of case studies covering how its current AI models have been misused.
Anthropic reported and arrested several scientists who were using its AI models to create potential bioweapons, according to a new report shared by the company on Thursday. “Biological misuse” is just one example of the troubling use of AI described in the report, but it is of growing importance as newer models like Fable 5 become increasingly effective in aiding scientific research.
The company’s report includes five case studies on Anthropic’s models used to develop biological weapons, the security measures that brought this misuse to Anthropic’s attention, and how it responded. Detecting possible abuse is complicated, as the company pointed out. The New York Timesbecause research into a new vaccine could be a lot like creating a bioweapon. Either way, the company says it made a mistake by being too cautious because of the possible consequences.
“You don’t see someone in a comic book saying, ‘Hey, I want to build a bioweapon to kill everyone,'” said Jacob Klein, head of threat intelligence at Anthropic. The New York Times. “It’s an incredibly nuanced situation.”
In a case study from May, Anthropic says its “biosafety classifier” flagged a request for Claude to write a grant for gain-of-function research on the chikungunya virus. The virus has no approved treatment and can cause debilitating symptoms for weeks or months, the company says. Gain-of-function research explores methods to genetically modify an organism, and in the case of this specific virus, the grant proposes to increase its transmissibility and ability to evade the immune response. The combination seems dangerous, and Anthropic says the fact that the proposed research was affiliated with a military research institute made it even more questionable.
The company raised a similar issue with gain-of-function research in avian flu and a separate case of a researcher using Claude to develop a peptide atlas of venom toxins and “a generative pipeline that optimized toxin characteristics.” The full report also includes case studies of Anthropic models used as a surveillance tool and to create software exploits, propaganda and weapons systems.
According to the company, anyone misusing the Claude or Anthropic templates has had their account banned, and the results of the investigation are being used to further improve the templates’ warranties and prevent future misuse. Given the sensitive nature of biologics misuse, Anthropic also does not name individuals or institutions whose investigations are linked to the development of possible bioweapons. “The people involved in these case studies are working scientists,” Anthropic explains. “We are not asserting that they intended harm, and identifying them or their laboratories could expose them to harm.”
