Updated at 8:30 p.m. ET.
What were the five cases?
Today, Anthropic published a report describing five cases in which scientists used its Claude chatbot for research that could have supported the development of biological weapons. Anthropic caught these, but it also warned: its own safeguards are the only thing standing in the way.
The five cases read like a modern biodefense syllabus:
A group tied to a military research institute asked Claude to help write a grant application for gain-of-function research, engineering chikungunya to spread more easily or cause more harm. Chikungunya is a virus that causes fever, joint pain, headache, muscle pain, joint swelling, and rash. The joint pain can be severe and last months. Chikungunya outbreaks occur in tropical and subtropical parts of the world.
A scientist in a country where Claude is blocked spent weeks trying to turn H5N1 bird flu into a strain better suited to spreading between people. Note that Claude use is blocked in countries like China, Russia, Iran, and North Korea.
Another researcher used Claude to write a funding application to genetically alter orthopoxviruses, the family that includes smallpox and mpox, so they could better dodge immune defenses.
A scientist with state funding built an atlas of toxin peptides drawn from multiple venomous animal species and a pipeline to optimize them, framed as research toward new painkillers and antidepressants. GLP-1 drugs are derived from Gila monster venom. But some of the same structures are paralytic toxins with potential use as incapacitating agents.
A researcher redesigned lethal toxins, including bacterial toxins and proteins from hemorrhagic-fever viruses, while requesting deliberately vague identification. This is what your current “optimize lethal toxins” bullet actually describes.
The same information that helps build a weapon can also produce a vaccine or a cure. According to Gregory Koblentz, a biodefense scholar at George Mason University:
"Biological research is not just dual-use like Anthropic said, it is multi-use and can be used for peaceful civilian scientific and commercial purposes, for defensive purposes, and for offense. All of the research discussed by Anthropic was on pathogens that are endemic in different parts of the world and are current public health threats, so studying them is not suspicious on its own. The military affiliation of an institute does not automatically mean that its research is part of an offensive [bioweapons] BW program; for example, the United States military conducts and sponsors research on a range of infectious diseases that pose a potential risk to our troops."
Dr. Tom Inglesby, who directs the Johns Hopkins Center for Health Security, credited Anthropic for catching the attempts and for publishing them:
“The bad news is that those efforts were voluntary. And the more serious bad news is that there is no US government policy requiring frontier models or biological AI models [to] be assessed for major vulnerabilities that could be exploited, or policy requiring any evaluation for potential national security level biological risks.”
No U.S. law required Anthropic to build the classifier that caught these requests. No law required Anthropic to issue this report.
The chikungunya case illustrates additional gaps. When Claude refused to answer, the group routed the request to a different AI model with weaker safeguards. Anthropic didn’t say which model that was or whether it answered.
In a separate 30-day sweep of activity associated with what Anthropic calls “adversarial state institutions,” the company found roughly 35 biological research efforts. Most were ordinary civilian science, Anthropic said, while some had notable multi-use potential. The company did not identify the projects or say how many fell into that category.
Did Anthropic catch a bioweapons program?
Koblentz cautioned:
“it is not possible to determine what, if any, relationship this research had to a state-run biological weapon program.”
He said that judging whether research belongs to a weapons program requires much more context. He would look at the researcher’s publication record and interests, the institution and its affiliations, funding, and other evidence that the country has an active biological weapons program.
The efforts to get around Anthropic’s access controls add another signal. Some researchers used VPNs, dummy accounts or intermediaries to reach models they weren’t supposed to access. Koblentz said that could reflect a desire to keep the work secret, which can be an indicator of a weapons program. It could also reflect something much more ordinary: scientists trying to reach the most powerful tools available for their research and careers. He cautioned that it would be a leap to treat the cases as evidence of a secret biological weapons program.
Another expert in AI and biosecurity, who asked to remain anonymous due to the political sensitivities around these issues, told me that
“these case studies are compatible with legitimate researchers circumventing geoblocks to access the same level of assistance available to their non-geoblocked US/EU counterparts.”
And a military institute’s involvement doesn’t automatically mean an offensive program, because the US military itself funds infectious disease research to protect its troops. In fact, up to $2 billion could move from the National Institutes of Health to the Pentagon under a new biodefense and pandemic-preparedness agreement the two agencies signed this month.
How does an AI answer become real biology?
The other layer of defense doesn’t depend on which chatbot a scientist happens to pick. Synthetic DNA companies ship ordered genetic sequences to labs on request. No federal law requires them to screen those orders for dangerous pathogens. A May 2025 executive order gave the White House Office of Science and Technology Policy 90 days to write nucleic acid synthesis rules.
In February, Senators Tom Cotton (R-AR) and Amy Klobuchar (D-MN) introduced the Biosecurity Modernization and Innovation Act, which would require gene synthesis providers to screen orders and customers. It hasn’t passed.
In July, HHS banned federally funded high-risk gain-of-function research. That rule doesn’t take effect until November, and it only reaches work the federal government pays for. None of it touches what a lab can simply order and receive.
The system that got tested this week belonged to Anthropic. The system Inglesby wants, a government backstop with legal teeth, still doesn’t exist. Anthropic decides what counts as too dangerous to answer. It also decides which countries get access to Claude, and which of its models are safe enough to trust with biology questions at all.
Andrew Weber, a former assistant secretary of defense for nuclear, chemical, and biological programs and now at the Council on Strategic Risks, told CBS News the frontier labs “have been very responsible.” He added that cases like these had never surfaced in public before this week, and it wasn’t the FBI or any regulator that surfaced them.
Weber also wants President Trump to press the issue when he meets President Xi Jinping, because Chinese AI companies, in his words, are “less sensitive about the risks” and aren’t putting comparable guardrails on their open-weight models, which can be downloaded by anyone to their own computer and modified for their specific needs. Asked whether a small group or a lone actor could pull off something similar, Weber said, “unfortunately chillingly realistic.”
Kevin Esvelt, an evolutionary biologist and associate professor at the MIT Media Lab, told me:
“It would seem prudent to give all legitimate researchers trusted user access to the best frontier models — to accelerate our efforts to improve the world within our individual fields — while ensuring that insecure open-weight models know as little as possible about molecular and cellular biology.”
Anthropic’s classifiers, the company says, cannot tell a virus researcher from a weapons developer.
Closing the gap Anthropic’s report describes means acting on both ends of the pipeline:
Test the most powerful AI and biology tools for serious biological risks before they ship or update, not after a company catches a problem and writes a report about it.
Build the same detection system Anthropic built — classifiers that can spot and block dangerous biological requests — into what regulators require of every AI model. Congress is now writing rules for the biggest risks AI poses generally, and biological threats belong near the top of that list, not as an afterthought.
Meanwhile, attempts to regulate gain-of-function research go back well over a decade, across both Democratic and Republican administrations. None of them has closed this particular gap.
Why hasn’t the U.S. government already restricted dangerous gain-of-function work?
This doesn’t necessarily argue for banning gain-of-function research outright. Gain-of-function can mean making a virus easier to study or helping a vaccine work better. Some experiments make a pathogen more dangerous, and those need much tighter oversight, not a ban on the whole category. Governments can restrict funding, raise biosafety standards, or prohibit specific experiments.
And the U.S. can’t ban gain-of-function research outside the U.S. That’s why the answer isn’t just regulation of science. It’s also defense: vaccines, drugs, and a health system that can respond if a bioweapon gets used anyway.
Could AI really help someone build a bioweapon?
The anonymous AI-biosecurity also told me that nobody knows for sure yet. Today’s AI models can already beat most human experts on biology tests, and can help with tricky lab problems and hard biology questions. The real question is whether AI can help people with some lab skills go all the way to building something dangerous.
Esvelt said that an AI model recently disclosed a viable way to build a new category of bioweapon he hadn't known was possible, by combining pieces of information from different fields. "They still aren't innovative," he said. "The pieces were quite obvious once you had all of them. But I'd be surprised if any human scientist did." He's not concerned by the five cases Anthropic just disclosed, he said, "save for the likely intent." None of them "registers on my own ranking of threats." But what worries him is what they represent: AI, in his words, "democratizes competence." We shouldn't assume, he told me, that a lack of knowledge or lab skill will keep someone from causing harm, or that experts can anticipate every threat AI puts within reach.
I am very, very worried.
None of this is evidence of an imminent bioweapons attack. But this is what has experts like me very, very worried: AI systems that can accelerate advanced biology + a drug and vaccine supply chain that sources raw materials overseas + eroding public trust in the institutions (e.g. CDC) that would respond to a natural or engineered biothreat + increasing vaccine skepticism. Together, this describes a country that could find out, in real time, exactly how vulnerable and unprepared it is.