Coding agents rebuilt parts of biological AI tools for $4–$20, AISI finds

Keeping a biological AI tool’s code private may not keep its capabilities private. In an October 26 research account, the UK AI Security Institute and SecureBio described coding agents rebuilding selected functions of two tools from limited descriptions and supplied training data. Successful attempts typically cost $4–$20, but the agents failed on the most complex target.

The evaluation, called ReproBAIT, asks whether agents can design, implement and train biological AI tools when given restricted information about the originals. The research paper, submitted October 14, describes three targets: two open-source protein-design tools, ProteinMPNN and RFdiffusion3, and a closed-source molecular-design tool called CMD.

Each attempt ran without internet access in an isolated environment with one GPU. Agents received the original tool’s training data, a description and fixed time and computing budgets. Time limits varied with complexity. Researchers compared abstract-only and full-paper conditions; for the open-source tools, they also tested agents given the original code.

A separate evaluator scored the rebuilt tools on data withheld during training. The researchers tested Claude Opus 5, GPT-5.6 Sol, Kimi K3 and Qwen3.8-2.4T, running each configuration three times and reporting its best attempt. Those scores therefore describe the strongest observed runs, rather than an average across attempts.

The study measured one capability per tool, not complete replacements. Its “reproduction fidelity” score expresses the rebuilt tool’s result as a percentage of the original’s result on the same measure. A score of 100% means parity on that measure—not that every function has been recreated.

• CMD: All four agents closely matched or exceeded the original’s rate of generating chemically valid molecules. But that measure was already near its ceiling. Agents optimized for it at a modest cost to the novelty of their molecules, unlike the original developers’ broader objectives.
• ProteinMPNN: Claude Opus 5 reached 96.5% fidelity with the full paper, though results varied across agents. The test measured how often designed protein sequences matched the amino acids found in nature at each position of a known protein structure.
• RFdiffusion3: No agent successfully reproduced even the evaluated function of generating protein backbones within its time budget. A partially successful attempt filtered training data toward easier structures. Other runs were excluded for producing near-duplicates of known Protein Data Bank entries.

The targets are small in training-compute terms compared with the largest released biological AI tools. AISI says larger rebuilds would cost considerably more and expects more time and compute to improve performance. Those possibilities were not established by the successful cheaper runs.

Giving agents a full paper rather than an abstract did not consistently improve results. One possible explanation is memorisation: the open-source tools’ papers and code were likely present in the agents’ pretraining data, allowing them to draw on implementation details that were not supplied during the test.

Researchers checked agent transcripts for recalled details and compared generated code with the originals. They found moderate evidence consistent with memorisation for ProteinMPNN and little for RFdiffusion3. These checks are imperfect: they neither prove memorisation nor establish how much success it explains.

For developers trying to limit misuse while supporting legitimate research, AISI’s conclusion is that withholding code alone does not assure control over access to a tool’s capabilities. It identifies control of underlying biological data as a more promising safeguard to investigate. Data access was supplied in this evaluation; its effectiveness as a restriction remains a question for further research.