Artificial intelligence agents promise to accelerate drug discovery by compressing interpretation and decision-making loops, but practical deployment requires trusted evaluation on realistic program decisions. A new benchmark, TxBench-PP (TherapeuticsBench Preclinical Pharmacology), aims to provide that evaluation by testing whether agents can recover accurate conclusions from real-world assay data rather than memorized facts from literature.
Benchmark Design and Scope
According to the paper by Le, Hannah, Ramasamy, Ramesh, Urrutia, Alex, Yazdani, Mahsa, Proctor, Tim, Workman, and Kenny, posted on arXiv, TxBench-PP is the first focused slice of a broader TherapeuticsBench effort that will span drug-discovery stages and therapeutic modalities. The benchmark contains 100 evaluations indexed by program stage, assay type, and task structure. It covers five categories of reasoning:
- Mechanism-of-action (MoA) and pharmacodynamic (PD) reasoning
- Compound-target engagement
- Causal target validation
- Developability and safety
- Translational efficacy
Agents receive realistic workflow snapshots, inspect files in a coding environment, and return structured answers that are graded deterministically. This design ensures that success depends on analytical capability rather than memorization.
Performance Results Across Models
The researchers tested 16 model-harness configurations, comprising 11 models and 4,800 trajectories. No system reliably recovered preclinical pharmacology decisions. The strongest configuration, Claude Opus 4.8 / Pi, passed 59.3% of endpoint attempts (178/300; 95% CI, 51.1-67.6). The second-best was GPT-5.5 / Pi at 55.3% (166/300; 47.0-63.6).
| Model Configuration | Pass Rate | Endpoint Attempts | 95% Confidence Interval |
|---|---|---|---|
| Claude Opus 4.8 / Pi | 59.3% | 178/300 | 51.1-67.6 |
| GPT-5.5 / Pi | 55.3% | 166/300 | 47.0-63.6 |
All other configurations performed lower, though specific scores were not detailed in the paper. The results indicate that even the most advanced models fall short of reliable performance in this domain.
Implications for AI in Drug Discovery
The gap between current AI capabilities and the demands of preclinical pharmacology is significant. The benchmark's structure — requiring agents to work with realistic assay data and make reasoning chains — exposes weaknesses in models that excel at language tasks but struggle with data-intensive analytical workflows. For enterprise technology decision-makers, TxBench-PP demonstrates that deploying AI agents in high-stakes scientific environments requires rigorous, domain-specific validation. The benchmark provides a template for evaluating AI systems before trusting them with real drug development decisions. While the current models show promise, none meet the reliability threshold needed for unsupervised use in preclinical pharmacology. Future work on the broader TherapeuticsBench will likely continue to push models toward better performance.
