Enterprise AI agents are increasingly expected to handle complex, data-heavy workflows, but current benchmarks often test code and data capabilities in isolation. A new preprint from researchers including Zhang, Yuxin, Fan, Ju, Meihao, Shaolei, Du, and Xiaoyong introduces CODA-BENCH, the first benchmark designed to jointly evaluate code intelligence and data intelligence in a realistic, data-intensive environment. The findings, posted on arXiv, reveal that even the most advanced code agents struggle to integrate data discovery with code generation, achieving only a 61.1% success rate.
Benchmark Design and Scope
CODA-BENCH is built around a Linux sandbox that mirrors the Kaggle ecosystem, containing hundreds of datasets. Agents must actively explore complex file hierarchies to identify relevant resources and then generate code for data-driven analytical tasks. The benchmark comprises 1,009 tasks spanning 31 communities, with each task environment containing an average of 980 files—simulating the scale and noise of real-world data environments.
The researchers explicitly designed CODA-BENCH to bridge the gap between existing benchmarks, which typically evaluate code-centric or data-centric abilities separately, and real development scenarios where both are required simultaneously.
Findings: A Significant Capability Gap
Evaluations of several advanced code agents on CODA-BENCH showed that even top-performing systems struggle effectively to combine data discovery with code execution. The highest success rate recorded was 61.1%, indicating a substantial shortfall in current agentic capabilities for data-intensive tasks. The results suggest that agents tend to perform well on either code generation or data retrieval, but fail to coordinate the two seamlessly.
| Benchmark Metric | Value |
|---|---|
| Total tasks | 1,009 |
| Communities represented | 31 |
| Average files per task | 980 |
| Top agent success rate | 61.1% |
The paper notes that these results "highlight a substantial gap in current agentic capabilities for data-intensive tasks and point to promising directions for future research."
Why This Matters for Enterprise AI
For enterprise technology decision-makers, CODA-BENCH serves as a realistic stress test for AI agents that might be deployed to handle data-heavy operations such as supply chain analytics, trade documentation processing, or financial data reconciliation. The benchmark's reliance on large file systems and heterogeneous data sources mirrors the complexity of corporate databases and data lakes. The 61.1% success ceiling suggests that current agent technology is not yet reliable enough for unsupervised deployment in data-intensive enterprise workflows without human oversight. Vendors developing code-generation tools and AI agents should prioritize integrating robust data-discovery capabilities alongside coding skills.
The researchers have made the benchmark available through arXiv, and the underlying code and data are expected to be released to the community, enabling further research and improvement of agent architectures. CODA-BENCH provides a clear metric for progress: until agents consistently score above 90% in such realistic settings, enterprises should approach autonomous data-task automation with caution.