A new study published on arXiv examines how industry developers perceive, prioritize, and address risks in agentic AI products. The research, conducted with 35 developers, finds a significant gap between the risks they focus on and the tools available to manage them.
Study Methodology and Findings
The study defines agentic AI systems as those that act autonomously, use tools, adapt to context, and operate in complex real-world environments. According to the researchers, these very characteristics create or exacerbate product risks.
Developers' perceptions of risk were closely tied to the qualities that made their products agentic: autonomy, tool use, and real-world context. However, their prioritization of risks skewed heavily toward product and business concerns.
Risk Prioritization: Business Before Society
| Risk Category | Examples | Priority Level |
|---|---|---|
| Product/Business | System failures, cost overruns, competitive disadvantage | High |
| Downstream Societal | Job displacement, end-user privacy erosion, ethical harms | Low |
This prioritization, in turn, impacted developers' ability and motivation to address agentic risks. The study found that developers lacked mature controls for containing these risks, often resorting to constraining the same characteristics that make agents useful, such as autonomy and goal complexity.
The Capability vs. Risk Control Tension
The researchers summarize this as a 'capability vs. risk control tension' in agentic AI development: developers need to address risks that emerge from agentic capabilities, yet they currently have limited support for doing so without constraining agentic functionality.
"Developers lacked mature controls for containing agentic risks, often relying on constraining the same characteristics that make agents useful: e.g., autonomy and goal complexity."
This finding has direct implications for enterprise technology leaders deploying agentic AI in supply chain, logistics, or other critical systems. The study suggests that without better risk management frameworks, businesses may inadvertently expose themselves to both operational and reputational harm.
Implications for Enterprise Leaders
For CTOs and digital transformation leaders, the study underscores the need for dedicated risk governance for agentic AI. Current development practices appear insufficient, as teams struggle to balance innovation with safety. The researchers call for more mature controls that do not sacrifice the very capabilities that deliver business value.
The study also highlights a blind spot: societal risks like end-user privacy and job displacement are being deprioritized. While this may be rational in a competitive environment, it could lead to regulatory backlash or public distrust.
Further research and tooling are needed to help developers systematically identify, assess, and mitigate agentic risks without crippling product functionality. Until then, enterprise adopters must proceed with caution, balancing the promise of autonomous AI with the perils of uncontrolled agency.
