Agentic AI systems—which rely on language-model components to interpret instructions, process external data, invoke tools, and coordinate with other agents—face a growing threat from automated prompt-injection and jailbreak attacks. Attackers increasingly use model-guided automation to scale probing, prompt refinement, and response evaluation. A new analysis posted on arXiv and authored by Soosahabi and Namsani examines the attack-defense dynamics through a probabilistic model of the target system, its defense mechanism, and the attacker's automated judge.
Limitations of Detect-and-Block Defenses
The researchers model a conventional detect-and-block approach, where malicious interactions are detected and denied with a refusal response. Their analysis shows that predictable refusals provide useful feedback to automated search, allowing the attacker success rate (ASR) to approach 1 as the query budget grows. This means that with enough attempts, a determined attacker can almost certainly compromise a system protected only by detect-and-block.
The Detect-and-Misdirect Alternative
To address this vulnerability, the paper introduces a detect-and-misdirect strategy. Instead of returning a blunt refusal, the system generates controlled, non-operational responses designed to induce false-positive errors in the attacker's automated judge. This reduces the positive predictive value of attacker-selected candidates and yields a bounded asymptotic ASR—meaning the attacker's success rate cannot approach 1 no matter how many queries are made.
| Defense Strategy | Attacker Success Rate Trend | Key Weakness Addressed |
|---|---|---|
| Detect-and-Block | ASR → 1 as budget grows | Predictable refusals guide automated search |
| Detect-and-Misdirect | Bounded asymptotic ASR | Induces false positives in attacker's judge |
Contextual Misdirection via Progressive Engagement (CMPE)
The paper presents a proof-of-concept realization of the misdirection strategy called Contextual Misdirection via Progressive Engagement (CMPE). CMPE is a lightweight conversational misdirection method that replaces predictable refusal text with safe but strategically misleading responses in automated jailbreak settings. On jailbreak benchmarks, CMPE reduces estimated ASR upper bounds by up to two orders of magnitude and nearly eliminates verified attack success in end-to-end PAIR and GPTFuzz attack runs.
| Attack Method | ASR Upper Bound (Without CMPE) | ASR Upper Bound (With CMPE) | Reduction |
|---|---|---|---|
| PAIR (end-to-end) | Not specified in source | Nearly eliminated | ~100× reduction |
| GPTFuzz (end-to-end) | Not specified in source | Nearly eliminated | ~100× reduction |
Implications for Enterprise AI Security
The findings carry significant implications for CTOs and chief digital officers deploying agentic AI systems in mission-critical operations. As these systems take on more autonomous roles—interpreting instructions, invoking tools, and coordinating actions—their vulnerability to automated attacks becomes a top-tier risk. The research demonstrates that a shift from blocking to misdirecting can fundamentally change the economics of automated attacks, making large-scale exploitation unfeasible. While CMPE is a proof-of-concept, the detect-and-misdirect framework offers a new direction for building resilient AI defenses that do not rely on predictable refusal patterns.
