Enterprises increasingly deploy hybrid AI pipelines that combine large language models (LLMs) with formal solvers for safety-critical decision-making, such as supply chain compliance or contract verification. But new research from Zunchen Huang and Songgaojun Deng reveals a crucial weakness in the final step of these pipelines: the narration gap, where an LLM's summarization of a solver's output can be manipulated even when the solver itself is sound.
The Problem: Narration Gap in Hybrid AI Pipelines
Formal tools like SAT and SMT solvers are being embedded in LLM reasoning pipelines when a safety or security critical question can be expressed in logic. Unlike chain-of-thought reasoning, a solver produces a sound, independently verifiable answer. However, Huang and Deng model the hybrid pipeline as three components: formalizing the question, deciding it, and narrating the result. Prior work focused on formalization and decision, but not narration — the step that turns a formal tool's output into the answer a user reads. This gap is the focus of their study.
How the Vulnerability Works
The researchers first model the LLM-solver loop as a verified decision procedure. They then evaluate five open-source models under prompt injection. Their key finding: certificate gating makes the solver's verdict sound, meaning the formal result cannot be tampered with. However, an adversary can invert a verified conclusion across phrasings and channels. This means the LLM's narrative output can contradict the verified solver result, potentially leading users to act on wrong information.
| Protection Mechanism | Attack Type | Effect on Conclusion |
|---|---|---|
| Certificate gating | Prompt injection | Solver verdict remains sound; adversary inverts the narrated conclusion |
| Hardened prompt | Prompt injection | Reduces injection significantly but does not eliminate it; still vulnerable under adaptive attack |
Mitigations and Limitations
Huang and Deng study mitigation through a hardened prompt that reduces prompt injection significantly but cannot eliminate it. Even with this defense, the system still suffers under adaptive attack. The paper states: "Combining the formal analysis and empirical studies, we show in the LLM-solver loop, robustness does not reach to the answer that the user finally reads." This highlights that no currently known prompt engineering method fully closes the narration gap.
Implications for Enterprise AI
For enterprise technology leaders deploying AI in compliance, trade finance, or logistics optimization, this research underscores a critical risk: a formally verified decision can be silently inverted in the final user-facing answer. If an LLM narrates that a shipment is compliant when the solver determined it is not, the business consequences could be severe. The findings suggest that enterprises must implement multi-layer verification beyond the model's output, such as auditing the narration step with independent checks or using dedicated smaller models for summarization. Until the narration gap is closed, human-in-the-loop validation remains essential for safety-critical decisions.
