A new study from Zhengheng, Panrui, Liu, Xuyang, Xia, and Puzhi, published on arXiv, reveals that where a query is placed in in-context learning (ICL) examples for diffusion large language models (dLLMs) significantly impacts generation quality. Unlike autoregressive (AR) models, dLLMs use bidirectional attention and offer spatial flexibility, but current practices inherit AR-style trailing-query templates that overlook this structural difference.
The Problem: Positional Bias in Diffusion LLMs
While ICL is well-studied in AR LLMs, its behavior in dLLMs remains largely unexplored, according to the paper. The authors systematically decouple positional variance from example semantics and find that query position is a first-order variable — its impact on generation quality is on par with the quality of the examples themselves. Internally, this sensitivity stems from a spatial "Recency Effect" in attention flow and task-dependent shifts in decoding trajectories.
"This paper presents a comprehensive analysis unveiling that query position is actually a first-order variable in dLLMs."
Benchmarking Positional Sensitivity
The researchers establish foundational spatial ICL baselines across reasoning and perception tasks. Key findings include:
| Finding | Description |
|---|---|
| Positional variance impact | Affects generation quality as much as example semantic quality |
| Cause | Spatial "Recency Effect" in attention flow and task-dependent decoding trajectory shifts |
| Traditional confidence metric | Single-step confidence ($C_{decoded}$) fails in dLLMs |
| Proposed metric | Average Confidence ($\overline{C}$) tracks iterative decoding process |
Mitigation Strategy: Auto-ICL and Average Confidence
To address the instability without requiring ground-truth labels, the authors propose Average Confidence ($\overline{C}$), a novel metric that monitors the iterative decoding process. Unlike traditional single-step confidence, which they show fails in dLLMs, $\overline{C}$ provides a more reliable signal.
Building on this, they introduce Auto-ICL, a training-free adaptive routing strategy that dynamically optimizes query placement. According to the paper, Auto-ICL robustly approaches oracle performance across heterogeneous reasoning and perception tasks, effectively mitigating positional bias.
Implications for Enterprise AI
For CTOs and technology leaders evaluating next-generation LLMs for enterprise applications — including document generation, automated reasoning, and data analysis — this research highlights the importance of understanding model architecture-specific behaviors. Diffusion LLMs offer promising bidirectional attention capabilities, but their deployment requires new inference-time strategies like Auto-ICL to ensure consistent output quality. The findings underscore that simply porting AR-model prompting templates to dLLMs may lead to significant performance degradation, and adaptive query placement is critical for production systems.
