Multimodal large language models (MLLMs) have made significant strides in visual understanding, but most rely on autoregressive generation, which limits efficiency when captioning multiple regions in an image. In a new paper, researchers introduce PerceptionDLM, a multimodal diffusion language model designed for efficient parallel region perception. The work, posted on arXiv, demonstrates that diffusion-based architectures can deliver substantial speed improvements for multi-region perception tasks while maintaining competitive accuracy.
The core innovation of PerceptionDLM is its ability to fully leverage the parallel decoding nature of diffusion language models (DLMs). Built upon PerceptionDLM-Base, a foundational baseline that already achieved state-of-the-art performance among open-source diffusion MLLMs, the new architecture introduces two key mechanisms:
- Efficient prompting to condition the model on multiple regions simultaneously.
- Structured attention masking that allows the model to generate region descriptions in parallel at both the sequence and token levels.
This approach contrasts sharply with existing methods that process regions one after another, delivering a marked improvement in inference efficiency. The authors state that to their knowledge, this is the first work to achieve parallel region caption and perception using diffusion language models.
To systematically evaluate the parallelism property of diffusion-based visual perception, the researchers constructed a new benchmark: the Parallel Detailed Localized Captioning Benchmark (ParaDLC-Bench). This benchmark extends the existing DLC-Bench by including multiple region masks per image, enabling joint evaluation of both caption quality and inference speed.
| Feature | PerceptionDLM | Typical Autoregressive MLLM |
|---|---|---|
| Generation method | Diffusion (parallel decoding) | Autoregressive (sequential) |
| Region processing | Simultaneous (multiple regions at once) | Sequential (one region at a time) |
| Inference efficiency | Higher (reduced latency) | Lower (linear with region count) |
| Performance on region captioning | Competitive | Baseline |
Experiments reported in the paper show that PerceptionDLM maintains competitive performance in region captioning while achieving significant speed improvements for multi-region tasks. The results highlight the potential of multimodal diffusion language models for efficient, parallel visual perception.
The authors have released the code, models, and datasets to facilitate further research. The project is linked from the arXiv page under the Computer Vision and Pattern Recognition category. This work points to a promising direction for scaling visual perception tasks that require analyzing many regions of an image simultaneously—an area with potential applications in document understanding, medical imaging, and automated inspection.
For enterprise technology leaders, the ability to accelerate multi-region perception without sacrificing accuracy could reduce computational costs and improve throughput in vision-based systems. The open-source nature of the models also lowers barriers to adoption.