Visual state space models (SSMs), such as Mamba, have demonstrated strong efficiency in modeling long visual sequences. However, when token reduction is applied to structurally enhanced Mamba variants, these models exhibit severe performance collapse. According to a paper published on arXiv, researchers attribute this degradation to the spatially agnostic nature of existing reduction methods, which violate the two-dimensional structural premise required by the selective scanning mechanism.
The Problem: Performance Collapse Under Token Reduction
Token reduction is crucial for scaling vision models to handle high-resolution images or long video sequences. In Mamba-based architectures, standard reduction techniques treat tokens as independent, ignoring the spatial grid topology. This leads to a breakdown in the model's ability to capture local and global context, causing a sharp drop in accuracy. The paper notes that this collapse is especially pronounced in structurally enhanced Mamba variants like VMamba and PlainMamba.
STORM: A Spatial-Aware Solution
To address this, the team proposed STORM (Spatial-Aware Reduction Framework), a token reduction framework designed to maintain structural integrity throughout the compression process. STORM reformulates reduction into a structured operation on spatial units, enforcing localized constraints to preserve both grid topology and neighborhood coherence. As a plug-and-play module, STORM can be integrated into existing reduction pipelines without any training.
Key features of STORM include:
- Explicit spatial awareness: Reduction is guided by the 2D structure of the input, not just feature similarity.
- Training-free integration: No fine-tuning or retraining is required; STORM works with pre-trained models.
- Compatibility: Works across diverse vision Mamba backbones, including VMamba and PlainMamba.
Performance Benchmarks
The researchers validated STORM on multiple architectures under training-free settings. The results are striking, as shown in the table below:
| Model | Method | Top-1 Accuracy | Recovery vs. Prior Methods |
|---|---|---|---|
| VMamba | Standard reduction | Baseline (low) | - |
| VMamba | STORM | Up to 63.3% recovery | 63.3% higher than prior best |
| PlainMamba | Standard reduction | Baseline | - |
| PlainMamba | STORM | 1.0% drop from full model | Comparable to ViT |
The paper reports that STORM achieves state-of-the-art pruning accuracy across diverse vision Mamba backbones under training-free settings. The most notable result: on VMamba, STORM delivers a substantial accuracy recovery, outperforming prior methods by up to 63.3% in top-1 accuracy. On PlainMamba, STORM incurs only a 1.0% accuracy drop, achieving performance comparable to Vision Transformers (ViT).
"STORM delivers a substantial accuracy recovery on VMamba, outperforming prior methods by up to 63.3% in top-1 accuracy."
Implications for Enterprise AI
While the research is purely academic, it has implications for enterprises deploying vision-based AI in production. Mamba-based models are attractive for their efficiency in long-sequence tasks, such as video analytics, autonomous inspection, or document processing. However, the need for token reduction to meet latency constraints has been a barrier due to accuracy loss. STORM provides a training-free method to retain accuracy, potentially enabling faster inference without costly model retraining. For technology leaders evaluating vision models, Mamba variants combined with STORM offer a promising path to balance efficiency and fidelity.
The paper is authored by Lv, Jindi; Li, Aoyu; Zhou, Yuhao; Zhu, Zheng; Wang, Xiaofeng; Ye, Qing; Duan, Yueqi; Feng, Wentao; and Jiancheng. It is available on arXiv under a Creative Commons license.