Artificial Intelligence #cascaded sparse autoencoders#multimodal llms
Cascaded Sparse Autoencoders Enable Hierarchical Visual Concept Learning in Multimodal LLMs
Researchers introduce cascaded sparse autoencoders (CSAEs) that learn hierarchical visual concepts in multimodal large language models. By training a second-level SAE on the decoder weights of the first, CSAEs achieve 'concepts of concepts' without nesting or stacking bottlenecks. Experiments on Qwen3-VL, Gemma-3, and LLaVA show improved interpretability and effective group-level steering.
Jun 16, 2026 1 source