Visit IGEN World Explore IGEN Expo

EXPLORE UPGRADE PLANS

BREAKING

Hidden Failure Modes in AI Reasoning: Study Reveals Oversight Paradox and Context-Injection Vulnerabilities InstantForget: New Update-Free Backdoor Unlearning Method Uses Inference-Time Feature Reset for AI Security Beyond Weights and Gradients: New Taxonomy Classifies Federated Learning Messages into Three Categories Token Reduction in Generative Models Must Evolve Beyond Efficiency, New Research Argues Semantic Flip: Synthetic OOD Generation for Robust Refusal in Embodied Question Answering and Spatial Localization Emergent Strategic Reasoning Risks in AI: New Taxonomy-Driven Framework Evaluates Deception and Gaming in LLMs Federated Medical Image Segmentation under Real-World Label Noise: A Benchmark Suite for Noisy Label Learning Method Selection Reservoir Attention Network: Cross-Pass State in Pretrained Transformers via Content-Addressable Reservoir Injection Explainable deep learning improves human mental models of self-driving cars, study finds SkillsBench Benchmark Measures How Agent Skills Boost LLM Performance Across Diverse Tasks Hidden Failure Modes in AI Reasoning: Study Reveals Oversight Paradox and Context-Injection Vulnerabilities InstantForget: New Update-Free Backdoor Unlearning Method Uses Inference-Time Feature Reset for AI Security Beyond Weights and Gradients: New Taxonomy Classifies Federated Learning Messages into Three Categories Token Reduction in Generative Models Must Evolve Beyond Efficiency, New Research Argues Semantic Flip: Synthetic OOD Generation for Robust Refusal in Embodied Question Answering and Spatial Localization Emergent Strategic Reasoning Risks in AI: New Taxonomy-Driven Framework Evaluates Deception and Gaming in LLMs Federated Medical Image Segmentation under Real-World Label Noise: A Benchmark Suite for Noisy Label Learning Method Selection Reservoir Attention Network: Cross-Pass State in Pretrained Transformers via Content-Addressable Reservoir Injection Explainable deep learning improves human mental models of self-driving cars, study finds SkillsBench Benchmark Measures How Agent Skills Boost LLM Performance Across Diverse Tasks

Home ›› Topics ›› self-attention

Topic

self-attention

1 story

Sub-Quadratic Vision Transformers Cut Self-Attention Cost for Faster Image Captioning

Artificial Intelligence #vision transformers#image captioning

Sub-Quadratic Vision Transformers Cut Self-Attention Cost for Faster Image Captioning

A new arXiv preprint from Ghosh et al. proposes a sub-quadratic vision transformer architecture for image captioning. By replacing standard self-attention with a Gaussian Mixture Model (GMM) clustering mechanism, the model reduces computational complexity from quadratic O(n²) to linear O(nK). The approach uses an autoregressive GPT-based decoder and achieves competitive results on the Flickr30K dataset.

Jun 16, 2026 1 source