iGEN
Visit IGEN World Explore IGEN Expo
EXPLORE UPGRADE PLANS
BREAKING
Amazfit Cheetah 2 Ultra: The Most Expensive Smartwatch Yet—Is It Worth the Price? New Automated Jailbreak Attack UNIATTACK Achieves High Success Rate Against Multi-Layered LLM Defenses UXBench: Measuring the Actionability of LLM-Generated UX Critiques LaWAM: Latent World Action Model Enables Efficient, Dynamics-Aware Robot Control with Low Latency Sub-Quadratic Vision Transformers Cut Self-Attention Cost for Faster Image Captioning NordVPN's Private Server Add-On Gives Enterprises Isolated Hardware and Static IP for Secure Remote Access India Soyabean Acreage Seen Rising Up to 10% on High Prices, Weak Monsoon Outlook FlowMPC: New Framework Combines Flow Matching and World Models to Improve Robot Manipulation DYNA Framework Uses Temporal Knowledge Graphs to Reduce LLM Forgetting Without Retraining RAMS: Resource-Adaptive Model Switching for Embedded Edge Perception Under Load Amazfit Cheetah 2 Ultra: The Most Expensive Smartwatch Yet—Is It Worth the Price? New Automated Jailbreak Attack UNIATTACK Achieves High Success Rate Against Multi-Layered LLM Defenses UXBench: Measuring the Actionability of LLM-Generated UX Critiques LaWAM: Latent World Action Model Enables Efficient, Dynamics-Aware Robot Control with Low Latency Sub-Quadratic Vision Transformers Cut Self-Attention Cost for Faster Image Captioning NordVPN's Private Server Add-On Gives Enterprises Isolated Hardware and Static IP for Secure Remote Access India Soyabean Acreage Seen Rising Up to 10% on High Prices, Weak Monsoon Outlook FlowMPC: New Framework Combines Flow Matching and World Models to Improve Robot Manipulation DYNA Framework Uses Temporal Knowledge Graphs to Reduce LLM Forgetting Without Retraining RAMS: Resource-Adaptive Model Switching for Embedded Edge Perception Under Load
Home ›› Topics ›› mixture of experts

Topic

mixture of experts

2 stories
SPRI: SVD-Partitioned Residual Initialization Boosts Data-Constrained MoE Upcycling for Multilingual Translation Technology
Artificial Intelligence #artificial intelligence#machine learning

SPRI: SVD-Partitioned Residual Initialization Boosts Data-Constrained MoE Upcycling for Multilingual Translation

Researchers propose SPRI, a method that initializes Mixture-of-Experts (MoE) models from pretrained dense models using SVD-partitioned residuals. Evaluated on multilingual speech-to-text translation, SPRI achieves gains of 2.58 BLEU and 3.32 COMET over fine-tuned dense models, and outperforms prior MoE upcycling baselines by 3.39 BLEU and 4.34 COMET points.

Jun 16, 2026 1 source
Expert Tying Reduces Memory Footprint of Mixture-of-Experts LLMs by Nearly Half Technology
Artificial Intelligence #tied expert layers#mixture-of-experts

Expert Tying Reduces Memory Footprint of Mixture-of-Experts LLMs by Nearly Half

A new arXiv paper from Jaggi proposes Expert Tying, an architectural modification for Mixture-of-Experts LLMs that shares expert parameters across consecutive transformer layers. Pretraining experiments show memory footprint reduction by almost 2x with virtually no degradation in perplexity or downstream quality, evaluated on OLMoE, Qwen3, and DeepSeek-style architectures.

Jun 16, 2026 1 source