iGEN
Visit IGEN World Explore IGEN Expo
EXPLORE UPGRADE PLANS
BREAKING
NordVPN's Private Server Add-On Gives Enterprises Isolated Hardware and Static IP for Secure Remote Access India Soyabean Acreage Seen Rising Up to 10% on High Prices, Weak Monsoon Outlook FlowMPC: New Framework Combines Flow Matching and World Models to Improve Robot Manipulation DYNA Framework Uses Temporal Knowledge Graphs to Reduce LLM Forgetting Without Retraining RAMS: Resource-Adaptive Model Switching for Embedded Edge Perception Under Load Open-SWE-Traces: 207K Multilingual Trajectories Set New Standard for Autonomous Software Engineering Agents Infant-Inspired Noise Boosts Deep RL Exploration, Research from arXiv Shows Mutual Distillation of Dual Foundation Models Achieves State-of-the-Art PET/CT Segmentation with Only 5 Labeled Cases SPARK Method Activates Latent Security Knowledge in LLMs for Secure Code Generation Apple explains why Siri AI took so long: first version ready last year but rebuilt from ground up NordVPN's Private Server Add-On Gives Enterprises Isolated Hardware and Static IP for Secure Remote Access India Soyabean Acreage Seen Rising Up to 10% on High Prices, Weak Monsoon Outlook FlowMPC: New Framework Combines Flow Matching and World Models to Improve Robot Manipulation DYNA Framework Uses Temporal Knowledge Graphs to Reduce LLM Forgetting Without Retraining RAMS: Resource-Adaptive Model Switching for Embedded Edge Perception Under Load Open-SWE-Traces: 207K Multilingual Trajectories Set New Standard for Autonomous Software Engineering Agents Infant-Inspired Noise Boosts Deep RL Exploration, Research from arXiv Shows Mutual Distillation of Dual Foundation Models Achieves State-of-the-Art PET/CT Segmentation with Only 5 Labeled Cases SPARK Method Activates Latent Security Knowledge in LLMs for Secure Code Generation Apple explains why Siri AI took so long: first version ready last year but rebuilt from ground up
Home ›› Topics ›› long-context

Topic

long-context

2 stories
Parallel Hybrid Architecture Combines GSS and Attention for Efficient Long-Context Language Modeling Technology
Artificial Intelligence #long-context#transformer

Parallel Hybrid Architecture Combines GSS and Attention for Efficient Long-Context Language Modeling

Researchers propose the Parallel Hybrid Architecture (PHA), combining Gated State Spaces, Grouped Query Attention, and Feed-Forward Networks in parallel branches fused by a learnable mixing mechanism. On WikiText-103, PHA achieves 16.51 PPL at 125M parameters, outperforming comparable models, and scales to 180M parameters with 16.42 PPL while delivering 24% higher throughput and up to 40% lower memory usage.

Jun 16, 2026 1 source
MMLongEmbed Benchmark Reveals Limitations in Long-Context Multimodal Embedding Models Technology
Artificial Intelligence #multimodal#embedding

MMLongEmbed Benchmark Reveals Limitations in Long-Context Multimodal Embedding Models

MMLongEmbed is the first comprehensive benchmark for evaluating multimodal embedding models (MEMs) in long-context scenarios. It comprises four retrieval tasks covering text, document, and video modalities. The evaluation reveals that current MEMs rely heavily on superficial feature matching and struggle with deep semantic and structural dependencies, with performance degrading systematically based on context length and key information placement.

Jun 16, 2026 1 source