Visit IGEN World Explore IGEN Expo

EXPLORE UPGRADE PLANS

BREAKING

Zepto IPO: Can 10-Minute Delivery Sustain Profitability Under Public-Market Scrutiny? CLoVE: New Federated Learning Algorithm Clusters Loss Vectors for Personalization SceneConductor Generates 3D Scenes from Single Images Using Multi-Agent Orchestration From Detection to Recovery: Operational Analysis of LLM Pre-training on 504 NVIDIA B200 GPUs Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention New EEG Benchmark Promises Standardized Evaluation of Foundation Models DCP-Prune: New Token Pruning Method Preserves AI Model Performance at Ultra-Low Budgets Robot Learning Reveals Emergent 'Self' Subnetwork in Continual Learning Studies New Book on Optimal Transport Offers Machine Learning Practitioners a Unified Framework Lightweight Distillation of SAM 3 and DINOv3 for Edge-Deployable Livestock Monitoring Zepto IPO: Can 10-Minute Delivery Sustain Profitability Under Public-Market Scrutiny? CLoVE: New Federated Learning Algorithm Clusters Loss Vectors for Personalization SceneConductor Generates 3D Scenes from Single Images Using Multi-Agent Orchestration From Detection to Recovery: Operational Analysis of LLM Pre-training on 504 NVIDIA B200 GPUs Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention New EEG Benchmark Promises Standardized Evaluation of Foundation Models DCP-Prune: New Token Pruning Method Preserves AI Model Performance at Ultra-Low Budgets Robot Learning Reveals Emergent 'Self' Subnetwork in Continual Learning Studies New Book on Optimal Transport Offers Machine Learning Practitioners a Unified Framework Lightweight Distillation of SAM 3 and DINOv3 for Edge-Deployable Livestock Monitoring

Home ›› Topics ›› llm pretraining

Topic

llm pretraining

1 story

AC-ODM: Actor-Critic Online Data Mixing for Sample-Efficient LLM Pretraining – A New Reinforcement Learning Approach

Artificial Intelligence #ac-odm#actor-critic

AC-ODM: Actor-Critic Online Data Mixing for Sample-Efficient LLM Pretraining – A New Reinforcement Learning Approach

Researchers introduce AC-ODM, an actor-critic online data mixing method that treats data composition as a reinforcement learning problem. On Pythia-1B, it achieves up to 66% fewer training steps to optimal perplexity, 27.5% relative MMLU accuracy improvement, and 2.23× higher HumanEval pass@1, with only 0.4% per-step wall-clock increase and 2% memory overhead. The method supports proxy and non-proxy modes for flexible deployment.

Jun 16, 2026 1 source