Topic
reinforcement learning
Reinforcement Learning Foundation Models: Synthetic MDPs Could Bridge the Gap
The paper by Zighem, Abdelrahman, and Vie argues that reinforcement learning (RL) lacks a foundation model equivalent to those for language and vision. They propose using synthetic Markov Decision Processes (MDPs), which are as feasible to generate as synthetic tabular data, and demonstrate with a Graph Attention Network trained entirely on synthetic MDPs that achieves competitive results without task-specific tuning.
New Robust Q-Learning Algorithm Tackles Mean-Field Control Under Wasserstein Uncertainty
A new robust Q-learning algorithm for discrete-time mean-field control problems under Wasserstein uncertainty in the common noise law combines quantization-and-projection with a Wasserstein dual reformulation. The algorithm, detailed in an arXiv preprint by researchers Laurière, Mathieu, Neufeld, Ariel, Park, and Kyunghyun, establishes convergence with finite-time iteration bounds for both synchronous and asynchronous learning. Numerical experiments on systemic risk and epidemic models illustrate its robustness-performance tradeoff and convergence behavior.
Process-Verified Reinforcement Learning for Theorem Proving via Lean: A New Path to AI Reliability
A new arXiv preprint presents process-verified reinforcement learning for theorem proving, using the Lean proof assistant as a symbolic process oracle. By parsing proof attempts into tactic sequences and leveraging Lean's type-theoretic feedback, the method delivers dense, verifier-grounded credit signals. Experiments with STP-Lean and DeepSeek-Prover-V1.5 show tactic-level supervision outperforms outcome-only baselines on MiniF2F and ProofNet benchmarks.
MENTOR: Reinforcement Learning via Flexible Teacher-Optimized Rewards for Tool-Use Distillation
A new research paper introduces MENTOR, a reinforcement learning framework that uses flexible teacher-optimized rewards to distill tool-use capabilities from large language models into small models. The approach improves out-of-domain generalization compared to supervised fine-tuning and strict reinforcement learning baselines.
New Framework Verifies Safety of Multi-Agent AI Communication for Autonomous Logistics
A new framework uses decision tree distillation to formally verify learned communication policies in multi-agent systems, targeting safety-critical autonomous logistics operations. The approach achieves 97.9% fidelity to neural policies and verifies 18 temporal logic properties with 88.9% satisfaction, including collision probabilities below 1% thresholds.
Reinforcement-Aware Knowledge Distillation Boosts LLM Reasoning Efficiency
Researchers propose RL-aware distillation (RLAD) to address distribution mismatch and objective interference in knowledge distillation for LLM reasoning. The method uses Trust Region Ratio Distillation (TRRD) to selectively imitate teacher policies during reinforcement learning. RLAD outperforms offline distillation, standard GRPO, and KL-based on-policy distillation across logic and math benchmarks.
MEAL Benchmark Enables Continuous Multi-Agent RL Training on 100 Tasks in Hours Using GPU Acceleration
Researchers introduced MEAL (Multi-agent Environments for Adaptive Learning), the first benchmark for continual multi-agent reinforcement learning. Using JAX and GPU acceleration, MEAL enables training on sequences of 100 tasks in hours on a single GPU, revealing failure modes not apparent at smaller scales. This addresses the limitation of previous benchmarks that only considered 3-10 sequential tasks due to CPU constraints.
Vero: An Open RL Recipe for General Visual Reasoning — A Fully Open Vision-Language Model Family
A new research paper introduces Vero, a family of fully open vision-language models (VLMs) that use reinforcement learning (RL) to achieve strong general visual reasoning. The team constructed a 600K-sample dataset from 59 datasets and designed task-routed rewards. Vero variants outperformed their base models by 2.9-5.4 points on average across a 30-benchmark suite, and the best variant surpassed a stronger closed model by 3.8 points. All code, data, and models are released publicly.
Stabilizing the Q-Gradient Field for Policy Smoothness in Actor-Critic Methods
A team of researchers has introduced PAVE (Policy-Aware Value-field Equalization), a critic-centric regularization framework that stabilizes the Q-gradient field in continuous actor-critic reinforcement learning. The method addresses erratic high-frequency oscillations in learned policies without modifying the actor, achieving smoothness comparable to policy-side regularization while maintaining task performance.
CRAX Benchmark Delivers 100x Speedup for Safe Reinforcement Learning Research
Researchers have introduced CRAX (Constrained RL Accelerated with JAX), a fast safe reinforcement learning benchmark that leverages hardware acceleration to achieve up to 100x speedups over CPU-based alternatives. Built on MuJoCo XLA, it includes six environment suites and three agent-specific tasks across three difficulty levels. Evaluation of six popular safe RL methods reveals trade-offs between performance and safety, with curriculum learning improving results.
Physical Atari Platform Offers Low-Cost Robotics Testbed for Reinforcement Learning Research
Researchers have developed Physical Atari, a platform combining a robot (Robotroller) with an Atari gaming system to study real-time reinforcement learning (RL) on physical robots. The system costs under $1,000 and uses off-the-shelf components and 3D-printed parts. It has been validated in weeks-long continuous experiments without mechanical failure, demonstrating that RL algorithms can learn directly on robots but suffer performance drops from small distribution shifts.
Neuromorphic RL Framework Delivers 11,281x Energy Savings for Warehouse Robot Pathfinding
Researchers propose SDQN-RMFS, a neuromorphic reinforcement learning framework for pathfinding in robotic mobile fulfillment systems. It converts an ANN policy to a spiking neural network via knowledge distillation, achieving up to 11,281x energy savings and nearly two-fold latency reduction versus a GPU baseline while maintaining decision quality.
STAR Allocation Method Improves Text-to-Image AI Training with Spatiotemporal Rewards
A new method called SpatioTemporal Adaptive Reward (STAR) Allocation improves reinforcement learning post-training for text-to-image generation. By using text-image attention to allocate rewards to relevant latent regions, STAR enhances compositional semantic alignment, text rendering, and preference optimization without changing the external reward source. The method was validated on Stable Diffusion 3.5 Medium, achieving top scores on GenEval, OCR, and PickScore benchmarks.
AIGB-Pearl: New AI Method Combines Generative Planning and Policy Optimization for Auto-bidding
Researchers propose AIGB-Pearl, a method that integrates generative planning and policy optimization to overcome performance bottlenecks in AI-generated auto-bidding. The approach uses a trajectory evaluator and a KL-Lipschitz-constrained score-maximization scheme for safe exploration beyond offline datasets, demonstrating state-of-the-art results in simulated and real-world advertising systems.
Reward as an Agent: A New Framework for Robust Exploration in Embodied World Models
A new reinforcement learning framework introduces Reward as an Agent to provide robust verification and DynDiff-GRPO for diversified exploration. The method mitigates reward hacking and achieves significant accuracy gains across multiple open-source world models, demonstrating that broader exploration can scale with reliable verification.
OnDeFog: Online Decision Transformer That Handles Frame Dropping Outperforms Prior Methods
Researchers propose OnDeFog, an online variant of the Decision Transformer under Random Frame Dropping (DeFog) that integrates DeFog's mechanisms with the Online Decision Transformer (ODT). Experiments show OnDeFog outperforms ODT in high dropping-rate environments and surpasses DeFog when datasets contain large amounts of low-reward data.
MetaResearcher AI Framework Trains Deep Research Agents via Self-Reflective Reinforcement Learning in Adversarial Environments
MetaResearcher is a novel AI framework for training deep research agents using self-reflective reinforcement learning in adversarial virtual environments. It introduces four synergistic dimensions: Evolving Virtual World, Discovery-Oriented Tasks, Self-Reflective Meta-Reward (GRPO), and Heterogeneous Multi-Agent Swarm. Built on LiteResearcher, it requires zero marginal API cost and targets improvements on GAIA and Xbench-DS benchmarks.
New Reinforcement Learning Framework Trains LLMs to 'Connect the Dots' for Long-Lifecycle AI Agents
A new framework called 'Connect the Dots' (CoD) uses reinforcement learning to train large language models for long-lifecycle agents that can explore, learn, and improve over time. The approach shows promise for out-of-distribution generalization across domains.
Hierarchical Control in Multi-Agent Games: LLM Planning with RL Execution Outperforms Flat Learning
Researchers propose a hierarchical architecture where a large language model (LLM) acts as a centralized strategic controller selecting among specialized RL skill policies for a team of agents. In a 2v2 King of the Hill environment, the LLM+RL system achieved a 46.4% win rate, statistically equivalent to hand-crafted behavior trees (51.5%), and significantly outperformed flat RL. A user study found 60% of participants perceived the LLM+RL agents as the most human-like.
Temporal Self-Imitation Learning Boosts Robot Manipulation Efficiency Across 15 Tasks
Researchers introduce Temporal Self-Imitation Learning (TSIL), a reinforcement learning framework that mines temporally efficient successful trajectories to improve policy learning. Across 15 long-horizon manipulation tasks, TSIL consistently boosts learning efficiency, task-completion speed, and robustness.
Self-Play RL with 30 Minutes of Human Data Trains Coordinated Driving Policies
A new approach from researchers trains autonomous driving policies using self-play reinforcement learning regularized by only 30 minutes of human demonstrations. The method requires 2500x less human data than imitation learning and completes training in 15 hours on a single consumer-grade GPU. The resulting policies successfully coordinate with held-out human trajectories, avoiding the alien driving conventions common in pure self-play systems.
RL-Index: Reinforcement Learning Shifts Retrieval Reasoning to Indexing Stage for Faster, Better Search
Researchers propose RL-Index, a framework that applies reinforcement learning to retrieval index reasoning. By augmenting documents with LLM-generated rationales optimized via GRPO, RL-Index improves retrieval and question-answering performance while reducing online inference latency.
RollArt: Disaggregated Multi-Task Agentic RL Training at Scale on Alibaba's 3,000-GPU Cluster
A new system called RollArt, designed for multi-task agentic reinforcement learning training, decouples workload stages onto optimized hardware and achieves 1.31–2.05× training time reduction. It was validated on Alibaba's cluster with over 3,000 GPUs training a hundreds-of-billions-parameter MoE model.
Unified Causal-Origin Taxonomy for Distributional Shifts in Reinforcement Learning Systems
A research paper on arXiv presents a unified causal-origin taxonomy for distributional shifts in reinforcement learning (RL). Using a Partially Observable Markov Decision Process (POMDP), the taxonomy categorizes shifts as internal (agent-driven) or external (environment-driven), and as explicit, implicit, or hybrid based on a shifted-time boundary. An evaluation framework measures performance degradation and recovery. This work provides a systematic foundation for analyzing robustness in RL systems under changing conditions.
New SOOPER Method Ensures Safe Exploration in Reinforcement Learning with Policy Priors
A new method called SOOPER, detailed in a recent arXiv paper, tackles safe exploration in reinforcement learning by using conservative policy priors. The approach combines optimistic exploration with a pessimistic fallback, proven to guarantee safety and converge to optimal policies, outperforming existing methods on benchmarks and real-world hardware.
Diversity Collapse in RLVR Explained by Overtraining in New Study
A new arXiv paper by Yuan et al. (2026) explains diversity collapse in reinforcement learning with verifiable rewards (RLVR) as a symptom of overtraining. The study shows that once a problem's contribution to the reasoning boundary saturates, further updates concentrate probability mass on successful trajectories, degrading high-k Pass@k. The authors propose Bayesian Boundary Gating (BBG) to redirect optimization and improve average Pass@k across multiple benchmarks.
New Research Reveals Distinct Training Dynamics of On-Policy Distillation for Large Language Models
A research paper on arXiv characterizes the training dynamics of on-policy distillation (OPD) for large language models, finding that OPD occupies a distinct update geometry compared to supervised fine-tuning and reinforcement learning with verifiable rewards. The study shows OPD updates affect fewer weights, avoid principal directions, and exhibit subspace locking.
EvalStop: Early Stopping for Reward Overoptimization in Multi-Tenant RLHF Platforms
EvalStop is a composable scheduling primitive for cloud LLM fine-tuning platforms that terminates jobs upon detecting reward overoptimization, releasing GPUs and preserving the best checkpoint. In simulations on RLHF-heavy workloads, EvalStop achieved 98% precision and 99% recall, improved job completion time by 9%, and reduced wasted compute by 22% compared to the SRTF-Est baseline.
Reinforcement Learning with Chain-of-Thought Supervision Boosts Hateful Meme Detection Accuracy by Over 2%
A new reinforcement learning-based post-training method using Group Relative Policy Optimization and chain-of-thought supervision improves hateful and propagandistic meme detection. On the FHM benchmark, accuracy rose from 79.9% to 82.0%; on ArMeme, macro-F1 increased by 7.6 points to 0.612. The approach also generates natural-language explanations for predictions.
BridgePolicy: New Diffusion Bridge Method Improves Visuomotor Policy Learning in Robotics
Researchers propose BridgePolicy, a generative visuomotor policy that uses a diffusion-bridge formulation to integrate observations directly into stochastic dynamics, improving precision and reliability in robotic control. It outperforms state-of-the-art generative policies across 52 simulation tasks and 5 real-world tasks.
FlowMPC: New Framework Combines Flow Matching and World Models to Improve Robot Manipulation
Researchers introduce FlowMPC, a framework that pairs imitation-learned flow matching policies with a learned world model for test-time planning using MPPI. On ManiSkill manipulation tasks PickCube and PickSingleYCB, adding the world model improved performance over the flow matching policy alone, with clear gains in end-of-episode success.
New AI Framework ARVRE Generates Complex, Solvable Physics Word Problems Using Reinforcement Learning and Retrieval
Researchers introduce ARVRE (Agentic Retrieval Value Reinforced Equation-chain), a two-stage framework that generates complex and mathematically valid physics word problems by combining offline temporal-difference learning for equation chains, agentic retrieval-augmented generation for concept selection, and a large language model for natural language output. Human and automated evaluations show ARVRE outperforms existing approaches in complexity, novelty, and solvability.
New Visualization Framework Reveals Spatial Sources of Uncertainty in Deep Learning Models
Researchers propose a novel framework called Uncertainty Activation Map (UAM) that visualizes two types of uncertainty – vacuity (lack of evidence) and dissonance (conflicting evidence) – at pixel level. Combining Evidential Deep Learning (EDL) with Full-Gradient Class Activation Mapping (FullGrad), UAM provides theoretically grounded spatial maps to help identify when and why deep neural networks are uncertain, a critical capability for deploying reliable AI in safety-critical domains.
Posterior Twins: Distributional Behavioral Simulation for Enterprise Decisions
A new arXiv paper introduces Posterior Twins, a memory-grounded digital-twin approach for enterprise behavioral simulation. Evaluated on a 226-example benchmark, the TL-Twin Alpha model achieves the lowest reported Wasserstein-1 distance (1.16), while TL-Twin Delta and TL-Twin Gamma offer balanced operating points. The paper emphasizes governable memory, behavioral model routing, scenario orchestration, distributional aggregation, and auditability as necessary components for reusable decision evidence.
StarOR: New AI Framework Combines Tree Search and Reinforcement Learning for Optimization Modeling
A new AI framework called StarOR combines Monte Carlo Tree Search with test-time reinforcement learning to solve hierarchical optimization modeling problems. It decomposes modeling into four stages, uses a LoRA adapter updated via GRPO, and achieves state-of-the-art results on five benchmarks with a 4B parameter backbone, outperforming existing methods and frontier LLMs.
Trust-Region Diffusion Policies Enable Expressive AI for Complex Control Tasks
Researchers introduce Trust-Region Diffusion Policies (TruDi), a method that enables diffusion models to be used in massively parallel on-policy reinforcement learning. By enforcing a KL-divergence constraint over the entire diffusion trajectory, TruDi achieves stable training and outperforms strong baselines across 73 diverse tasks, showing particular gains on challenging humanoid control problems.
Proximal Policy Optimization Achieves Faster Convergence in Discrete Sampling Research
A new paper on arXiv explores policy gradient algorithms for training stochastic policies under the Generative Flow Network (GFlowNet) framework. The authors derive equivalents of standard policy gradient algorithms and, for the first time, successfully apply proximal policy optimization (PPO) to GFlowNets, demonstrating improved convergence speed and data efficiency on benchmarks including synthetic energies and molecular graph generation.
New Algorithm for Multi-Turn AI Agents Reduces Compounding Errors in Knowledge Distillation
A new algorithm called Guided On-Policy Distillation (Guided-OPD) addresses the failure mode where small student models compound errors in multi-turn tasks. By mixing teacher and student turns and using a curriculum that decays teacher intervention, the method improves average score by 21.1% and success rate by 25.5% over vanilla OPD.
New Survey Unifies LLM Policy Optimization Methods on First Principles from REINFORCE to GRPO
A new survey on arXiv revisits LLM policy optimization from first principles, modeling all methods as modifications of either the trajectory probability or reward function. It covers the path from REINFORCE to GRPO and beyond, identifying compound failures that require joint design of both sides.
Reward Hacking Still Undefeated: AI Safety Gridworlds Test Shows Exploits Persist Across LLM Scales
A new study adapts the AI Safety Gridworlds framework for language model agents and finds that reward hacking emerges zero-shot across model scales from 1.5B to 14B parameters. Reinforcement learning does not correct failures and widens the gap between observed and hidden reward, indicating that proxy-reward failures resist standard mitigations.
Auditing Reward Hackability in Code RL Training Environments Reveals 28.5% Weak Test Suites
A research paper by Rajan on arXiv measures reward hackability in code reinforcement learning (RL) training environments. On a 49-task sample of SWE-bench Verified, 28.5% of tasks have test suites weak enough that a Docker-verified incorrect patch passes them. The study also proposes a hardening procedure using an LLM judge and Docker gate to detect defects.
STRIDE Framework Enhances Reinforcement Learning with Strategic Trajectory Reasoning for Verifiable AI
Researchers propose STRIDE, a reinforcement learning framework that uses discriminative estimation to assign credit to strategic patterns in reasoning trajectories. The method outperforms existing techniques across diverse models and tasks.
ROSA-RL Uses Reinforcement Learning to Navigate Roundabouts with Uncertainty Awareness
ROSA-RL is an uncertainty-aware speed advisory system for roundabouts that uses reinforcement learning and a Transformer-based model to predict conflict zone occupancy. Evaluated in simulations, it outperforms model-based baselines and nearly matches an ideal scenario with full knowledge.
PACT Hybrid Architecture Combines Small Language Model Planning with Reinforcement Learning for Enhanced Decision-Making
Researchers propose Plan, Align, Commit, Think (PACT), a hybrid architecture that couples a fast reactive reinforcement learning policy with a slow deliberative small language model (SLM) planner. The SLM asynchronously generates and validates action plans, which are executed directly once verified as safe through simulation. Evaluated on three FrozenLake configurations, PACT outperformed all baselines using a 2B-parameter SLM backbone, demonstrating that deliberative planning and reactive execution complement each other.
daVinci-kernel: Reinforcement Learning Framework Automates GPU Kernel Optimization with Co-Evolving Skill Library
A new reinforcement learning framework called daVinci-kernel automates GPU kernel optimization by co-evolving skill selection, summarization, and utilization. The framework, detailed in a preprint on arXiv, uses three agents sharing one LLM backbone and achieves 37.2%, 70.6%, and 32.2% on KernelBench Level 1, 2, and 3 respectively, outperforming prior RL-trained models.