Topic
gradient descent
Artificial Intelligence #fastmix#data mixture
FastMix: Gradient-Based Data Mixture Optimization Reduces Search Cost in AI Training
FastMix is a novel framework that automates data mixture discovery by training only a single proxy model and jointly optimizing mixture coefficients and model parameters via gradient descent. It reformulates mixture selection as a bilevel optimization problem, enabling efficient, scalable optimization that outperforms baselines.
Jun 17, 2026 1 source
Artificial Intelligence #gradient descent#recurrent networks
New Architecture GRIL Enables Gradient Descent-Like Learning in Linear Recurrent Networks
Researchers introduce the Gradient-based Recurrent In-context Learner (GRIL), a linear recurrent network architecture with windowed cross-product self-attention that can implement minibatch gradient descent on a task-specific predictor in a single forward pass. The design achieves strong performance on synthetic in-context learning tasks, Long Range Arena, and language modeling.
Jun 16, 2026 1 source