Modern recommendation systems increasingly rely on dynamically routing diverse queries to multiple embedding models. Despite its practical significance, this problem remains poorly understood under realistic conditions like adversarial queries, bandit feedback, and limited observability of models, according to a new paper on arXiv.
The research team, including Dai, Yan, Golrezaei Negin, and Jaillet Patrick, formalizes embedding model routing as an adversarial contextual linear bandit with low-rank experts. In this framework, contexts are queries, actions are items, and experts are the embedding models working on low-rank latent representation spaces. The authors first establish that standard regret notions suffer from structural misspecification or statistical intractability, and they identify a log-quadratic policy class that is expressive enough to capture query-dependent model routing, yet structured enough to allow efficient online learning.
Key Theoretical Contributions
The paper's core contribution is a policy gradient algorithm called Hypentropy Policy Gradient (HPG). It provably adapts to the unknown low-rank structure under incomplete information and attains $\tilde{\mathcal O}(s\sqrt{M T})$ linearized policy regret — where $s$, $M$, and $T$ are the intrinsic rank of the experts, the number of models, and the number of rounds — thus avoiding a curse of dimensionality. The regret bound scales with the intrinsic rank rather than the full dimensionality of the embedding space, enabling efficient routing even when many models are available.
| Parameter | Description |
|---|---|
| $s$ | Intrinsic rank of the experts |
| $M$ | Number of models |
| $T$ | Number of rounds |
| Regret | $\tilde{\mathcal O}(s\sqrt{M T})$ |
The HPG Algorithm
HPG is designed to be computationally efficient and parameter-free, according to the paper. This means practitioners can deploy it without extensive hyperparameter tuning, a significant advantage in real-world systems where embeddings are updated frequently. The algorithm operates under bandit feedback — only the reward for the chosen action is observed — and handles adversarial queries, making it robust to shifts in user behavior or malicious inputs.
Industry Implications
For enterprise technology leaders, embedding model routing is a critical component of large-scale recommendation systems used in e-commerce, content platforms, and advertising. The ability to dynamically select the best embedding model for each query can improve relevance and user engagement while reducing computational cost. HPG's theoretical guarantees and practical design could make it attractive for implementation in production environments. The paper provides a foundation for future work on embedding model selection under limited observability.
The research is published on arXiv and has not yet been peer-reviewed, but it offers a rigorous theoretical framework for a problem that has seen little formal analysis. As recommendation systems continue to scale, routing algorithms like HPG may become essential infrastructure.