Topic
routing
New Algorithm SLARouter Cuts LLM Inference Costs by 2.2x While Guaranteeing SLA Compliance
A new research paper introduces SLARouter, an online routing algorithm that learns a cost-optimal policy from sparse user feedback while guaranteeing SLA compliance. Experiments show up to 2.2x cost reduction over existing baselines without per-benchmark tuning.
Low-Policy-Regret Algorithm for Embedding Model Routing in Contextual Bandits
A new paper on arXiv formalizes embedding model routing as an adversarial contextual linear bandit problem. The authors propose Hypentropy Policy Gradient (HPG), which provably adapts to unknown low-rank structure and attains low linearized policy regret.
ChatPlanner: LLM Framework Personalizes Public Transit Routing with Fine-Tuning and RAG
Researchers present ChatPlanner, a novel framework that leverages fine-tuned Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) to capture diverse user preferences for public transit routing. The system extracts routing parameters from natural language queries, integrates preferences into the routing algorithm, and generates feasible, personalized alternatives. Three experiments show that the combined fine-tuning and RAG approach achieves highest accuracy and uncovers valuable solutions overlooked by existing route planners.
New Attack Forces Costly Model Usage in Multimodal LLM Cascades
A research paper introduces the Forced Deferral Attack (FDA), which manipulates confidence thresholds in multimodal large language model cascades, causing queries to be routed to more expensive strong models. The attack raises security concerns for enterprises deploying cost-optimized AI systems.