Distilling the tool-use capabilities of large language models (LLMs) into small language models (SLMs) is a critical step for deploying AI in resource-constrained environments. According to a paper published on arXiv (2510.18383), the widely used supervised fine-tuning (SFT) method suffers from poor out-of-domain (OOD) generalization due to rigid alignment with static teacher trajectories. The research team proposes MENTOR, a reinforcement learning framework that introduces a flexible, process-aware reward structure to bridge the capacity gap between large and small models.
The Problem with Supervised Fine-Tuning
SFT trains SLMs to mimic the exact outputs of a teacher LLM. However, this rigid replication leads to poor performance when the model encounters tasks outside the training distribution. The paper notes that "distilling the tool-use capabilities of large language models (LLMs) into small language models (SLMs) is essential for their practical application," but SFT's dependence on static trajectories limits adaptability.
The Reinforcement Learning Dilemma
Reinforcement learning (RL) offers an alternative, but capacity limitations of SLMs create a dilemma. Sparse outcome rewards provide insufficient guidance, while strict trajectory matching imposes overly restrictive constraints. The researchers describe this as a "capacity-driven gap" that neither extreme addresses effectively.
Introducing MENTOR
MENTOR addresses this gap by using the teacher's reference to guide tool-use behavior without enforcing rigid replication. The method balances behavioral alignment with downstream performance. According to the paper, "instead of enforcing rigid replication, MENTOR uses the teacher's reference to guide tool-use behavior, balancing behavioral alignment with downstream performance." This flexible yet process-aware reward structure is designed for verifiable tool-use environments.
Experimental Results
The team conducted extensive experiments on controlled executable-tool benchmarks. The results show that MENTOR improves OOD tool-use performance compared to both SFT and strict RL baselines. The paper states: "Our findings suggest that within verifiable tool-use environments, flexible tool-use alignment offers a more effective approach than strict trajectory replication for developing adaptable small models." The following table summarizes the comparative performance:
| Method | OOD Generalization | Guidance Density | Behavioral Alignment |
|---|---|---|---|
| SFT | Poor | High (static) | Rigid |
| Strict RL | Moderate | Low (sparse) | Restrictive |
| MENTOR | Improved | Moderate (flexible) | Adaptive |
Implications for Enterprise AI
For enterprise technology leaders, this research underscores the importance of distillation techniques that balance efficiency with adaptability. While the experiments focus on tool-use, the principles could extend to other domains where SLMs must handle diverse and unexpected inputs. The authors—ChangSu Choi, Hoyun Song, Dongyeon Kim, WooHyeon Jung, Minkyung, Sunjin Park, NohHyeob Bae, Seona Yu, and KyungTae Lim—provide an open-access paper on arXiv. The approach may influence future development of small models for applications like supply chain automation, where tools and APIs vary across contexts. However, real-world deployment would require further validation on industry-specific benchmarks.