High-school tutoring powered by large language models (LLMs) often falls short because static prompts cannot adapt to different academic subjects. Researchers from Leiden University and elsewhere have developed a system that uses subject-aware prompting, dynamically selecting the best pedagogical strategy based on 14 features extracted from raw conversation transcripts.
The system first trains a prompt routing model in a simulation environment, then deploys it for online adaptation with real students. In a simulation benchmark, the router outperformed two static baselines with a score of 0.694 compared to 0.647 and 0.64 (p < 0.001), according to the paper "Learning to Prompt" by Chang, Po-Chin; Hogan, Nicholas; Plaat, Aske; and Van Der Meer, Michiel T, published on arXiv.
Real-World Validation
A/B testing was conducted with 656 conversations from 359 high-school students. The results demonstrated sim-to-real transfer: the model switched from analytical to scaffolding learning strategies when moving from simulation to real deployment. The adaptive prompt selection mechanism improved instructional efficiency, maintaining pedagogical quality while reducing interactions by around 3 turns (p = 0.007).
The key metric for tutoring effectiveness was the exercise conversion rate — the percentage of conversations where the student successfully completed an exercise.
| Strategy | Exercise Conversion Rate |
|---|---|
| Static baseline | 19.6% |
| Greedy router | 19.1% |
| Stochastic router | 28.1% |
While a greedy router that always picks the best-known strategy achieved a comparable 19.1% conversion rate, a stochastic router that samples from a distribution of strategies led to a significantly higher 28.1%. The researchers attribute this to better exploration of teaching approaches, which keeps students more engaged.
Implications for Adaptive Learning
The study highlights that LLM-based tutoring systems can benefit from dynamic prompt selection rather than using the same instruction model for all subjects and students. The 14 pedagogical features — including tutor scaffolding and student understanding — were extracted from raw transcripts to inform the routing decision.
The authors note that the simulation environment was crucial for training the router before deployment, and that the sim-to-real transfer was successful. Future work may extend the approach to more subjects and larger student populations.
For enterprise decision-makers in education technology, this research suggests that adaptive LLM interfaces can yield measurable improvements in learning outcomes, with 28.1% conversion representing a substantial jump over static methods. The reduction of 3 interactions per conversation also means less chat time to achieve the same pedagogical goal.
As LLMs become more common in enterprise training and customer education, the principle of subject-aware, adaptive prompting could apply beyond high school to corporate learning management systems, where engagement and completion rates are critical success metrics.
