Organizations increasingly use large language models (LLMs) for complex mathematical optimization, often without explicit awareness. A survey published on arXiv by researchers Peran, Roko, Hobor, Luka, Kovac, Mihael, and Brcic (2026) categorizes the field of LLM-as-optimizer into three paradigms: direct optimization, tool-augmented optimization, and tool-creating optimization.
Three Paradigms of LLM Optimization
According to the survey, direct optimization relies on iterative prompting and heuristic generation to navigate solution spaces. Tool-augmented optimization translates natural language problems into formal specifications and orchestrates external solvers. Tool-creating optimization goes further, using LLMs to discover reusable algorithms or heuristics that can be deployed at zero marginal LLM cost.
| Paradigm | Description | Key Characteristic |
|---|---|---|
| Direct optimization | Iterative prompting and heuristic generation | No external tools; zero marginal LLM cost after creation |
| Tool-augmented optimization | Translates problems into formal specs, orchestrates external solvers | Auditability via external solver outputs |
| Tool-creating optimization | LLMs discover reusable algorithms/heuristics | Zero marginal LLM cost after creation |
Performance Frontiers Based on Benchmarks
The survey describes current performance frontiers based on benchmarks from the literature. It notes that tool-augmented optimization offers auditability—a key advantage for enterprise deployments requiring traceability. In contrast, direct optimization, while potentially more flexible, may suffer from a reasoning gap that limits its effectiveness on complex problems.
The Critical Reasoning Gap
The survey identifies a critical reasoning gap in current architectures. This gap affects the ability of LLMs to perform reliable optimization without external validation. The authors argue that even future, more powerful models might opt for tool-making to improve operational efficiency for repetitive families of problems.
Trade-offs Between Future Potential and Auditability
The survey argues for trade-offs between the future potential of direct optimization and the auditability of tool-augmented optimization. For enterprise technology leaders, this trade-off is crucial: direct methods may offer lower cost and faster iteration, but lack the verifiability required in regulated environments like supply chain or trade finance.
In summary, the survey provides a structured framework for understanding how LLMs can function as optimizers. As these models become more capable, the choice between direct, tool-augmented, or tool-creating approaches will depend on the specific requirements for accuracy, cost, and auditability in each use case.
