monadic

QMFOL Benchmark Reveals LLM Reasoning Degrades with Logical Complexity, New Framework Enables Precise Evaluation

A new automated framework called QMFOL generates deductive reasoning tasks with quantifiable logical complexity, enabling precise evaluation of LLM reasoning. The associated benchmark, QMFOLBench, comprises 2,880 instances across 960 configurations. Evaluations on six large reasoning models (LRMs) and two LLMs show performance degrades and computational overhead increases with rising logical complexity, with models performing better on True-labeled tasks than False or Unknown ones.