Artificial Intelligence #llms#legal reasoning
Study Finds LLMs' Legal Reasoning Unfaithful: Scope Laundering and Formalization Flaws Identified
A study comparing LLM classification, LLM-based formal reasoning, and solver-based reasoning on ContractNLI finds that while formal reasoning improves accuracy, it does not guarantee faithfulness. Researchers identify three recurring failure modes: scope laundering, implicit constraint blindness, and program synthesis failures. The findings raise concerns about relying on LLM-based formal reasoning as a proxy for symbolic execution.
Jun 16, 2026 1 source