Topic
llm evaluation
MA-ProofBench: New Benchmark Tests LLMs on Formal Theorem Proving in Mathematical Analysis
Researchers introduce MA-ProofBench, the first formal theorem-proving benchmark dedicated to mathematical analysis. It contains 200 theorems across six topics at two difficulty levels. Evaluations show that even the best model, GPT-5.5, achieves only 16% Pass@8 on undergraduate-level problems and 5% on Ph.D.-level problems, highlighting significant limitations of current LLMs in formal mathematical reasoning.
New Method Resolves Drift Attribution Ambiguity in LLM Evaluation Pipelines
A research paper introduces an anytime-valid attribution method for LLM evaluation pipelines that resolves the ambiguity between product drift and judge model changes. Using a fixed human-labeled anchor set and betting e-processes, the method achieved zero misattribution on silent version bumps and correctly attributed prompt changes in 110 of 120 runs, while the industry-default rolling z-test false-alarmed on 75% of drift-free streams.