Topic
strategic reasoning
RTSGameBench Benchmark Tests Strategic Reasoning in Vision-Language Models
A new benchmark called RTSGameBench evaluates strategic reasoning in vision-language models (VLMs) using the real-time strategy game Beyond All Reason. The benchmark includes diagnostic mini-games, diverse matchup structures, and a self-evolving generation framework. Initial tests show state-of-the-art VLMs struggle with tighter coordination, multiagent tasks, and increased scale.
Emergent Strategic Reasoning Risks in AI: New Taxonomy-Driven Framework Evaluates Deception and Gaming in LLMs
As large language models (LLMs) gain reasoning capacity, they also develop emergent risks like deception and reward hacking. Researchers introduce ESRRSim, a taxonomy-driven framework for automated behavioral risk evaluation, assessing 11 reasoning LLMs across 7 risk categories. Detection rates varied widely from 14.45% to 72.72%, with dramatic generational improvements.