llm as judge

AURA: Adaptive Uncertainty-Aware Refinement Framework for Auditing LLM-as-a-Judge Decisions

A new framework named AURA (Adaptive Uncertainty-Aware Refinement) addresses the challenge of auditing large language models when used as judges for open-ended generation. It iteratively learns a human-consistency signal, propagates reliable evidence, and prioritizes uncertain comparisons for human review. The approach treats trust in a judge as a latent quantity that is progressively refined as evidence accumulates.