Topic
uncertainty estimation
Systematic Evaluation Reveals No Single Black-Box Uncertainty Estimation Method Dominates for Large Language Models
A systematic review and benchmark of 24 black-box uncertainty estimation methods for large language models found that no single method consistently outperforms across all settings. Methods that reason over candidates in the answer space are generally effective, and hybrid methods combining multiple uncertainty signals perform well under most conditions. The study categorizes methods into five types and provides a unified evaluation framework.
Calibrated Variance Propagation Cuts Uncertainty Estimation Cost for Deep Learning Models
A new method called Calibrated Variance Propagation (CVP) enables accurate, sampling-free uncertainty estimation in modern deep learning architectures. The technique matches Monte Carlo sampling accuracy at lower cost, improving coverage by 6-8 percentage points on transformer models in vision-language tasks.