ArXiv

ORCE: Order-Aware Alignment of Verbalized Confidence in Large Language Models

2026-05-12 · 17:39 UTC ·Chen Li, Xiaoling Hu, Songzhu Zheng... ·1 min read

Authors: Chen Li, Xiaoling Hu, Songzhu Zheng...
Categories: cs.LG, cs.CL
arXiv: https://arxiv.org/abs/2605.12446v1
PDF: https://arxiv.org/pdf/2605.12446v1

Brief

ORCE introduces a decoupled, order-aware approach to verbalized confidence: answers are generated first and confidence is estimated conditioned on the fixed question–answer pair. Using a sampling-based surrogate of multiple completions and rank-based RL to encourage higher confidence for more likely-correct responses, the method improves calibration and failure prediction without substantially hurting answer accuracy, addressing interference from prior joint optimization techniques.

Why it matters

ORCE (Chen Li, Xiaoling Hu, Songzhu Zheng, Jiawei Zhou, Chao Chen; arXiv 2026-05-12) proposes a decoupled, order-aware verbalized-confidence framework that first generates an answer then conditions confidence estimation on the fixed question–answer pair.

Key details

The method builds a sampling-based surrogate from multiple model completions and optimizes rank-based reinforcement-learning objectives to align confidence ordering; experiments on reasoning and knowledge-intensive benchmarks report improved calibration and failure prediction while largely preserving answer accuracy (paper: 18 pages, 2 figures).

Source evidence

Abstract

Large language models (LLMs) often produce answers with high certainty even when they are incorrect, making reliable confidence estimation essential for deployment in real-world scenarios. Verbalized confidence, where models explicitly state their confidence in natural language, provides a flexible and user-facing uncertainty signal that can be applied even when token logits are unavailable. However, existing verbalized-confidence methods often optimize answer generation and confidence generation jointly, which can cause confidence-alignment objectives to interfere with answer accuracy. In this work, we propose a decoupled and order-aware framework for verbalized confidence calibration. Our method first generates an answer and then estimates confidence conditioned on the fixed question--answer pair, allowing confidence optimization without directly perturbing the answer-generation process. To align confidence with correctness likelihood, we construct a sampling-based surrogate from multiple model completions and optimize rank-based reinforcement learning objectives that encourage responses with higher estimated correctness likelihood to receive higher verbalized confidence. Experiments on reasoning and knowledge-intensive benchmarks show that our method improves calibration and failure prediction performance while largely preserving answer accuracy. These results demonstrate that verbalized confidence can be more reliably aligned by decoupling confidence estimation from answer generation and optimizing the relative ordering of confidence across responses.

Comment: 18 pages, 2 figures