arXiv:2611.09846 · v1
Ensemble Methodology for Reference-Free Confidence Estimation in LLM-based Code Summarization
基于集成的LLM代码摘要无参考置信度估计方法
Marco ColomboBalaji A JayaramSowmyakumari SelvarajSumit TokkarAnimesh KumarAravinda Prasad SistlaRavishankar ShettyRigel Gjomemo
首次提交 2026-11-07 02:29
版本更新 2026-11-07 02:29
雷达观测 2026-11-09 13:22
摘要
This paper describes an approach and use case to estimate the correctness of LLM-generated COBOL summaries without access to human-authored references. The core intuition is that inter-model agreement on semantics is informative about truth, and that an LLM-as-judge can provide higher-fidelity pairwise assessments than traditional string- or embedding-based metrics. We operationalise this intuition with (i) controlled summary generation from diverse models; (ii) a LLM evaluator that compares pairs of summaries and emits structured judgments; and (iii) a calibrated aggregation rule that combines supportive evidence while penalising only corroborated contradictions.
作者备注：9 pages, 3 figures, 1 table