基于集成的LLM代码摘要无参考置信度估计方法
Ensemble Methodology for Reference-Free Confidence Estimation in LLM-based Code Summarization
Marco Colombo, Balaji A Jayaram, Sowmyakumari Selvaraj, Sumit Tokkar, Animesh Kumar, Aravinda Prasad Sistla, Ravishankar Shetty, Rigel Gjomemo
AI 速读 基于摘要
提出一种无需人工参考摘要、通过多模型语义一致性与LLM评判器聚合来估计LLM生成COBOL摘要正确性的集成方法。
cs.AI · cs.PL
查看原文摘要
This paper describes an approach and use case to estimate the correctness of LLM-generated COBOL summaries without access to human-authored references. The core intuition is that inter-model agreement on semantics is informative about truth, and that an LLM-as-judge can provide higher-fidelity pairwise assessments than traditional string- or embedding-based metrics. We operationalise this intuition with (i) controlled summary generation from diverse models; (ii) a LLM evaluator that compares pairs of summaries and emits structured judgments; and (iii) a calibrated aggregation rule that combines supportive evidence while penalising only corroborated contradictions.