arXiv:2604.02923 · v7

An Inspectable LLM Council for Multi-Model Research Answer Aggregation

首次提交 2026-04-03 17:40

版本更新 2026-11-07 19:16

雷达观测 2026-11-10 11:14

摘要

Long-form research questions often require answers that combine complementary details, reconcile conflicting estimates, and retain qualifications across several large language model outputs. We present Standard Council, a two-stage synthesis workflow in which an analyst organizes three independent candidate answers into a structured textual state and a separate writer composes the final response from the question and that state. We evaluate the workflow on 100 closed-book tasks from the Deep Research Accuracy, Completeness, and Objectivity benchmark, DRACO, retaining 600 final answers, 100 analyst states, and 720 whole-answer judgments. Under the primary rubric judge, Council scores 70.98, exceeding GPT-5.6 Sol by 6.18 points and Gemini 3.8 Flash by 8.34 points; its 0.47-point difference from Claude Opus 5 has a 95% paired interval of $[-0.69,1.61]$. In the four-arm fusion comparison, Council scores 1.87 points above direct fusion, with a 95% paired interval of $[0.33,3.48]$. It covers 51.1% of positive criteria met by exactly one candidate, compared with 43.7% for direct fusion. On an exploratory 68-task subset with matching returned model labels, the Council-minus-direct-fusion difference is $-1.69$ points, with a 95% interval of $[-2.80,-0.62]$. The full comparison characterizes workflows that differ in structure and generation budget; estimating the analyst stage's effect requires a controlled component study. Traced cases show how selected estimates, provenance qualifications, and selection decisions pass through the saved state into final answers.

作者备注：26 pages, 8 figures, 13 tables