← 研踪 PaperTrailarXiv:2607.14386 · v2
CIPHER: A Decoupled Exploration-Selection Framework for Test-Time Scaling of Data Science Agents
首次提交 2026-07-16 06:01
版本更新 2026-10-27 00:15
雷达观测 2026-10-29 00:35
摘要
Enterprise data science and analytics spans closed-ended information extraction with verifiable answers and open-ended insight discovery. Agents are evaluated on one or the other, so there is no evidence on whether one system can serve both. We present \texttt{CIPHER}, a data science agent built on a Decoupled Exploration--Selection (DES) framework that applies test-time scaling at the plan level: generating $N$ candidate analysis plans, selecting $M{<}N$ for parallel execution, and aggregating the results. We evaluate $48$ distinct design choices for the DES framework (spanning generation, selection, budget and aggregator capacity) jointly on $357$ tasks from Infi-DA-Bench (closed-ended) and InsightBench (open-ended), on a proprietary and an open-weights base model. A single zero-tuning configuration is competitive against purpose-built specialist baselines under matched models, within $1.2$pp of \texttt{DataWise} and $1.3$pp of \texttt{Agent-Poirot}. The joint sweep exposes that generation and aggregation are the most influential performance levers, while random selection --- the only strategy with zero overhead --- matches its tuned alternatives. These results offer new insights on the design of test-time scaling strategies, and on how a fixed inference budget is best allocated across them, for agents deployed on enterprise analytics workloads.
作者备注：Presented at the 2nd Workshop on Agentic AI Benchmarks and Applications for Enterprise Tasks (NeurIPS 2026)
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-10-27 00:15 · 最新观测版本
CIPHER: A Decoupled Exploration-Selection Framework for Test-Time Scaling of Data Science Agents
Enterprise data science and analytics spans closed-ended information extraction with verifiable answers and open-ended insight discovery. Agents are evaluated on one or the other, so there is no evidence on whether one system can serve both. We present \texttt{CIPHER}, a data science agent built on a Decoupled Exploration--Selection (DES) framework that applies test-time scaling at the plan level: generating $N$ candidate analysis plans, selecting $M{<}N$ for parallel execution, and aggregating the results. We evaluate $48$ distinct design choices for the DES framework (spanning generation, selection, budget and aggregator capacity) jointly on $357$ tasks from Infi-DA-Bench (closed-ended) and InsightBench (open-ended), on a proprietary and an open-weights base model. A single zero-tuning configuration is competitive against purpose-built specialist baselines under matched models, within $1.2$pp of \texttt{DataWise} and $1.3$pp of \texttt{Agent-Poirot}. The joint sweep exposes that generation and aggregation are the most influential performance levers, while random selection --- the only strategy with zero overhead --- matches its tuned alternatives. These results offer new insights on the design of test-time scaling strategies, and on how a fixed inference budget is best allocated across them, for agents deployed on enterprise analytics workloads.
查看此版本解读 →查看此版本原文 ↗