CIPHER: A Decoupled Exploration-Selection Framework for Test-Time Scaling of Data Science Agents
Maxime Heuillet, Sharadind Peddiraju
原文摘要
Enterprise data science and analytics spans closed-ended information extraction with verifiable answers and open-ended insight discovery. Agents are evaluated on one or the other, so there is no evidence on whether one system can serve both. We present \texttt{CIPHER}, a data science agent built on a Decoupled Exploration--Selection (DES) framework that applies test-time scaling at the plan level: generating $N$ candidate analysis plans, selecting $M{<}N$ for parallel execution, and aggregating the results. We evaluate $48$ distinct design choices for the DES framework (spanning generation, selection, budget and aggregator capacity) jointly on $357$ tasks from Infi-DA-Bench (closed-ended) and InsightBench (open-ended), on a proprietary and an open-weights base model. A single zero-tuning configuration is competitive against purpose-built specialist baselines under matched models, within $1.2$pp of \texttt{DataWise} and $1.3$pp of \texttt{Agent-Poirot}. The joint sweep exposes that generation and aggregation are the most influential performance levers, while random selection --- the only strategy with zero overhead --- matches its tuned alternatives. These results offer new insights on the design of test-time scaling strategies, and on how a fixed inference budget is best allocated across them, for agents deployed on enterprise analytics workloads.
cs.AI