Jongseok Park, Jiaxiang Yu, Eric Zhou, Tony Hong, Sunga Kim, Yusuf Afifi, Ian Wong, Ion Stoica 等 9 位作者
原文摘要
AI agent harnesses play the critical role of providing LLMs with an interface to a computing environment. Recent studies show that AI agents' task-execution accuracy depends mostly on the LLM used, while harness-side variances such as system prompts or tools have minimal impact. Rather, such harness-side variances show a larger impact on the cost of agent executions, achieving cost reductions up to 2.6 times on the same model and task. To further explore these harness-side cost optimization opportunities, we analyze the cost accumulation patterns across various agent traces and identify system opportunities to reduce token use and LLM calls without modifying the model or affecting task accuracy. Based on this analysis, we design Agency, a harness interposition platform that provides a unified abstraction and allows easy and portable implementation of harness-side system optimizations on existing off-the-shelf harnesses such as Claude Code or OpenAI Codex. Using Agency, we implement proof-of-concept optimization techniques based on our analysis and achieve cost reductions up to 35% across various real-world benchmarks on existing harnesses without compromising accuracy.
Li Shi, Houjiang Liu, Yian Wong, Utkarsh Mujumdar, Dan Zhang, Jacek Gwizdka, Matthew Lease
原文摘要
Multi-persona debate systems powered by large language models (LLMs) show promise in reducing confirmation bias; however, empirical evidence remains limited. Through a within-subjects experiment, we compared an LLM-generated multi-persona debate system with an opposing-stances retrieval system. We evaluated system effectiveness in mitigating confirmation bias over 312 study trials across 8 controversial topics, using behavioral measures (changes in eye-fixation patterns) and attitudinal measures (shifts between pre/post-intervention beliefs), with topic familiarity and cognitive tendencies as moderators. Our results show that: (i) participants allocated more than half of their visual attention to attitude-inconsistent content in both systems, with no significant between-system differences; (ii) the debate system showed a suggestive but statistically non-significant buffering pattern among participants with stronger cognitive tendencies, and topic familiarity did not significantly moderate its effect on visual attention; and (iii) reading effort and perceived argument quality influenced participants' engagement with attitude-inconsistent content. These findings suggest that multi-persona debate systems can support engagement with diverse viewpoints, while their comparative advantage over opposing-stances retrieval was not established; individual cognitive tendencies remained an important predictor of visual attention. We provide further implications for designing multi-persona debates to better mitigate confirmation bias, focusing on balancing facts and persuasion and on deciding when to explore diverse viewpoints versus offer concrete solutions.