From Chats to Markets: AgenticPay for LLM-Powered Negotiation in Multi-Agent Commerce
Xianyang Liu, Shangding Gu, Fan Xu, Manxi Wu, Boyi Li, Jun Wang, Costas Spanos, Dawn Song
原文摘要
Large language model agents are increasingly expected to negotiate and transact autonomously, yet existing benchmarks rarely combine multimodal product grounding, multi-issue bargaining, and competitive multi-agent markets. We introduce AgenticPay, a unified benchmark for evaluating multimodal agents' ability to reach high-welfare, multidimensional agreements in realistic markets. AgenticPay includes 160 multimodal tasks across four business scenarios (E-commerce, Food Delivery, Ride-hailing, and Apartment Rental) and eight market topologies, from bilateral bargaining to many-to-many competition. Agents interpret product images, infer opponents' hidden preferences, and negotiate binding contract terms such as price, lease duration, return policy, and delivery speed through multi-round natural-language dialogue. A utility-based framework evaluates agreement quality and joint welfare. Experiments with proprietary (GPT-5.4, Claude Sonnet 4.6, Gemini 3.1 Pro Preview) and open-weight (Qwen3-VL-32B-Instruct, InternVL3-38B) multimodal models reveal substantial limitations in value creation and market reasoning. Even the strongest model, Gemini 3.1 Pro Preview, achieves a GlobalScore of only 42.3/100. As markets shift from bilateral to multi-sided, average GlobalScore falls by 5.7 points, with open-weight models declining by up to 11.5 points. AgenticPay provides a testbed for evaluating multimodal agents in realistic commerce. Code and data: https://github.com/SAIL-Research-Lab/AgenticPay.
cs.AI · cs.LG