← 研踪 PaperTrailarXiv:2608.07855 · v2
CommitKV: Commit-Aligned KV Cache Compression for Multimodal Agents
首次提交 2026-08-08 09:50
版本更新 2026-11-18 09:47
雷达观测 2026-11-19 19:08
摘要
Multimodal agents gather visual evidence through reasoning and tool use, extending their ability to solve tasks beyond the initial input. However, accumulating observations increases KV cache memory and inference costs. Token eviction reduces these costs by retaining useful cached tokens. Existing methods use attention, redundancy, or stage-specific queries to rank tokens, without explicitly comparing the same history across tool interactions. When observations shift attention, they may mistake temporarily reduced influence for lost usefulness and evict information needed later. We propose CommitKV, a training-free KV cache compression method that accounts for this changing evidence use. Commit-aligned measurement compares the deletion effects of the same page before a tool call completes and after its observation returns. Lifecycle-aware compression retires pages with low influence in both windows and temporarily protects pages influential before the call. Joint scoring combines attention importance, key redundancy, and token-level deletion effects to fill the remaining budget. Extensive experiments cover four open-weight multimodal models and eight benchmarks. CommitKV achieves the highest average accuracy among the compared compression methods on all four models. On Qwen3-VL-8B, it achieves 69.85% at a 512-token budget, only 0.91% below FullKV, and exceeds the strongest compression baseline by 8.44% at 256 tokens. Deploying the Qwen3-VL-8B agent with SGLang and vLLM at a 256-token budget reduces peak allocated GPU memory by 54.37%-55.23% and accelerates end-to-end inference by 2.12-2.17 over FullKV on the matched workload.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-18 09:47 · 最新观测版本
CommitKV: Commit-Aligned KV Cache Compression for Multimodal Agents
Multimodal agents gather visual evidence through reasoning and tool use, extending their ability to solve tasks beyond the initial input. However, accumulating observations increases KV cache memory and inference costs. Token eviction reduces these costs by retaining useful cached tokens. Existing methods use attention, redundancy, or stage-specific queries to rank tokens, without explicitly comparing the same history across tool interactions. When observations shift attention, they may mistake temporarily reduced influence for lost usefulness and evict information needed later. We propose CommitKV, a training-free KV cache compression method that accounts for this changing evidence use. Commit-aligned measurement compares the deletion effects of the same page before a tool call completes and after its observation returns. Lifecycle-aware compression retires pages with low influence in both windows and temporarily protects pages influential before the call. Joint scoring combines attention importance, key redundancy, and token-level deletion effects to fill the remaining budget. Extensive experiments cover four open-weight multimodal models and eight benchmarks. CommitKV achieves the highest average accuracy among the compared compression methods on all four models. On Qwen3-VL-8B, it achieves 69.85% at a 512-token budget, only 0.91% below FullKV, and exceeds the strongest compression baseline by 8.44% at 256 tokens. Deploying the Qwen3-VL-8B agent with SGLang and vLLM at a 256-token budget reduces peak allocated GPU memory by 54.37%-55.23% and accelerates end-to-end inference by 2.12-2.17 over FullKV on the matched workload.
查看此版本解读 →查看此版本原文 ↗