跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2608.28008 · v2

Visual Token Coding for Video Multimodal Large Language Models

首次提交 2026-08-28 15:21
版本更新 2026-11-18 21:20
雷达观测 2026-11-19 15:04
arXiv ↗PDF ↗

摘要

In this paper, we propose a new token compression paradigm for video Multimodal Large Language Models (MLLMs), termed Visual Token Coding (VTCVTC). Inspired by classical video coding principles, e.g., HEVC, VTC performs structured compression by predicting the I/P frames of a video and measuring their frame-wise residuals to estimate token redundancy. Based on this base paradigm, we further extend VTC with a set of dynamic budgeting designs towards more cost-effective token coding, termed VTCdyVTC_{dy}. To validate VTC and VTCdyVTC_{dy}, we apply them to three MLLMs and conduct experiments on a set of video understanding benchmarks. The experimental results show that as a base paradigm, VTC can achieve an average performance retention of 98.4% on Qwen3-VL under a 50% token compression ratio, and the enhanced VTCdyVTC_{dy} can further improve the retention to 100.3%, outperforming a set of compared methods. Moreover, as a plug-and-play design, VTC requires no additional tuning of MLLMs for token coding. Our code is given in the supplementary materials.

作者备注:11 pages, 5 figures

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-18 21:20 · 最新观测版本

Visual Token Coding for Video Multimodal Large Language Models

In this paper, we propose a new token compression paradigm for video Multimodal Large Language Models (MLLMs), termed Visual Token Coding (VTCVTC). Inspired by classical video coding principles, e.g., HEVC, VTC performs structured compression by predicting the I/P frames of a video and measuring their frame-wise residuals to estimate token redundancy. Based on this base paradigm, we further extend VTC with a set of dynamic budgeting designs towards more cost-effective token coding, termed VTCdyVTC_{dy}. To validate VTC and VTCdyVTC_{dy}, we apply them to three MLLMs and conduct experiments on a set of video understanding benchmarks. The experimental results show that as a base paradigm, VTC can achieve an average performance retention of 98.4% on Qwen3-VL under a 50% token compression ratio, and the enhanced VTCdyVTC_{dy} can further improve the retention to 100.3%, outperforming a set of compared methods. Moreover, as a plug-and-play design, VTC requires no additional tuning of MLLMs for token coding. Our code is given in the supplementary materials.

查看此版本解读 →查看此版本原文 ↗