跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2609.33100 · v3

Octree-based Video Representation

首次提交 2026-09-27 10:14
版本更新 2026-11-17 22:14
雷达观测 2026-11-18 19:41
arXiv ↗PDF ↗

摘要

Representing videos on uniform spatio-temporal grids produces large token sequences and substantial computational costs. Yet visual complexity is unevenly distributed: smooth regions can be represented compactly, while detailed regions require finer resolution. We introduce OctVideo, an octree-based video representation that adapts cell size to local approximation error. Each leaf stores local RGB values and spatio-temporal gradients, supporting analytic reconstruction of the video. A lightweight learned residual enriches these descriptors for neural processing. For reconstruction, a Conv1D VAE maps the serialized cells to a regular latent grid and selectively refines details during decoding. Our VAE achieves 36.12 dB PSNR with 38.2M parameters and 188.9 GFLOPs per clip on Kinetics-400 (K400). It also generalizes zero-shot to the high-resolution Densely Annotated VIdeo Segmentation (DAVIS) 2016 dataset with reconstruction quality comparable to the best evaluated models. On both datasets, it requires the fewest model FLOPs and achieves the fastest reconstruction among the evaluated models. OctVideo also supports video understanding, achieving competitive recognition performance with few input tokens when trained from scratch. By exploiting the redundancy already present in video signals and efficiently processing sparse structures, OctVideo provides an efficient representation for video.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v3,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v3 · 2026-11-17 22:14 · 最新观测版本

Octree-based Video Representation

Representing videos on uniform spatio-temporal grids produces large token sequences and substantial computational costs. Yet visual complexity is unevenly distributed: smooth regions can be represented compactly, while detailed regions require finer resolution. We introduce OctVideo, an octree-based video representation that adapts cell size to local approximation error. Each leaf stores local RGB values and spatio-temporal gradients, supporting analytic reconstruction of the video. A lightweight learned residual enriches these descriptors for neural processing. For reconstruction, a Conv1D VAE maps the serialized cells to a regular latent grid and selectively refines details during decoding. Our VAE achieves 36.12 dB PSNR with 38.2M parameters and 188.9 GFLOPs per clip on Kinetics-400 (K400). It also generalizes zero-shot to the high-resolution Densely Annotated VIdeo Segmentation (DAVIS) 2016 dataset with reconstruction quality comparable to the best evaluated models. On both datasets, it requires the fewest model FLOPs and achieves the fastest reconstruction among the evaluated models. OctVideo also supports video understanding, achieving competitive recognition performance with few input tokens when trained from scratch. By exploiting the redundancy already present in video signals and efficiently processing sparse structures, OctVideo provides an efficient representation for video.

查看此版本解读 →查看此版本原文 ↗