GenRA: Ultra-Low-Bitrate Generative Video Compression via Representation Adaptation
Jiancheng Zhao, Jixiang Luo, Haibin Huang, Chi Zhang, Xuelong Li
原文摘要
Preserving fine visual details in ultra-low-bitrate video compression remains challenging due to the loss of critical information. Recent generative video compression methods exploit generative priors to improve perceptual quality. These pretrained generative representations are generally not optimized for compression. Some approaches therefore use an additional learned codec to map them into a separate latent space for compression. To this end, we propose GenRA, which adapts pretrained video representations for direct quantization and entropy coding without an additional learned codec. Specifically, we use vector quantization to adapt a pretrained video VAE into a discrete video tokenizer. A lightweight categorical autoregressive entropy model combines local and global context to model the probabilities of the resulting token indices. We jointly optimize the tokenizer and entropy model using a differentiable rate surrogate, allowing the encoder to adjust token assignments based on coding cost and reconstruction quality. Experiments on UVG, MCL-JCV, and HEVC Class B show average bitrate savings of 84.1% and 82.3% over GLC-Video in terms of LPIPS and DISTS, respectively.
cs.CV