arXiv:2501.12935 · v2
OMINI: Towards Consistent 3D Object Manipulation via Generative Texturing and Adaptive Lighting
摘要
Enabling object manipulation based on a single static image, from geometric editing to animation, remains a significant challenge. Existing methods often struggle to jointly support these tasks, while also suffering from fidelity degradation in object appearance and scene illumination. In this work, we introduce OMINI, a unified framework that addresses these challenges through two complementary strategies. First, OMINI lifts the 2D object into a 3D representation and adopts a Refine-before-Render design. Both geometric editing and animation are then performed by manipulating and rendering the refined 3D asset, maintaining appearance consistency throughout the manipulation process. Second, we propose two dedicated modules for fidelity enhancement: TextuRefiner refines rendered views of the 3D asset under appearance and geometric guidance, thereby generating fine-grained texture details; IllumiCorrector corrects environmental illumination using HSL-based color correction and VLM-guided intensity optimization, enabling visually plausible renderings with shadows. Extensive experiments demonstrate that OMINI outperforms existing methods, producing visually consistent results. The GPU stages of OMINI are run on a single NVIDIA RTX 3090.
作者备注:Accepted for publication in IEEE Transactions on Image Processing (TIP). The first version is submitted to OpenReview in September 2024: https://openreview.net/forum?id=xxzukMsYs9