arXiv:2503.00868 · v3
Physics-Driven Extraction of Dynamic Fluid Asset from Monocular Video with Generative 3DGS
摘要
Existing methods for dynamic 3D scene synthesis from video rely on per-frame deformation fields optimized through pixel-level reconstruction losses. While effective for closed systems such as elastic solids, these approaches face challenges with open systems like fluids where material continuously enters and exits the scene, topology changes freely, and no stable point correspondence exists across frames. We propose a simulation-driven alternative that optimizes the parameters of a fluid simulator to reproduce dynamics observed in a single-view video, rather than fitting per-point deformation to pixel values. Our framework leverages generative 3D Gaussian Splatting (3DGS) models to obtain initial fluid geometry including unseen surfaces, estimates a physically grounded velocity field from surface motions with a physics-constraint correction algorithm, and converts it into a divergence-free, grid-based representation. A differentiable Affine Particle-In-Cell (APIC) simulation then optimizes effective simulation parameters, such as viscosity, boundary conditions, and inflow/outflow velocities, to match the observed dynamics. Because the dynamics are governed by forward simulation, the output assets are inherently editable, interactive, and temporally extendable, which reconstruction-based approaches cannot readily provide. Stage-wise ablations confirm that each stage is necessary, and hyperparameter sweeps show the pipeline to be robust without sensitive per-scene tuning. Our method handles various liquid fluids, including inviscid and viscous types, demonstrating the potential for automatically creating simulation-ready 3D fluid assets from readily available internet videos.
作者备注：Accepted to IEEE Transactions on Visualization and Computer Graphics. 21 pages (18 pages + 3 pages supplementary material), 28 figures