GaussFluids: Reconstructing Lagrangian Fluid Particles from Videos via Gaussian Splatting

Fluid simulation typically depends on manual modeling and visual assessment to achieve desired outcomes, which lacks objectivity and efficiency. To address this limitation, we propose GaussFluids, a novel approach for directly reconstructing temporally and spatially continuous Lagrangian fluid particles from videos. We employ a Lagrangian particle-based method instead of an Eulerian grid as it provides a direct spatial mass representation and is more suitable for capturing fine fluid details. First, to make discrete fluid particles differentiable over time and space, we extend Lagrangian particles with Gaussian probability densities, termed Gaussian Particles, constructing a differentiable fluid particle renderer that enables direct optimization of particle positions from visual data. Second, we introduce a fixed-length transform feature for each Gaussian Particle to encode pose changes over continuous time. Next, to preserve fundamental fluid physics—particularly incompressibility—we incorporate a density-based soft constraint to guide particle distribution within the fluid. Furthermore, we propose a hybrid loss function that focuses on maintaining visual, physical, and geometric consistency, along with an improved density optimization module to efficiently reconstruct spatiotemporally continuous fluids. We demonstrate the effectiveness of GaussFluids on multiple synthetic and real-world datasets, showing its capability to accurately reconstruct temporally and spatially continuous, physically plausible Lagrangian fluid particles from videos. Additionally, we introduce several downstream tasks, including novel view synthesis, style transfer, frame interpolation, fluid prediction, and fluid editing, which illustrate the practical value of GaussFluids.