UniMGS: Unifying Mesh and 3D Gaussian Splatting with Single-Pass Rasterization and Proxy-Based Deformation
College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);Fysics Intelligence Technologies Co., Ltd.(菲视智能科技有限公司);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学)
Joint rendering and deformation of mesh and 3D Gaussian Splatting (3DGS) have significant value as both representa tions offer complementary advantages for graphics applica tions. However, due to differences in representation and ren dering pipelines, existing studies render meshes and 3DGS separately, making it difficult to accurately handle occlusions and transparency. Moreover, the deformed 3DGS still suffers from visual artifacts due to the sensitivity to the topology quality of the proxy mesh. These issues pose serious obsta cles to the joint use of 3DGS and meshes, making it diffi cult to adapt 3DGS to conventional mesh-oriented graphics pipelines. We propose UniMGS, the first unified framework for rasterizing mesh and 3DGS in a single-pass anti-aliased manner, with a novel binding strategy for 3DGS deformation based on proxy mesh. Our key insight is to blend the col ors of both triangle and Gaussian fragments by anti-aliased α-blending in a single pass, achieving visually coherent re sults with precise handling of occlusion and transparency. To improve the visual appearance of the deformed 3DGS, our Gaussian-centric binding strategy employs a proxy mesh and spatially associates Gaussians with the mesh faces, signifi cantly reducing rendering artifacts. With these two compo nents, UniMGS enables the visualization and manipulation of 3D objects represented by mesh or 3DGS within a unified framework, opening up new possibilities in embodied AI, vir tual reality, and gaming. We will release our source code to facilitate future research.
Accelerating 3D Gaussian Splatting using Tensor Cores
University of Pittsburgh(匹兹堡大学);Rice University(莱斯大学);New York University(纽约大学);Shanghai Jiao Tong University(上海交通大学);Rutgers University(罗格斯大学);Illinois Institute of Technology(伊利诺伊理工学院)
StochasticSplats: Stochastic Rasterization for Sorting-Free 3D Gaussian Splatting
Google DeepMind(谷歌DeepMind);University of British Columbia(不列颠哥伦比亚大学);Google(谷歌);Runway ML;Simon Fraser University(西蒙菲莎大学);University of Toronto(多伦多大学)
NVGS: Neural Visibility for Occlusion Culling in 3D Gaussian Splatting
Digital Future Lab, Flanders Make, Hasselt University(数字未来实验室,弗拉芒制造,哈塞尔特大学);Computer Graphics Lab, TU Braunschweig(计算机图形学实验室,图林根工业大学);Physics and Astronomy, University of New Mexico(物理与天文学,新墨西哥大学)
Chongqing University(重庆大学);MAIS, Institute of Automation, Chinese Academy of Sciences and UCAS(自动化研究所,中国科学院,UCAS);King Abdullah University of Science and Technology (KAUST)(卡塔尔科学与技术大学)
MeshSuite: A Unified Benchmark for Triangle- and Mesh-Based Neural Rendering
Nanjing University of Science and Technology(南京理工大学);State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery(先进工程机械智能制造国家重点实验室);University of Surrey(萨里大学)
A Hybrid and Hierarchical Ray Tracing Method for Efficient 3D Gaussian Rendering
Hong Kong University of Science and Technology(香港科技大学);Georgia Institute of Technology(佐治亚理工学院);Shanghai Jiao Tong University(上海交通大学);Peking University(北京大学)
Hybrid Explicit Representation for Ultra-Realistic Head Avatars
University of Science and Technology of China(中国科学技术大学);ShanghaiTech University(上海科技大学);Xi’an Jiaotong University(西安交通大学);Chinese Academy of Sciences(中国科学院);University of Hong Kong(香港大学)