Using parts of existing models to rebuild new models, commonly termed as example-based modeling, is a classical methodology in the realm of computer graphics. Previous works mostly focus on shape composition, making them very hard to use for realistic composition of 3D objects captured from real-world scenes. This leads to combining multiple NeRFs into a single 3D scene to achieve seamless appearance blending. However, the current SeamlessNeRF method struggles to achieve interactive editing and harmonious stitching for real-world scenes due to its gradient-based strategy and grid-based representation. To this end, we present an example-based modeling method that combines multiple Gaussian fields in a point-based representation using sample-guided synthesis. Specifically, as for composition, we create a GUI to segment and transform multiple fields in real time, easily obtaining a semantically meaningful composition of models represented by 3D Gaussian Splatting (3DGS). For texture blending, due to the discrete and irregular nature of 3DGS, straightforwardly applying gradient propagation as SeamlssNeRF is not supported. Thus, a novel sampling-based cloning method is proposed to harmonize the blending while preserving the original rich texture and content. Our workflow consists of three steps: 1) real-time segmentation and transformation of a Gaussian model using a well-tailored GUI, 2) KNN analysis to identify boundary points in the intersecting area between the source and target models, and 3) two-phase optimization of the target model using sampling-based cloning and gradient constraints. Extensive experimental results validate that our approach significantly outperforms previous works in terms of realistic synthesis, demonstrating its practicality. More demos are available at https://ingra14m.github.io/gs_stitching_website.
Practical, Fast and Robust Point Cloud Registration for 3D Scene Stitching and Object Localization
East China University of Science and Technology(华东理工大学)
三维点云配准是遥感、摄影测量、机器人学和几何计算机视觉等领域中最基本的问题之一。由于3D特征匹配技术的有限准确性,在对应关系中可能存在离群值,有时甚至是非常大的数量。由于现有的鲁棒求解器可能会遇到高计算成本或有限的鲁棒性,我们提出了一种新的,快速和高度鲁棒的解决方案,命名为VOCRA(VOting与成本函数和旋转平均),极端离群率的点云配准问题。我们的第一个贡献是采用Tukey的Biwight鲁棒成本引入一个新的投票和通信排序技术,这被证明是相当有效的区分真正的内点从离群值,即使极端(99%)离群率。我们的第二个贡献包括在设计一个时间效率的共识最大化范例的基础上强大的旋转平均,服务于寻求内点候选人之间的对应关系。最后,我们应用Graduated Non-Convexity with Tukey's Bigweight(GNC-TB)来估计正确的变换与获得的内点候选,然后用于找到完整的内点集。两个标准的基准测试和现实的实验,应用到两个真实的数据问题进行,我们表明,我们的求解VOCRA是强大的对超过99%的离群值和更高效的时间比国家的最先进的竞争对手。
Multivariate Gaussian NeRF for Wide Field-of-View Ultrasound Reconstruction
Chair for Computer Aided Medical Procedures (CAMP), Technical University of Munich, Germany(计算机辅助医学程序系(CAMP),慕尼黑技术大学,德国);Munich Center for Machine Learning (MCML), Munich, Germany(慕尼黑机器学习中心,慕尼黑,德国);LUMA Vision, Beech Hill Road, Dublin, Ireland(LUMA视觉,Beech Hill Road,都柏林,爱尔兰)
MixedGaussianAvatar: Realistically and Geometrically Accurate Head Avatar via Mixed 2D-3D Gaussians
University of the Chinese Academy of Sciences(中国科学院大学);Peking University(北京大学);Nankai University(南开大学);Tsinghua University(清华大学);Intel Labs China(英特尔中国实验室)
Spatiotemporal Degradation-Aware 3D Gaussian Splatting for Realistic Underwater Scene Reconstruction
School of Astronautics Beihang University Beijing China(航天学院 北航 北京 中国);School of Artificial Intelligence Beihang University Beijing China(人工智能学院 北航 北京 中国);Zhongguancun Academy Beijing China(中关村学院 北京 中国);Beihang University School of Astronautics Beijing China(北航 航天学院 北京 中国);Beihang University(北航);Zhongguancun Academy(中关村学院)
You Need a Transition Plane: Bridging Continuous Panoramic 3D Reconstruction with Perspective Gaussian Splatting
Institute of Information Science, Beijing Jiaotong University(北京交通大学信息科学研究所);Visual Intelligence +X International Cooperation Joint Laboratory of MOE(教育部视觉智能+X国际合作联合实验室);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
RePaintGS: Reference-Guided Gaussian Splatting for Realistic and View-Consistent 3D Scene Inpainting
Intelligence and Interaction Research Center, Korea Institute of Science and Technology(智能交互研究所,韩国科学技术院);Department of Computer Science and Engineering, Korea University(韩国大学计算机科学与工程系);AI-Robotics, KIST School, Korea National University of Science and Technology(人工智能机器人,KIST学院,韩国科学技术大学)
GaussianFluent: Gaussian Simulation for Dynamic Scenes with Mixed Materials
State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室);State Key Laboratory of General Artificial Intelligence, BIGAI(北京大学通用人工智能国家重点实验室,BIGAI)
HRAvatar: High-Quality and Relightable Gaussian Head Avatar
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院,清华大学);International Digital Economy Academy (IDEA) Intern(国际数字经济发展学院(IDEA)实习生)
GaRField++: Reinforced Gaussian Radiance Fields for Large-Scale 3D Scene Reconstruction
School of Information Science and Engineering, Yunnan University(云南大学信息科学与工程学院);Department of Electrical and Computer Engineering, Clarkson University(克拉克森大学电气与计算机工程系);Yunnan Key Laboratory of Intelligent Systems and Computing, Yunnan University(云南大学云南省智能系统与计算重点实验室)
HaloGS: Loose Coupling of Compact Geometry and Gaussian Splats for 3D Scenes
Zhejiang University(浙江大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);The Chinese University of Hong Kong(香港中文大学);Inria(法国国家信息与自动化技术研究院);The University of Hong Kong(香港大学)
SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Prometheus Vision Technology Co., Ltd.(天演智视科技有限公司);The Hong Kong University of Science and Technology(香港科技大学)
Topology-aware Human Avatars with Semantically-guided Gaussian Splatting
School of Computer Science, Wuhan University(武汉大学计算机学院);School of Software Engineering, Xi’an Jiao Tong University(西安交通大学软件学院);AI Institute, Shanghai Jiao Tong University(上海交通大学人工智能研究院);Huazhong University of Science and Technology(华中科技大学)
Hybrid Explicit Representation for Ultra-Realistic Head Avatars
University of Science and Technology of China(中国科学技术大学);ShanghaiTech University(上海科技大学);Xi’an Jiaotong University(西安交通大学);Chinese Academy of Sciences(中国科学院);University of Hong Kong(香港大学)
GSwap: Realistic Head Swapping with Dynamic Neural Gaussian Field
School of Mathematical Science, University of Science and Technology of China(中国科学技术大学数学科学学院);Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系)