本文提出 ICE-G 框架,利用单张参考图像的分割与 DINO 语义对应,将颜色和纹理编辑传播到多视图并重新训练 3D 场景,实现高质量、细粒度可控的 Gaussian Splats 编辑。
摘要:
最近涌现出许多用于创建高质量 3D 资产和场景的技术。然而,当涉及编辑这些对象时,现有方法要么速度慢、在质量上妥协,要么无法提供足够的定制能力。我们提出了一种新颖方法,可从单个参考视图快速编辑 3D 模型。我们的技术首先对编辑图像进行分割,然后使用 DINO 特征在选定的、已分割的数据集视图之间匹配语义对应区域。随后,来自编辑图像特定区域的颜色或纹理变化能够以语义合理的方式自动应用到其他视图。这些经过编辑的视图充当更新后的数据集,用于进一步训练并重新风格化 3D 场景。因此,最终结果是一个经过编辑的 3D 模型。我们的框架支持多种编辑任务,例如手动局部编辑、基于任意示例图像中对应关系的风格迁移,以及来自多张示例图像的不同风格组合。由于 Gaussian Splats 具有速度快且便于局部编辑的特点,我们将其用作主要 3D 表示,但我们的技术也适用于 NeRF 等其他方法。我们通过多个示例表明,我们的方法能够产生更高质量的结果,同时提供细粒度的编辑控制。项目页面:ice-gaussian.github.io
英文摘要:
Recently many techniques have emerged to create high quality 3D assets and scenes. When it comes to editing of these objects, however, existing approaches are either slow, compromise on quality, or do not provide enough customization. We introduce a novel approach to quickly edit a 3D model from a single reference view. Our technique first segments the edit image, and then matches semantically corresponding regions across chosen segmented dataset views using DINO features. A color or texture change from a particular region of the edit image can then be applied to other views automatically in a semantically sensible manner. These edited views act as an updated dataset to further train and re-style the 3D scene. The end-result is therefore an edited 3D model. Our framework enables a wide variety of editing tasks such as manual local edits, correspondence based style transfer from any example image, and a combination of different styles from multiple example images. We use Gaussian Splats as our primary 3D representation due to their speed and ease of local editing, but our technique works for other methods such as NeRFs as well. We show through multiple examples that our method produces higher quality results while offering fine-grained control of editing. Project page: ice-gaussian.github.io
ArtisanGS: Interactive Tools for Gaussian Splat Selection with AI and Human in the Loop
NVIDIA;NVIDIA Canada(NVIDIA 加拿大);University of Toronto Canada(多伦多大学 加拿大);NVIDIA France(NVIDIA 法国);University of Toronto(多伦多大学);Institute for Clarity in Documentation Dublin Ohio USA(文档清晰研究所 俄亥俄州 大致);Inria Paris-Rocquencourt(法国巴黎-罗克琴特研究所);Rajiv Gandhi University Doimukh Arunachal Pradesh India(拉吉夫·甘地大学 亚当穆克 阿鲁纳恰尔邦 印度);Tsinghua University Haidian Qu Beijing Shi China(清华大学 海淀区 北京市 中国);Palmer Research Laboratories San Antonio Texas USA(帕勒研究中心 肯塔基州 威斯康星州 美国)
DisCo3D: Distilling Multi-View Consistency for 3D Scene Editing
Institute for Artificial Intelligence, Beijing National Research Center for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University(人工智能研究院、北京信息科学与技术国家研究中心、计算机科学与技术系、清华大学);College of Computer, National University of Defense Technology(计算机学院、国防科技大学)
DynamicAvatars: Accurate Dynamic Facial Avatars Reconstruction and Precise Editing with Diffusion Models
School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件学院);National Engineering Research Center for Visual Information and Applications, Xi’an Jiaotong University(西安交通大学视觉信息与应用国家工程研究中心);Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人工智能与机器人研究所);National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University(人机混合增强智能全国重点实验室)
Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editing
University of California San Diego(加利福尼亚大学圣迭戈分校);Massachusetts Institute of Technology(麻省理工学院);Institute for Artificial Intelligence and Fundamental Interactions(人工智能与基本相互作用研究所)
Realistic and Controllable 3D Gaussian-Guided Object Editing for Driving Video Generation
School of Vehicle and Mobility, and State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University(车辆与移动系统学院,智能绿色车辆与移动系统国家重点实验室,清华大学);WUWEN AI;PhiGent Robotics
3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting
The Chinese University of Hong Kong(香港中文大学);Stanford University(斯坦福大学);Snap Inc.(Snap公司);University of California Los Angeles(加利福尼亚大学洛杉矶分校);ByteDance(字节跳动)
AdLift: Lifting Adversarial Perturbations to Safeguard 3D Gaussian Splatting Assets Against Instruction-Driven Editing
Sydney AI Centre, The University of Sydney(悉尼人工智能中心,悉尼大学);University of Technology Sydney(技术大学悉尼);University of Melbourne(墨尔本大学);Hong Kong Baptist University(香港 Baptist 大学);Mohamed bin Zayed University of Artificial Intelligence(Mohamed bin Zayed 人工智能大学)
SVGS: Single-View to 3D Object Editing via Gaussian Splatting
School of Computer Science and Technology, Zhejiang Gongshang University(浙江工商大学计算机科学与技术学院);Zhejiang Key Laboratory of Big Data and Future E-Commerce Technology(浙江省大数据与未来电子商务技术重点实验室);Jianpei Technology Co., Ltd.(建培科技有限公司);School of Artificial Intelligence and Computer Science, Nantong University(南通大学人工智能与计算机科学学院);Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University(沙特本·阿卜杜勒阿齐兹王子大学计算机工程与科学学院计算机科学系);Department of Computer Science, Faculty of Computers and Information, Qena University(基纳大学计算机与信息学院计算机科学系);Department of Computing Sciences, Faculty of Information Technology and Communication Sciences, Tampere University(坦佩雷大学信息技术与通信科学学院计算科学系);Division of Oral and Maxillofacial Surgery, Faculty of Dentistry, The University of Hong Kong(香港大学牙医学院口腔颌面外科);Systems Research Institute of the Polish Academy of Sciences(波兰科学院系统研究所);AGH University of Krakow(AGH科技大学);SAN University(SAN大学)
InterGSEdit: Interactive 3D Gaussian Splatting Editing with 3D Geometry-Consistent Attention Prior
MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(信息产业部模式分析与机器智能重点实验室,计算机科学与技术学院,南京航空航天大学);The Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology(教育部高维信息智能感知与系统重点实验室,计算机科学与工程学院,南京理工大学)
View-Consistent 3D Editing with Gaussian Splatting
Nanyang Technological University(南洋理工大学);Institute for Infocomm Research, A*STAR(新加坡资讯通信研究院(新加坡科技研究局下属));Singapore University of Technology and Design(新加坡科技设计大学);Skywork AI(天工AI);Hong Kong University of Science and Technology(香港科技大学)
TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulation
School of Information Science and Technology, University of Science and Technology of China(中国科学技术大学信息科学技术学院);Space AI, Li Auto(理想汽车空间人工智能部门);Department of Automation, Tsinghua University(清华大学自动化系)
3DSceneEditor: Controllable 3D Scene Editing with Gaussian Splatting
D Optical Metrology unit, Fondazione Bruno Kessler(3D光学测绘单元,布鲁诺·凯斯勒基金会);Department of Information Engineering and Computer Science, University of Trento(信息工程与计算机科学系,特伦托大学)
Neural Surface Priors for Editable Gaussian Splatting
IDEAS NCBR;Adam Mickiewicz University(亚当·密茨凯维奇大学);Wrocław University of Science and Technology(弗罗茨瓦夫理工大学);Jagiellonian University(雅盖隆大学);Tooploox;New Jersey Institute of Technology(新泽西理工学院)
SketchFaceGS: Real-Time Sketch-Driven Face Editing and Generation with Gaussian Splatting
Shandong Technology and Business University(山东科技与商务大学);Nanchang Hangkong University(南昌航空大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);University of New South Wales(新南威尔士大学)
CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splatting
Institute for AI Industry Research (AIR), Tsinghua University(人工智能产业研究院(AIR),清华大学);Beijing Institute of Technology(北京理工大学);Nanyang Technological University(南洋理工大学);Tsinghua University(清华大学);Renmin University of China(中国人民大学);Beijing University of Technology(北京理工大学);Baidu Inc(百度公司);Peking University(北京大学);Shanghai AI Lab(上海人工智能实验室);Westlake University(西湖大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
提出 CRUISE 框架,利用分解式 Gaussian Splatting 重建并编辑 V2X 场景,实现多视角数据增强,提升协同 3D 检测、跟踪及边角案例生成能力。