We present WonderWorld, a novel framework for interactive 3D scene generation that enables users to interactively specify scene contents and layout and see the created scenes in low latency. The major challenge lies in achieving fast generation of 3D scenes. Existing scene generation approaches fall short of speed as they often require (1) progressively generating many views and depth maps, and (2) time-consuming optimization of the scene geometry representations. We introduce the Fast Layered Gaussian Surfels (FLAGS) as our scene representation and an algorithm to generate it from a single view. Our approach does not need multiple views, and it leverages a geometry-based initialization that significantly reduces optimization time. Another challenge is generating coherent geometry that allows all scenes to be connected. We introduce the guided depth diffusion that allows partial conditioning of depth estimation. WonderWorld generates connected and diverse 3D scenes in less than 10 seconds on a single A6000 GPU, enabling real-time user interaction and exploration. We demonstrate the potential of WonderWorld for user-driven content creation and exploration in virtual environments. We release full code and software for reproducibility. Project website: https://kovenyu.com/WonderWorld/.
ϕ-RIE: From Photorealistic Reconstruction to Interactive Environments
INSAIT, Sofia University “St. Kliment Ohridski”(索非亚大学圣克利门特奥赫里德大学INSAIT研究所);vivo Mobile Communication Co., Ltd.(维沃移动通信有限公司);Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院)
ASSIST: Interactive Scene Nodes for Scalable and Realistic Indoor Simulation
Institute for AI Industry Research (AIR), Tsinghua University(清华大学智能产业研究院(AIR));Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Beijing University of Civil Engineering and Architecture(北京建筑大学);University of Science and Technology of China(中国科学技术大学);McGill University(麦吉尔大学);Beijing Institute(北京研究所)
Chat-Edit-3D++: Interactive 3D and 4D Scene Editing via Large Language Models
School of Electrical and Information Engineering, Beihang University(北京航空航天大学电子信息工程学院);Institute of Unmanned System, Beihang University(北京航空航天大学无人系统研究院);Atmanity Inc.(Atmanity公司);Megvii Research(旷视研究院);University of California at Merced(加州大学默塞德分校)
Seg-Wild: Interactive Segmentation based on 3D Gaussian Splatting for Unconstrained Image Collections
College of Computer Science and Engineering, Shandong University of Science and Technology(山东科技大学计算机科学与工程学院);State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室)
提出基于3D Gaussian Splatting的交互式分割方法Seg-Wild,通过多维特征嵌入和Spiky 3D Gaussian Cutter处理无约束图像中的瞬态遮挡,实现更优的野外场景分割与重建质量。
3DSceneEditor: Controllable 3D Scene Editing with Gaussian Splatting
D Optical Metrology unit, Fondazione Bruno Kessler(3D光学测绘单元,布鲁诺·凯斯勒基金会);Department of Information Engineering and Computer Science, University of Trento(信息工程与计算机科学系,特伦托大学)
Image-Driven Furniture Style for Interactive 3D Scene Modeling
New Jersey Institute of Technology(新泽西理工学院);Northeastern University(东北大学);University of California, Irvine(加州大学欧文分校);Facebook Inc.(脸书公司);Wayfair Inc.(维费尔公司)
Scene-Generalizable Interactive Segmentation of Radiance Fields
Harbin Institute of Technology(哈尔滨工业大学);Y-tech, Kuaishou Technology(快手科技Y-tech);Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies(广东省新型安全智能技术重点实验室);Dartmouth College(达特茅斯学院)
DragScene: Interactive 3D Scene Editing with Single-view Drag Instructions
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);College of Artificial Intelligence, Xi’an Jiaotong University(西安交通大学人工智能学院);School of Software, Beihang University(北京航空航天大学软件学院);The University of Texas at Austin(德克萨斯大学奥斯汀分校)
SceneCraft: Interactive System for Image Editing via Scene Graph
University of Science, Ho Chi Minh, Vietnam(胡志明市理科大学);Vietnam National University, Ho Chi Minh, Vietnam(越南国家大学胡志明市);University of Dayton, Dayton, Ohio, USA(代顿大学)
LightFuse: Relightable Interactive Gaussian Scene Reconstruction via Multi-Scan Fusion and 2D Gaussian Ray Tracing
ZJU-UIUC Institute, Zhejiang University(浙江大学ZJU-UIUC学院);College of Mathematics, Sichuan University(四川大学数学学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Chengdu Minto Tech(成都敏途科技)
GraphiContact: Pose-aware Human-Scene Robust Contact Perception for Interactive Systems
Tsinghua University, China(清华大学, 中国);XR System Application Research Center, Nanchang Research Institute, Zhejiang University, China(浙江大学南昌研究院XR系统应用研究中心, 中国);Beijing University of Posts and Telecommunications, China(北京邮电大学, 中国);University of Chinese Academy of Sciences, China(中国科学院大学, 中国);National University of Singapore, Singapore(新加坡国立大学, 新加坡)
PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation
Sun Yat-sen University, China(中山大学);Peng Cheng Laboratory(鹏城实验室);Guangdong Key Laboratory of Big Data Analysis and Processing(广东省大数据分析与处理重点实验室);Huawei(华为)
Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China(中国科学院人工智能安全国家重点实验室,计算技术研究所,北京,中国);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学,北京,中国);China University of Mining & Technology, Beijing(中国矿业大学(北京));ETH Zürich(苏黎世联邦理工学院);City College of New York, City University of New York(纽约城市学院,纽约城市大学);Xiamen Institute of Data Intelligence, Xiamen, China(厦门数据智能研究院,厦门,中国)
MagicWorld: Towards Long-Horizon Stability for Interactive Video World Exploration
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);vivo BlueImage Lab, vivo Mobile Communication Co., Ltd.(vivo蓝影实验室,vivo移动通信有限公司);National University of Singapore(新加坡国立大学)
Social 3D Scene Graphs: Modeling Human Actions and Relations for Interactive Service Robots
Faculty of Robotics Perception and Learning, KTH Royal Institute of Technology(机器人感知与学习学院,皇家理工学院);Institute for Artificial Intelligence, University of Stuttgart(人工智能研究所,斯图加特大学);International Max Planck Research School for Intelligent Systems (IMPRS-IS)(智能系统国际马克斯·普朗克研究学校)