Confronting the challenges of data scarcity and advanced motion synthesis in human-scene interaction modeling, we introduce the TRUMANS dataset alongside a novel HSI motion synthesis method. TRUMANS stands as the most comprehensive motion-captured HSI dataset currently available, encompassing over 15 hours of human interactions across 100 indoor scenes. It intricately captures whole-body human motions and part-level object dynamics, focusing on the realism of contact. This dataset is further scaled up by transforming physical environments into exact virtual models and applying extensive augmentations to appearance and motion for both humans and objects while maintaining interaction fidelity. Utilizing TRUMANS, we devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length, taking into account both scene context and intended actions. In experiments, our approach shows remarkable zero-shot generalizability on a range of 3D scene datasets (e.g., PROX, Replica, ScanNet, ScanNet++), producing motions that closely mimic original motion-captured sequences, as confirmed by quantitative experiments and human studies.
Inter-object Discriminative Graph Modeling for Indoor Scene Recognition
Shandong University(山东大学);School of Control Science and Engineering(控制科学与工程学院);Center for Robotics(机器人中心);Engineering Research Center of Intelligent Unmanned System(智能无人系统工程研究中心)
Social 3D Scene Graphs: Modeling Human Actions and Relations for Interactive Service Robots
Faculty of Robotics Perception and Learning, KTH Royal Institute of Technology(机器人感知与学习学院,皇家理工学院);Institute for Artificial Intelligence, University of Stuttgart(人工智能研究所,斯图加特大学);International Max Planck Research School for Intelligent Systems (IMPRS-IS)(智能系统国际马克斯·普朗克研究学校)
我们介绍了一种方法来模型的表面属性在日常场景中的反弹。我们的模型从传感器输入开始进行端到端学习,以预测反弹后的轨迹,并推断出控制反弹的两个基本物理属性-恢复和有效碰撞法线。我们的模型,反弹和学习,包括两个模块-物理推理模块(PIM)和视觉推理模块(VIM)。VIM学习在给定单个静止图像的情况下推断场景中位置的物理参数,而PIM学习在给定物理参数和观察到的碰撞前3D轨迹的情况下为预测任务建模物理交互。为了实现我们的结果,我们引入了Bounce Dataset,其中包括泡沫球弹跳轨迹的5 K RGB-D视频,以探测包括家庭和办公室在内的日常场景中不同形状和材料的表面。我们提出的模型从我们收集的真实世界反弹数据集中学习,并从简单的物理模拟中获得额外的信息。我们在新收集的数据集上显示,我们的模型在预测反弹后的轨迹和推断场景的物理属性方面优于基线,包括与牛顿物理学的轨迹拟合。
Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation
Tsinghua University(清华大学);University of Electronic Science and Technology of China(电子科技大学);Tencent AI Lab(腾讯AI实验室);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院)
Unified Modeling of Lane and Lane Topology for Driving Scene Reasoning
School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);Zhongguancun Academy, Beijing, China(中关村学院,北京,中国);Hangzhou International Innovation Institute, Beihang University(杭州国际创新院,北京航空航天大学);Meituan, Beijing, China(美团,北京,中国)
Learning with Nested Scene Modeling and Cooperative Architecture Search for Low-Light Vision
Dalian University of Technology(大连理工大学)
低光照场景下拍摄的图像往往会受到可见度低、偏色和噪声等严重影响,这些因素不仅影响图像质量,而且会降低下游低光照视觉(LLV)应用的性能。已经提出了各种深度学习方法来增强低光图像的视觉质量。然而,这些方法大多依赖于重要的建筑工程,以获得适当的低光模型,往往遭受高计算负担。此外,将这些增强技术扩展到处理其他LLV仍然具有挑战性。为了部分解决上述问题,我们建立了Retinex启发的Unrolling with Architecture Search(RUAS),这是一个通用的学习框架,它不仅可以解决弱光增强任务,而且还可以灵活地处理其他更具挑战性的下游视觉应用。具体来说,我们首先建立一个嵌套的优化配方,连同展开策略,探索一系列LLV任务的基本原则。此外,我们构建了一个可区分的策略,以合作搜索特定的场景和任务架构的RUAS。最后但并非最不重要的是,我们演示了如何将RUAS应用于低级和高级LLV应用程序(例如,增强、检测和分割)。大量的实验验证了RUAS的灵活性,有效性和效率。
From Data to Modeling: Fully Open-vocabulary Scene Graph Generation
Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系);CAIR, HKISI-CAS(中国科学院香港创新研究院);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所马克思主义学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Modeling Scientific Experiment Scenes: Dataset and Model
College of Computer Science and Engineering, Shandong University of Science and Technology(山东科技大学计算机科学与工程学院);School of Computer Science and Informatics, Cardiff University(卡迪夫大学计算机科学与信息学院);ABC FINTECH Company Limited(ABC金融科技有限公司);NOVA Information Management School, Universidade Nova de Lisboa(里斯本新大学NOVA信息管理学院);School of Biomedical Engineering, Shenzhen University(深圳大学生物医学工程学院)
MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data
The University of Texas at Austin(德克萨斯大学奥斯汀分校);Adobe Research(奥多比研究院);Stony Brook University(石溪大学);Oregon State University(俄勒冈州立大学);Cornell University(康奈尔大学)
提出用程序化生成的70万场景数据集MegaSynth训练三维重建模型,消除语义信息实现可扩展,联合训练或预训练提升重建质量1.2-1.8 dB PSNR,仅用合成数据即可媲美真实数据训练效果。
LocalDyGS: Multi-view Global Dynamic Scene Modeling via Adaptive Local Implicit Feature Decoupling
Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology Shenzhen Graduate School, Peking University(广东省超高清沉浸媒体技术重点实验室 香港大学深圳研究生院);Pengcheng Lab(鹏城实验室);University of Birmingham(伯明翰大学)
MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution
School of Data Science, Fudan University(复旦大学数据科学学院);Shanghai Innovation Institute(上海创新研究院);University of Surrey(Surrey大学);Imperial College London(伦敦帝国理工学院)
DreamScape: 3D Scene Creation via Gaussian Splatting joint Correlation Modeling
School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
THYME: Temporal Hierarchical-Cyclic Interactivity Modeling for Video Scene Graphs in Aerial Footage
University of Arkansas(亚拉巴马大学);University of Science, VNU-HCM(科学大学,VNU-HCM);Vietnam National University, Ho Chi Minh(越南国家大学,胡志明市);Ohio State University(俄亥俄州立大学)