We present ObPose, an unsupervised object-centric inference and generation model which learns 3D-structured latent representations from RGB-D scenes. Inspired by prior art in 2D representation learning, ObPose considers a factorised latent space, separately encoding object location (where) and appearance (what). ObPose further leverages an object's pose (i.e. location and orientation), defined via a minimum volume principle, as a novel inductive bias for learning the where component. To achieve this, we propose an efficient, voxelised approximation approach to recover the object shape directly from a neural radiance field (NeRF). As a consequence, ObPose models each scene as a composition of NeRFs, richly representing individual objects. To evaluate the quality of the learned representations, ObPose is evaluated quantitatively on the YCB, MultiShapeNet, and CLEVR datatasets for unsupervised scene segmentation, outperforming the current state-of-the-art in 3D scene inference (ObSuRF) by a significant margin. Generative results provide qualitative demonstration that the same ObPose model can both generate novel scenes and flexibly edit the objects in them. These capacities again reflect the quality of the learned latents and the benefits of disentangling the where and what components of a scene. Key design choices made in the ObPose encoder are validated with ablations.
Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstruction
School of Engineering, Westlake University(西湖大学工学院);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖高等研究院先进技术研究院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院)
UNO: Unifying One-stage Video Scene Graph Generation via Object-Centric Visual Representation Learning
FPT Software AI Center(FPT软件AI中心);Aalborg University(奥尔堡大学);Pioneer Centre for AI(先锋人工智能中心);S-Lab, Nanyang Technological University(南洋理工大学S实验室);AICV Lab, University of Arkansas(阿肯色大学AICV实验室)
OSCAR-Net: Object-centric Scene Graph Attention for Image Attribution
Adobe Research(Adobe研究院);University of Surrey(萨里大学)
图像讲述了强大的故事,但并不总是可信的。将图像匹配回可信来源(归因)使用户能够对他们在网上遇到的图像做出更明智的判断。我们提出了一个强大的图像哈希算法来执行这样的匹配。我们的哈希对微妙的、突出的视觉细节的操纵很敏感,这些细节可以大大改变图像所讲述的故事。然而,散列对于良性转换(质量,编解码器,大小,形状等的变化)是不变的。在网上重新发布的图片所经历的。我们的主要贡献是OSCAR-Net(Object-centric Scene Graph Attention for Image Attribution Network);这是一个强大的图像哈希模型,灵感来自于最近Transformers在视觉领域的成功。OSCAR-Net构建了一个场景图表示,它关注每个对象的视觉外观及其空间关系的细粒度变化。该网络通过原始图像和操作图像的数据集上的对比学习进行训练,从而产生用于内容指纹识别的最新图像散列,该图像散列可扩展到数百万个图像。
HUNTER: Unsupervised Human-centric 3D Detection via Transferring Knowledge from Synthetic Instances to Real Scenes
ShanghaiTech University(上海科技大学);University of Chinese Academy of Sciences(中国科学院大学);Innovation Academy for Microsatellites, Chinese Academy of Sciences(中国科学院微小卫星创新研究院);The University of Hong Kong(香港大学);The Chinese University of Hong Kong(香港中文大学)
Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);University of Amsterdam(阿姆斯特丹大学);University of Tübingen(蒂宾根大学);Archimedes/Athena RC(阿基米德/雅典娜研究中心)
Reconstructing the Dynamic World: A Representation-Centric View of 4D Scene Reconstruction
University of Bologna(博洛尼亚大学);Peking University(北京大学);The Hong Kong Polytechnic University(香港理工大学);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东省人工智能与数字经济实验室(深圳));Huawei(华为);University of Trento(特伦托大学);University of the Chinese Academy of Sciences(中国科学院大学);Monash University(莫纳什大学);Google DeepMind(谷歌DeepMind);University of California, Merced(加州大学默塞德分校)
Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method
Shanghai Jiao Tong University(上海交通大学);Eastern Institute of Technology(东部技术研究院);School of Electronic Information and Electrical Engineering(电子信息与电气工程学院);Li Auto(力汽车);National University of Singapore(新加坡国立大学);Tsinghua University(清华大学);Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative(宁波空间智能与数字衍生实验室);Ningbo Institute of Digital Twin(宁波数字孪生研究院)
Human-centric Indoor Scene Synthesis Using Stochastic Grammar
UCLA Center for Vision, Cognition, Learning and Autonomy(加州大学洛杉矶分校视觉、认知、学习与自主中心);UPenn Computer Graphics Group(宾夕法尼亚大学计算机图形学组)
我们提出了一种以人为中心的方法来采样和合成3D房间布局及其2D图像,以获得大规模的2D/3D图像数据与完美的每像素地面实况。提出了一种属性空间与或图(S-AOG)来表示室内场景。S-AOG是一个概率文法模型,其中的终端节点是对象实体。人类上下文作为上下文关系在终端节点上用马尔可夫随机场(MRF)编码。我们从室内场景数据集学习分布,并使用Monte Carlo Markov Chain对新布局进行采样。实验表明,我们的方法可以基于三个标准鲁棒地对各种各样的现实房间布局进行采样:(i)与最先进的房间布置方法相比的视觉现实主义,(ii)关于地面实况的示能图的准确性,以及(ii)由人类受试者评估的合成房间的功能性和自然性。该代码可在https://github.com/SiyuanQi/human-centric-scene-synthesis上获得。
Exploring object-centric and scene-centric CNN features and their complementarity for human rights violations recognition in images
School of Computer Science and Electronic Engineering, University of Essex, UK(埃塞克斯大学计算机科学与电子工程学院,英国);School of Computer Science, University of Birmingham, UK(伯明翰大学计算机科学学院,英国)
COME: Adding Scene-Centric Forecasting Control to Occupancy World Model
School of Vehicle and Mobility, Tsinghua University(清华大学车辆与移动系统学院);State Key Laboratory of Intelligent Green Vehicle and Mobility(智能绿色车辆与移动系统国家重点实验室);Kargobot Inc(Kargobot公司);Nankai University(南开大学)
DC-Scene: Data-Centric Learning for 3D Scene Understanding
Shanghai University of Engineering Science(上海工程科学大学);The Australian National University(澳大利亚国立大学);Sun Yat-sen University(中山大学);La Trobe University(拉筹伯大学)