This paper proposes a novel graph-constrained generative adversarial network, whose generator and discriminator are built upon relational architecture. The main idea is to encode the constraint into the graph structure of its relational networks. We have demonstrated the proposed architecture for a new house layout generation problem, whose task is to take an architectural constraint as a graph (i.e., the number and types of rooms with their spatial adjacency) and produce a set of axis-aligned bounding boxes of rooms. We measure the quality of generated house layouts with the three metrics: the realism, the diversity, and the compatibility with the input graph constraint. Our qualitative and quantitative evaluations over 117,000 real floorplan images demonstrate that the proposed approach outperforms existing methods and baselines. We will publicly share all our code and data.
Intelligent Home 3D: Automatic 3D-House Design from Linguistic Descriptions Only
South China University of Technology(华南理工大学);Guangzhou Laboratory(广州实验室);Kujiale, Inc.(酷家乐公司);University of Adelaide(阿德莱德大学);Australian Centre for Robotic Vision(澳大利亚机器人视觉中心)
Graph Transformer GANs for Graph-Constrained House Generation
ETH Zurich(苏黎世联邦理工学院);Tencent Youtu Lab(腾讯优图实验室);Picsart AI Research(Picsart人工智能研究院);University of Trento(特伦托大学);University of Wurzburg(维尔茨堡大学);KU Leuven(鲁汶大学)
LabBuilder: Protocol-Grounded 3D Layout Generation for Interactable and Safe Laboratory
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Wuhan University(武汉大学);Beihang University(北航);Peking University(北京大学);Tsinghua University(清华大学);Shanghai Jiaotong University(上海交通大学);The Chinese University of Hong Kong(香港中文大学)
PARL: Position-Aware Relation Learning Network for Document Layout Analysis
Unisound AI Technology Co.Ltd(Unisound人工智能技术有限公司);MAIS, Institute of Automation, CAS(MAIS,自动化研究所,中国科学院);Beihang University(北航);School of Computer Science and Technology, Tiangong University(天工大学计算机科学与技术学院)
Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery
The Chinese University of Hong Kong(香港中文大学);Johns Hopkins University(约翰斯·霍普金斯大学)
自动手术手势识别对于机器人手术中的智能认知辅助至关重要。随着近年来机器人辅助微创手术的发展,可以记录包括手术视频和机器人运动学在内的丰富信息,这些信息为理解手术手势提供了补充知识。然而,现有的方法要么单独采用单模态数据或直接连接多模态表示,这不能充分利用视觉和运动学数据中固有的信息相关性来提高手势识别的准确性。在这方面,我们提出了一种新的多模态关系图网络的在线方法(即,MRG-Net)通过在潜在特征空间中的交互式消息传播来动态地整合视觉和运动学信息。具体来说,我们首先使用时间卷积网络和LSTM单元从视频和运动序列中提取嵌入。接下来,我们识别这些多模态嵌入中的多关系,并通过层次关系图学习模块来利用它们。我们的方法的有效性证明了国家的最先进的结果在公共JIGSAWS数据集,优于目前的单模态和多模态的方法在两个节点和结类型的任务。此外,我们在两个中心的da Vinci Research Kit(dVRK)平台收集的内部视觉运动学数据集上验证了我们的方法,并实现了一致的有希望的性能。
CasLayout: Cascaded 3D Layout Diffusion for Indoor Scene Synthesis with Implicit Relation Modeling
MAIS, Institute of Automation, Chinese Academy of Sciences and School of Artificial Intelligence, University of Chinese Academy of Sciences(MAIS,自动化研究所,中国科学院,人工智能学院,中国科学院大学);Tsinghua University(清华大学);SKLMS, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(系统科学研究所,中国科学院);Microsoft Research Asia(微软亚洲研究院)