用于对象功能性的预测和生成神经网络
Predictive and Generative Neural Networks for Object Functionality
摘要:
人类甚至可以在没有任何环境的情况下预测对象的功能,因为他们的知识和经验将使他们能够“幻觉”涉及对象的交互或使用场景。我们开发预测和生成的深度卷积神经网络来复制这一壮举。具体来说,我们的工作重点是功能的人造3D对象的特点是人与对象或对象与对象的相互作用。我们的网络是在场景上下文数据库上训练的,称为交互上下文,每个场景上下文由一个中心对象和一个或多个周围对象组成,这些对象代表对象功能。给定一个孤立的3D对象,我们的功能相似性网络(fSIM-NET),三元组网络的一个变体,被训练来通过推断功能揭示交互上下文来预测对象的功能。fSIM-NET由生成网络(iGEN-NET)和分割网络(iSEG-NET)补充。iGEN-NET采用具有功能标签的单个体素化3D对象,并合成体素化环绕,即,交互上下文,它直观地演示了相应的功能。iSEG-NET进一步根据交互类型将交互对象分为不同的组。
英文摘要:
Humans can predict the functionality of an object even without any surroundings, since their knowledge and experience would allow them to "hallucinate" the interaction or usage scenarios involving the object. We develop predictive and generative deep convolutional neural networks to replicate this feat. Specifically, our work focuses on functionalities of man-made 3D objects characterized by human-object or object-object interactions. Our networks are trained on a database of scene contexts, called interaction contexts, each consisting of a central object and one or more surrounding objects, that represent object functionalities. Given a 3D object in isolation, our functional similarity network (fSIM-NET), a variation of the triplet network, is trained to predict the functionality of the object by inferring functionality-revealing interaction contexts. fSIM-NET is complemented by a generative network (iGEN-NET) and a segmentation network (iSEG-NET). iGEN-NET takes a single voxelized 3D object with a functionality label and synthesizes a voxelized surround, i.e., the interaction context which visually demonstrates the corresponding functionality. iSEG-NET further separates the interacting objects into different groups according to their interaction types.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。