arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.12727cs.CV
收藏

2.5D视觉关系检测

2.5D Visual Relationship Detection

Google Research(谷歌研究院);UCLA(加州大学洛杉矶分校)

Yu-Chuan Su, Soravit Changpinyo, Xiangning Chen, Sathish Thoppay, Cho-Jui Hsieh, Lior Shapira, Radu Soricut, Hartwig Adam, Matthew Brown, Ming-Hsuan Yang, Boqing Gong

摘要:

视觉2.5D感知涉及通过推理环境中相对于观看者的对象关系来理解场景的语义和几何。然而,现有的工作在视觉识别主要集中在语义。为了弥补这一差距,我们研究了2.5D视觉关系检测(2.5VRD),其目标是联合检测对象并预测它们的相对深度和遮挡关系。与一般的VRD不同,2.5VRD是以自我为中心的,使用相机的视点作为所有2.5D关系的共同参考。与深度估计不同,2.5VRD是以对象为中心的,不仅关注深度。为了使这项任务取得进展,我们创建了一个新的数据集,其中包含来自11K图像的512K对象之间的220K人类注释的2.5D关系。我们分析了这个数据集,并进行了广泛的实验,包括基准测试多个国家的最先进的VRD模型在这项任务。我们的研究结果表明,现有的模型在很大程度上依赖于语义线索和简单的算法来解决2.5VRD,激励进一步研究模型的2.5D感知。新的数据集可在www.example.com上获得。

英文摘要:

Visual 2.5D perception involves understanding the semantics and geometry of a scene through reasoning about object relationships with respect to the viewer in an environment. However, existing works in visual recognition primarily focus on the semantics. To bridge this gap, we study 2.5D visual relationship detection (2.5VRD), in which the goal is to jointly detect objects and predict their relative depth and occlusion relationships. Unlike general VRD, 2.5VRD is egocentric, using the camera's viewpoint as a common reference for all 2.5D relationships. Unlike depth estimation, 2.5VRD is object-centric and not only focuses on depth. To enable progress on this task, we create a new dataset consisting of 220k human-annotated 2.5D relationships among 512K objects from 11K images. We analyze this dataset and conduct extensive experiments including benchmarking multiple state-of-the-art VRD models on this task. Our results show that existing models largely rely on semantic cues and simple heuristics to solve 2.5VRD, motivating further research on models for 2.5D perception. The new dataset is available at https://github.com/google-research-datasets/2.5vrd.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑