arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.02449cs.ROcs.AIcs.CVcs.LG
收藏

BYE:利用单序列探索数据构建你的编码器,用于长期动态场景理解

BYE: Build Your Encoder with One Sequence of Exploration Data for Long-Term Dynamic Scene Understanding

University of Freiburg(弗莱堡大学);University of Technology Nuremberg(纽伦堡工业大学)

Chenguang Huang, Shengchao Yan, Wolfram Burgard

AI总结:

本文提出类别无关的场景级点云编码器 BYE,仅用单序列探索数据即可实现长期动态场景中的物体关联,并通过集成 VLM 将成功率提升至 95%。

摘要:

动态场景理解在机器人应用中始终是一项持续存在的挑战。早期的动态建图方法通过掩蔽或跟踪特定类别,专注于减轻短期动态物体对相机运动估计的负面影响,但这些方法往往难以适应长期场景变化。近期的研究利用在合成数据集上训练的神经网络来处理长期动态环境中的物体关联问题,但它们仍然依赖预定义的物体形状和类别。其他方法结合视觉、几何或语义启发式规则进行关联,但通常缺乏鲁棒性。在本工作中,我们提出了 BYE,这是一种类别无关、面向单场景的点云编码器,无需预定义类别、形状先验或大规模关联数据集。BYE 仅在单序列探索数据上进行训练,即可在动态变化的场景中高效执行物体关联。我们进一步提出一种集成方案,将视觉语言模型(VLM)的语义优势与 BYE 的场景专用专长相结合,在物体关联任务中实现了 7% 的提升和 95% 的成功率。代码和数据集可在 https://byencoder.github.io 获取。

英文摘要:

Dynamic scene understanding remains a persistent challenge in robotic applications. Early dynamic mapping methods focused on mitigating the negative influence of short-term dynamic objects on camera motion estimation by masking or tracking specific categories, which often fall short in adapting to long-term scene changes. Recent efforts address object association in long-term dynamic environments using neural networks trained on synthetic datasets, but they still rely on predefined object shapes and categories. Other methods incorporate visual, geometric, or semantic heuristics for the association but often lack robustness. In this work, we introduce BYE, a class-agnostic, per-scene point cloud encoder that removes the need for predefined categories, shape priors, or extensive association datasets. Trained on only a single sequence of exploration data, BYE can efficiently perform object association in dynamically changing scenes. We further propose an ensembling scheme combining the semantic strengths of Vision Language Models (VLMs) with the scene-specific expertise of BYE, achieving a 7% improvement and a 95% success rate in object association tasks. Code and dataset are available at https://byencoder.github.io.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑