SD-OVON：动态场景中开放词汇目标导航的语义感知数据集与基准生成流水线
SD-OVON: A Semantics-aware Dataset and Benchmark Generation Pipeline for Open-Vocabulary Object Navigation in Dynamic Scenes
AI总结：
提出SD-OVON流水线，利用多模态基础模型生成动态场景的逼真变体，构建开放词汇目标导航数据集和基准，支持真实与模拟环境，提升导航智能体训练与评估效果。
摘要：
我们提出了动态场景中开放词汇目标导航的语义感知数据集与基准生成流水线（SD-OVON）。该流水线利用预训练的多模态基础模型生成无限数量的、符合真实世界语义和日常常识的独特逼真场景变体，用于导航智能体的训练和评估，并附带一个与Habitat模拟器兼容的目标导航任务片段生成插件。此外，我们提供了两个预生成的目标导航任务数据集，SD-OVON-3k和SD-OVON-10k，分别包含约3k和10k个开放词汇目标导航任务片段，这些片段源自包含2.5k个真实环境逼真扫描的SD-OVON-Scenes数据集和包含0.9k个经人工检查的扫描及艺术家创建的可操作对象模型的SD-OVON-Objects数据集。与先前仅限于静态环境的数据集不同，SD-OVON覆盖了动态场景和可操作对象，促进了真实到模拟和模拟到真实的机器人应用。该方法增强了导航任务的真实性，以及在复杂环境中开放词汇目标导航智能体的训练和评估。为了展示我们流水线和数据集的有效性，我们提出了两个基线，并在SD-OVON-3k上与最先进的基线一起进行了评估。数据集、基准和源代码均可公开获取。
英文摘要：
We present the Semantics-aware Dataset and Benchmark Generation Pipeline for Open-vocabulary Object Navigation in Dynamic Scenes (SD-OVON). It utilizes pretraining multimodal foundation models to generate infinite unique photo-realistic scene variants that adhere to real-world semantics and daily commonsense for the training and the evaluation of navigation agents, accompanied with a plugin for generating object navigation task episodes compatible to the Habitat simulator. In addition, we offer two pre-generated object navigation task datasets, SD-OVON-3k and SD-OVON-10k, comprising respectively about 3k and 10k episodes of the open-vocabulary object navigation task, derived from the SD-OVON-Scenes dataset with 2.5k photo-realistic scans of real-world environments and the SD-OVON-Objects dataset with 0.9k manually inspected scanned and artist-created manipulatable object models. Unlike prior datasets limited to static environments, SD-OVON covers dynamic scenes and manipulatable objects, facilitating both real-to-sim and sim-to-real robotic applications. This approach enhances the realism of navigation tasks, the training and the evaluation of open-vocabulary object navigation agents in complex settings. To demonstrate the effectiveness of our pipeline and datasets, we propose two baselines and evaluate them along with state-of-the-art baselines on SD-OVON-3k. The datasets, benchmark and source code are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
