我应该加入哪里？通过语言引导的目标预测实现机器人群体加入

中文摘要 AI 辅助

社交导航通常假设一个指定的目标，并专注于在遵守社会惯例的同时到达该目标，而机器人群体加入则需要根据群体的实时活动和队形来预测加入位置。这是一个高度语义化的任务，但对于机器人导盲犬和自动移动踏板车等应用而言却是一项重要能力。我们提出了语言引导的机器人群体加入问题：给定一个观察和一个目标群体的自然语言描述，机器人识别相关群体成员并预测符合社会规范的加入姿态。为了实现接地，我们通过递归谱分割生成结构化的候选子集，并使用语言条件下的图像-几何模型对其进行排序。在给定接地群体后，一个目标预测器利用人类队形先验来生成可行机器人姿态上的多模态能量-方向图。在对话、队列和观众等不同群体规模、人群密度和视觉模糊性下的实验表明，我们的方法在接地准确性上具有竞争力，推理时间低于一秒，并且在加入姿态预测上优于所有基线。真实机器人实验进一步展示了在静态和动态变化的交互中的群体加入。

英文摘要

Social navigation typically assumes a specified goal and focuses on reaching it while respecting social conventions, whereas robot group joining requires predicting where to join based on the group's real-time activity and formation. This is a highly semantic task, yet an important capability for applications such as robotic guide dogs and autonomous mobility scooters. We formulate language-grounded robot group joining: given an observation and a natural-language description of a target group, the robot identifies the relevant group members and predicts socially compliant joining poses. For grounding, we generate structured candidate subsets through recursive spectral partitioning and rank them with a language-conditioned image--geometry model. Given the grounded group, a goal predictor leverages human-formation priors to produce a multimodal energy--orientation map over feasible robot poses. Experiments on conversations, queues, and audiences across varying group sizes, crowd densities, and visual ambiguities show that our method achieves competitive grounding accuracy with sub-second inference and outperforms all baselines in joining-pose prediction. Real-robot experiments further demonstrate group joining in both static and dynamically changing interactions.

发表机构

• School of Computing（计算学院）
• Smart Systems Institute, National University of Singapore（新加坡国立大学智能系统研究所）

机构由 AI 辅助整理，请以论文原文为准。