以部分确定性进行观察：面向建成环境中机器人场景识别的保形预测
Seeing with Partial Certainty: Conformal Prediction for Robotic Scene Recognition in Built Environments
AI总结：
针对 VLM 地点识别中的幻觉与指令歧义问题，提出基于保形预测的 SwPC 框架，在 Matterport3D 上以免微调方式提供统计保证、提升成功率并减少人工干预。
摘要：
在服务于残障人士（PWD）的辅助机器人中，准确识别建成环境中的地点对于确保机器人在多样化室内空间中安全导航和交互至关重要。语言接口，尤其是由大型语言模型（LLM）和视觉语言模型（VLM）驱动的接口，在这一背景下具有显著前景，因为它们能够解释视觉场景并将其与语义信息相关联。然而，此类接口也以产生幻觉式预测而闻名。此外，人类提供的语言指令也可能含糊不清，缺乏关于具体位置、物体或动作的精确细节，从而加剧幻觉问题。在本研究中，我们提出 Seeing with Partial Certainty（SwPC）——一个旨在度量和校准基于 VLM 的地点识别中不确定性的框架，使模型能够识别自己何时缺乏置信度，并在必要时寻求协助。该框架建立在保形预测理论之上，为地点识别提供统计保证，同时在复杂室内环境设置中尽量减少对人类帮助的请求。通过在广泛使用、标注丰富的场景数据集 Matterport3D 上进行实验，我们表明，相对于现有技术，SwPC 显著提高了成功率，并减少了所需的人工干预量。SwPC 可直接与任何 VLM 配合使用，无需模型微调，提供了一种有前景的轻量级不确定性建模方法，可补充基础模型不断扩展的能力并随之扩展。
英文摘要：
In assistive robotics serving people with disabilities (PWD), accurate place recognition in built environments is crucial to ensure that robots navigate and interact safely within diverse indoor spaces. Language interfaces, particularly those powered by Large Language Models (LLM) and Vision Language Models (VLM), hold significant promise in this context, as they can interpret visual scenes and correlate them with semantic information. However, such interfaces are also known for their hallucinated predictions. In addition, language instructions provided by humans can also be ambiguous and lack precise details about specific locations, objects, or actions, exacerbating the hallucination issue. In this work, we introduce Seeing with Partial Certainty (SwPC) - a framework designed to measure and align uncertainty in VLM-based place recognition, enabling the model to recognize when it lacks confidence and seek assistance when necessary. This framework is built on the theory of conformal prediction to provide statistical guarantees on place recognition while minimizing requests for human help in complex indoor environment settings. Through experiments on the widely used richly-annotated scene dataset Matterport3D, we show that SwPC significantly increases the success rate and decreases the amount of human intervention required relative to the prior art. SwPC can be utilized with any VLMs directly without requiring model fine-tuning, offering a promising, lightweight approach to uncertainty modeling that complements and scales alongside the expanding capabilities of foundational models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
