基于异构多模态大语言模型智能体的物体SLAM语义增强
Semantic Enhancement for Object SLAM with Heterogeneous Multimodal Large Language Model Agents
AI总结：
针对物体SLAM在相似物体关联上的难题，提出融合异构MLLM智能体的SEO-SLAM框架，结合异步处理与多数据关联策略，在提升语义精度的同时保障效率，可赋能下游机器人任务。
摘要：
物体同时定位与建图（SLAM）系统难以正确关联近距离内语义相似的物体，在杂乱室内环境及场景变化时尤为突出。我们提出了面向物体SLAM的语义增强框架SEO-SLAM，这一全新框架通过整合异构多模态大语言模型（MLLM）智能体来增强语义建图效果。我们的方法在维持语义丰富地图的同时实现了场景自适应。为提升计算效率，我们提出了一种异步处理方案，可在不损失语义精度或SLAM性能的前提下，显著缩短智能体的推理时间。此外，我们引入了一种结合语义距离与马氏距离的代价矩阵多数据关联策略，将该问题建模为线性分配问题（LAP）以缓解感知混淆。实验结果表明，与基线方法相比，SEO-SLAM始终能实现更高的语义精度并减少误报，同时我们的异步MLLM智能体相比同步配置显著提升了处理效率。我们还证明SEO-SLAM具备改善机器人辅助等下游任务的潜力。我们的数据集公开发布于：jungseokhong.com/SEO-SLAM。
英文摘要：
Object Simultaneous Localization and Mapping (SLAM) systems struggle to correctly associate semantically similar objects in close proximity, especially in cluttered indoor environments and when scenes change. We present Semantic Enhancement for Object SLAM (SEO-SLAM), a novel framework that enhances semantic mapping by integrating heterogeneous multimodal large language model (MLLM) agents. Our method enables scene adaptation while maintaining a semantically rich map. To improve computational efficiency, we propose an asynchronous processing scheme that significantly reduces the agents' inference time without compromising semantic accuracy or SLAM performance. Additionally, we introduce a multi-data association strategy using a cost matrix that combines semantic and Mahalanobis distances, formulating the problem as a Linear Assignment Problem (LAP) to alleviate perceptual aliasing. Experimental results demonstrate that SEO-SLAM consistently achieves higher semantic accuracy and reduces false positives compared to baselines, while our asynchronous MLLM agents significantly improve processing efficiency over synchronous setups. We also demonstrate that SEO-SLAM has the potential to improve downstream tasks such as robotic assistance. Our dataset is publicly available at: jungseokhong.com/SEO-SLAM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
