基于异构无人机-水下机器人系统的LLM-VLM融合框架用于自主港口检测
LLM-VLM Fusion Framework for Autonomous Maritime Port Inspection using a Heterogeneous UAV-USV System
AI总结：
本文提出基于LLM和VLM的融合框架，利用异构无人机-水下机器人系统实现自主港口检测，通过符号规划与语义检测提升检测效率和安全性。
摘要：
海上港口检测在确保复杂海洋环境中安全、合规性和运营效率方面起着关键作用。然而，现有检测方法往往依赖于手动操作和传统计算机视觉技术，这些技术缺乏可扩展性和上下文理解能力。本研究介绍了一种新的集成工程框架，利用大型语言模型（LLM）和视觉语言模型（VLM）之间的协同作用，通过协作的空中和水面机器人平台实现自主海上港口检测。所提出的框架通过基于VLM的语义检测取代传统状态机任务规划器，通过LLM驱动的符号规划和改进的感知管道实现上下文感知和自适应监控。LLM模块将自然语言任务指令转换为可执行的符号计划，通过依赖图编码操作约束，确保安全的无人机-水下机器人协调。同时，VLM模块执行实时语义检测和合规性评估，生成带有上下文推理的结构化报告。该框架通过扩展的MBZIRC海洋模拟器进行验证，使用现实的港口基础设施，并通过实际的机器人检测试验进一步评估。轻量级的机载设计确保了适用于资源受限的海洋平台，推动了智能、自主检测系统的发展。项目资源（代码和视频）可在此处找到：https://github.com/Muhayyuddin/llm-vlm-fusion-port-inspection
英文摘要：
Maritime port inspection plays a critical role in ensuring safety, regulatory compliance, and operational efficiency in complex maritime environments. However, existing inspection methods often rely on manual operations and conventional computer vision techniques that lack scalability and contextual understanding. This study introduces a novel integrated engineering framework that utilizes the synergy between Large Language Models (LLMs) and Vision Language Models (VLMs) to enable autonomous maritime port inspection using cooperative aerial and surface robotic platforms. The proposed framework replaces traditional state-machine mission planners with LLM-driven symbolic planning and improved perception pipelines through VLM-based semantic inspection, enabling context-aware and adaptive monitoring. The LLM module translates natural language mission instructions into executable symbolic plans with dependency graphs that encode operational constraints and ensure safe UAV-USV coordination. Meanwhile, the VLM module performs real-time semantic inspection and compliance assessment, generating structured reports with contextual reasoning. The framework was validated using the extended MBZIRC Maritime Simulator with realistic port infrastructure and further assessed through real-world robotic inspection trials. The lightweight on-board design ensures suitability for resource-constrained maritime platforms, advancing the development of intelligent, autonomous inspection systems. Project resources (code and videos) can be found here: https://github.com/Muhayyuddin/llm-vlm-fusion-port-inspection
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
