面向机器人临床场景理解的轻量级结构化多模态推理
Lightweight Structured Multimodal Reasoning for Clinical Scene Understanding in Robotics
AI总结：
本文提出轻量级智能体多模态框架，结合 Qwen2.5-VL-3B-Instruct 与 SmolAgent，通过场景图和混合检索增强临床视频理解，在 Video-MME 及临床数据集上表现出竞争力。
摘要：
医疗机器人需要鲁棒的多模态感知与推理，以确保动态临床环境中的安全。现有视觉语言模型（VLM）展现出强大的通用能力，但在机器人规划所需的时序推理、不确定性估计和结构化输出方面仍存在局限。我们提出一种用于基于视频的场景理解的轻量级智能体多模态框架。该框架将 Qwen2.5-VL-3B-Instruct 模型与基于 SmolAgent 的编排层相结合，支持思维链推理、语音-视觉融合和动态工具调用。框架会生成结构化场景图，并利用混合检索模块实现可解释且自适应的推理。在 Video-MME 基准和一个定制临床数据集上的评估表明，与最先进的 VLM 相比，该框架具有具竞争力的准确率和更强的鲁棒性，展现了其在机器人辅助手术、患者监测和决策支持中的应用潜力。
英文摘要：
Healthcare robotics requires robust multimodal perception and reasoning to ensure safety in dynamic clinical environments. Current Vision-Language Models (VLMs) demonstrate strong general-purpose capabilities but remain limited in temporal reasoning, uncertainty estimation, and structured outputs needed for robotic planning. We present a lightweight agentic multimodal framework for video-based scene understanding. Combining the Qwen2.5-VL-3B-Instruct model with a SmolAgent-based orchestration layer, it supports chain-of-thought reasoning, speech-vision fusion, and dynamic tool invocation. The framework generates structured scene graphs and leverages a hybrid retrieval module for interpretable and adaptive reasoning. Evaluations on the Video-MME benchmark and a custom clinical dataset show competitive accuracy and improved robustness compared to state-of-the-art VLMs, demonstrating its potential for applications in robot-assisted surgery, patient monitoring, and decision support.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
