基于开源大语言模型的医疗解释机器人手势视觉语言系统
Vision-Language System using Open-Source LLMs for Gestures in Medical Interpreter Robots
AI总结：
本文提出一种基于开源大语言模型的医疗机器人系统，通过检测言语行为并生成相应手势，提升跨语言沟通效率。
摘要：
在医疗领域，有效沟通尤其关键，特别是在语言障碍情况下，非语言线索和手势尤为重要。本文提出了一种隐私保护的视觉语言框架，用于医疗解释机器人，以检测特定的言语行为（同意和指示）并生成相应的机器人手势。该系统基于本地部署的开源模型构建，利用大语言模型（LLM）通过少量示例提示进行意图检测。我们还引入了一个临床对话注释数据集，该数据集标注了言语行为并配以手势视频片段。我们的识别模块在准确率、加权精确率和加权F1分数方面分别达到了0.90、0.93和0.91。我们的方法显著提高了计算效率，并在用户研究中，比语音-手势生成基线在人类似然性方面表现更优，同时保持了相当的适当性。
英文摘要：
Effective communication is vital in healthcare, especially across language barriers, where non-verbal cues and gestures are critical. This paper presents a privacy-preserving vision-language framework for medical interpreter robots that detects specific speech acts (consent and instruction) and generates corresponding robotic gestures. Built on locally deployed open-source models, the system utilizes a Large Language Model (LLM) with few-shot prompting for intent detection. We also introduce a novel dataset of clinical conversations annotated for speech acts and paired with gesture clips. Our identification module achieved 0.90 accuracy, 0.93 weighted precision, and a 0.91 weighted F1-Score. Our approach significantly improves computational efficiency and, in user studies, outperforms the speech-gesture generation baseline in human-likeness while maintaining comparable appropriateness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
