AToM-Bot:基于情感心智理论的未言明人类需求的具身化满足
AToM-Bot: Embodied Fulfillment of Unspoken Human Needs with Affective Theory of Mind
AI总结:
提出AToM-Bot框架,利用情感心智理论提示的视觉语言模型推断人类状态,无需明确指令即可主动生成并执行具身任务以满足未言明的人类需求,在多项指标中获高评价。
摘要:
我们提出了AToM-Bot,一种用于主动机器人与人类交互的新型任务生成与执行框架,它利用了由情感心智理论(AToM)提示的视觉语言模型(VLM)对人类心理和物理状态的推断能力。无需人类发出明确指令,AToM-Bot就能主动生成并执行可行任务以提升人类的整体福祉。在人类周围时,AToM-Bot首先基于推断出的人类状态和对周围环境的观察来检测当前的人类需求。然后,它会在考虑自身具身约束的情况下生成任务以满足这些需求。我们设计了涵盖4种常见场景的16个日常生活情境,并将相同的视觉刺激分配给59名人类受试者和我们的机器人。我们使用人类开放式回答与机器人输出之间的相似度,以及人类满意度评分来衡量机器人的表现。AToM-Bot在需求检测(6.42/7,91.7%)、具身化解决方案(6.15/7,87.8%)和任务执行(6.17/7,88.1%)方面均获得了人类的高度评价。我们表明,AToM-Bot在生成和执行可行计划以满足未言明的人类需求方面表现出色。视频和代码可在https://affective-tom-bot.github.io获取。
英文摘要:
We propose AToM-Bot, a novel task generation and execution framework for proactive robot-human interaction, which leverages the human mental and physical state inference capabilities of the Vision Language Model (VLM) prompted by the Affective Theory of Mind (AToM). Without requiring explicit commands by humans, AToM-Bot proactively generates and follows feasible tasks to improve general human well-being. When around humans, AToM-Bot first detects current human needs based on inferred human states and observations of the surrounding environment. It then generates tasks to fulfill these needs, taking into account its embodied constraints. We designed 16 daily life scenarios spanning 4 common scenes and tasked the same visual stimulus to 59 human subjects and our robot. We used the similarity between human open-ended answers and robot output, and the human satisfaction scores to metric robot performance. AToM-Bot received high human evaluations in need detection (6.42/7, 91.7%), embodied solution (6.15/7, 87.8%) and task execution (6.17/7, 88.1%). We show that AToM-Bot excels in generating and executing feasible plans to fulfill unspoken human needs. Videos and code are available at https://affective-tom-bot.github.io.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。