HARMONI: 基于大语言模型的多用户人机交互多模态个性化
HARMONI: Multimodal Personalization of Multi-User Human-Robot Interactions with LLMs
AI总结：
HARMONI通过多模态个性化框架，利用大语言模型提升多用户人机交互的持续个性化和动态适应能力。
摘要：
现有的人机交互系统往往缺乏在多用户环境中持续个性化和动态适应的机制，限制了其在现实世界中的应用效果。我们提出了HARMONI，一种多模态个性化框架，利用大语言模型使社交辅助机器人能够管理长期多用户交互。该框架集成了四个关键模块：（i）感知模块，用于识别活跃说话者并提取多模态输入；（ii）世界建模模块，用于维护环境和短期对话上下文的表示；（iii）用户建模模块，用于更新长期说话者特定的资料；（iv）生成模块，用于生成上下文相关且符合伦理的回应。通过在四个数据集上的广泛评估和消融研究，以及在养老院环境中的真实场景驱动用户研究，我们证明HARMONI支持稳健的说话人识别、在线记忆更新和符合伦理的个性化，其在用户建模准确性、个性化质量和用户满意度方面均优于基线的大语言模型驱动方法。
英文摘要：
Existing human-robot interaction systems often lack mechanisms for sustained personalization and dynamic adaptation in multi-user environments, limiting their effectiveness in real-world deployments. We present HARMONI, a multimodal personalization framework that leverages large language models to enable socially assistive robots to manage long-term multi-user interactions. The framework integrates four key modules: (i) a perception module that identifies active speakers and extracts multimodal input; (ii) a world modeling module that maintains representations of the environment and short-term conversational context; (iii) a user modeling module that updates long-term speaker-specific profiles; and (iv) a generation module that produces contextually grounded and ethically informed responses. Through extensive evaluation and ablation studies on four datasets, as well as a real-world scenario-driven user-study in a nursing home environment, we demonstrate that HARMONI supports robust speaker identification, online memory updating, and ethically aligned personalization, outperforming baseline LLM-driven approaches in user modeling accuracy, personalization quality, and user satisfaction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
