arXiv 2604.153362026-07-30cs.HCcs.AI

面向情感的提示方法用于共情LLM辅导

Facial-Expression-Aware Prompting for Empathetic LLM Tutoring

Neurosciences Graduate Program, University of California San Diego（加州大学圣地亚哥分校神经科学研究生项目）；Department of Cognitive Science, University of California San Diego（加州大学圣地亚哥分校认知科学系）；Department of Computer Science and Engineering, University of California San Diego（加州大学圣地亚哥分校计算机科学与工程系）；Department of Electrical and Computer Engineering, University of California San Diego（加州大学圣地亚哥分校电气与计算机工程系）；Department of Mathematics, University of California San Diego（加州大学圣地亚哥分校数学系）；Face the FACS；Halıcıoğlu Data Science Institute, University of California San Diego（Halıcıoğlu数据科学研究所）

Shuangquan Feng, Laura Fleig, Ruisen Tu, Philip Chi, Edmund Bu, Melinda Ozel, Junhua Ma, Teng Fei, Virginia R. de Sa

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文研究了在LLM辅导中整合面部表情信号以提升共情响应的效果，通过四种不同方法对比发现，基于动作单元的条件化能提升对面部表情的共情响应，而引导峰值帧选择优于随机帧输入。

摘要：

大型语言模型（LLMs）能够实现越来越有能力的辅导式对话代理，但有效的辅导需要超越文本之外对学习者情感和认知状态的敏感性。面部表情提供即时且实用的困惑、沮丧或参与的线索，但尚未在LLM驱动的辅导中得到充分探索。我们调查面部表情意识信号是否能通过提示层面的整合改善共情辅导响应，而无需端到端重训练。我们构建了一个可扩展的模拟辅导环境，其中学生代理从大规模未标记面部表情视频数据集中表现出多样的面部行为，并比较了四种辅导变体：文本-only LLM基线、使用随机面部帧的多模态基线，以及两种基于动作单元估计模型（AUM）的方法，即注入文本AU描述或选择峰值表达帧进行视觉支撑。在960次多轮对话中，跨越三种辅导基础（GPT-5.1、Claude Ops 4.5和Gemini 2.5 Pro），我们评估了针对成对比较的五名人类评估者和一个详尽的AI评估者。基于动作单元的条件化在所有辅导基础中都提升了对面部表情的共情响应，而AUM引导的峰值帧选择优于随机帧视觉输入。文本AU抽象和峰值帧视觉注入显示了模型依赖的优势。控制分析表明，这种改进并未以更差的教育清晰度或对文本线索的响应为代价。最后，AI-人类一致性的最高水平出现在基于面部表情的共情上，支持了该维度的可扩展AI评估。总体而言，我们的结果表明，轻量级、结构化的面部表情表示可以有意义地增强LLM辅导系统中的共情，且具有最小的开销。

英文摘要：

Large language models (LLMs) enable increasingly capable tutoring-style conversational agents, yet effective tutoring requires sensitivity to learners' affective and cognitive states beyond text alone. Facial expressions provide immediate and practical cues of confusion, frustration, or engagement, but remain underexplored in LLM-driven tutoring. We investigate whether facial-expression-aware signals can improve empathetic tutoring responses through prompt-level integration, without end-to-end retraining. We build a scalable simulated tutoring environment where a student agent exhibits diverse facial behaviors from a large unlabeled human facial expression video dataset, and compare four tutor variants: a text-only LLM baseline, a multimodal baseline using a random facial frame, and two Action Unit estimation model (AUM)-based methods that either inject textual AU descriptions or select a peak-expression frame for visual grounding. Across 960 multi-turn conversations spanning three tutor backbones (GPT-5.1, Claude Opus 4.5, and Gemini 2.5 Pro), we evaluate targeted pairwise comparisons with five human raters and an exhaustive AI evaluator. AU-based conditioning consistently improves empathetic responsiveness to facial expressions across all tutor backbones, while AUM-guided peak-frame selection outperforms random-frame visual input. Textual AU abstraction and peak-frame visual injection show model-dependent advantages. Control analyses show that this improvement does not come at the expense of worse pedagogical clarity or responsiveness to textual cues. Overall, our results show that lightweight, structured facial expression representations can meaningfully enhance empathy in LLM-based tutoring systems with minimal overhead. Future work should evaluate these methods using real student facial data collected during authentic learning interactions.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。