EmoDistill: 对抗性谈判中语言模型代理的离线情感技能蒸馏
EmoDistill: Offline Emotion Skill Distillation for Language Model Agents in Adversarial Negotiation
AI总结：
提出EmoDistill离线框架，通过隐式Q学习选择情感和低秩适应策略表达情感，蒸馏情感谈判技能到语言模型代理，在四个高风险谈判领域取得最高效用。
摘要：
后训练的LLM通常被优化以对齐响应与人类偏好，使其安全、礼貌且适合对话。然而，在对抗性谈判中，这种对齐可能成为漏洞：情感框架语言可能引导代理朝向对手方利益。使用基于GoEmotions的情感提示，我们表明情感显著改变谈判结果，表明情感是战略行动渠道而非表面风格。因此，我们引入 extbf{EmoDistill}，一个用于将情感谈判技能蒸馏到语言模型代理中的离线框架。EmoDistill将情感策略分解为情感选择和情感表达：隐式Q学习（IQL）选择器学习表达\emph{哪种}情感，而基于低秩适应（LoRA）的策略通过监督微调（SFT）和裁判策略优化（JPO）学习\emph{如何}表达它。在四个情感敏感、高风险的谈判领域，在EmoDistill框架下训练的SLM策略实现了最高效用，优于普通SLM/LLM基线和仅IQL情感选择。消融实验表明情感条件化是必要的，迁移研究展示了跨领域、未见对手和训练对训练锦标赛的泛化能力。总体而言，EmoDistill从离线代理间交互中学习技能，避免了训练期间昂贵的在线谈判。
英文摘要：
Post-trained LLMs are often optimized to produce helpful, polite, and accommodating responses. In adversarial negotiation, however, such behavior can become a vulnerability: emotionally framed language may influence an agent's bargaining decisions in ways that conflict with its user's objectives. We therefore introduce EmoDistill, an offline framework for distilling emotional negotiation skills from LLM-LLM interactions into smaller language-model agents. Here, an emotional negotiation skill is a state-conditioned behavior that determines which explicit emotion to invoke in a bargaining state and how to realize that emotion as an effective negotiation utterance. EmoDistill learns these two components separately: an Implicit Q-Learning (IQL) selector learns which emotion to express in each bargaining state, while a LoRA-adapted 7B policy learns emotion-conditioned expression through Supervised Fine-Tuning (SFT) and Judge Policy Optimization (JPO). Across four emotion-sensitive negotiation domains, the full EmoDistill policy achieves competitive utility and improves over vanilla and IQL-only baselines in most settings. Emotion-free ablations show that removing the explicit emotion channel substantially reduces overall negotiation utility, while transfer experiments reveal partial, domain-dependent transfer and robustness to unseen LLM counterparties.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
