说出你不知道的内容:通过表示空间分析与编辑增强角色扮演智能体的拒绝能力
Tell Me What You Don't Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing
AI总结:
本研究针对角色扮演智能体难识别并恰当回应知识冲突查询的问题,构建评估基准测试其表现,通过表示空间分析发现拒绝与响应区域,提出轻量级表示编辑方法提升冲突请求拒绝准确率且保留通用角色扮演能力。
摘要:
角色扮演智能体(Role-Playing Agents, RPAs)在各类应用中表现出色,但往往难以识别与自身角色扮演知识相冲突的高难度查询,并做出恰当回应。为探究RPAs面对不同类型冲突请求时的表现,我们构建了一个评估基准,涵盖上下文知识冲突请求、参数化知识冲突请求与非冲突请求,用于评估RPAs识别冲突、合理拒绝回答且不过度拒绝的能力。经大量评估,我们发现多数RPAs在应对不同冲突请求时存在显著性能差距。为阐明原因,我们对多种冲突场景下的RPAs开展了深入的表示层面分析,发现模型前向传播表示中存在拒绝区域与直接响应区域,二者会影响RPA的最终响应行为。据此,我们提出一种轻量级表示编辑方法,可便捷地将冲突请求映射至拒绝区域,从而提升模型的拒绝准确率。实验结果验证了该编辑方法的有效性,在提升RPAs对冲突请求拒绝能力的同时,保留了其通用角色扮演能力。
英文摘要:
Role-Playing Agents (RPAs) have shown remarkable performance in various applications, yet they often struggle to recognize and appropriately respond to hard queries that conflict with their role-play knowledge. To investigate RPAs' performance when faced with different types of conflicting requests, we develop an evaluation benchmark that includes contextual knowledge conflicting requests, parametric knowledge conflicting requests, and non-conflicting requests to assess RPAs' ability to identify conflicts and refuse to answer appropriately without over-refusing. Through extensive evaluation, we find that most RPAs behave significant performance gaps toward different conflict requests. To elucidate the reasons, we conduct an in-depth representation-level analysis of RPAs under various conflict scenarios. Our findings reveal the existence of rejection regions and direct response regions within the model's forwarding representation, and thus influence the RPA's final response behavior. Therefore, we introduce a lightweight representation editing approach that conveniently shifts conflicting requests to the rejection region, thereby enhancing the model's refusal accuracy. The experimental results validate the effectiveness of our editing method, improving RPAs' refusal ability of conflicting requests while maintaining their general role-playing capabilities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。