CURATe：对话式AI助手的个性化对齐基准测试
CURATe: Benchmarking Personalised Alignment of Conversational AI Assistants
AI总结：
本文提出评估基于LLM的AI助手个性化对齐的多轮基准CURATe，发现现有模型在处理安全关键上下文时存在系统性偏差，而提示考虑安全上下文能显著提升性能。
摘要：
我们引入了一个多轮基准测试，用于评估基于LLM的AI助手中的个性化对齐，重点关注其处理用户提供的安全关键上下文的能力。我们对十种领先模型在五种场景下（每种包含337个用例）的评估显示，在维持针对用户的特定考量方面存在系统性不一致，甚至顶级的“无害”模型也会做出在给定上下文下对用户明显有害的建议。主要失败模式包括对冲突偏好权衡不当、谄媚（将欲望置于安全之上）、对上下文窗口内关键用户信息缺乏关注，以及用户特定知识应用的不一致。在OpenAI的o1中也观察到了同样的系统性偏差，这表明强大的推理能力不一定能转化为这种个性化思考。我们发现，提示LLM考虑安全关键上下文可显著提升性能，而通用的“无害且有用”指令则不然。基于这些发现，我们提出了在AI助手中嵌入自我反思能力、在线用户建模和动态风险评估的研究方向。我们的工作强调，在为持久人机交互设计的系统中，需要对齐采用细致入微、上下文感知的方法，以助力开发安全且体贴的AI助手。
英文摘要：
We introduce a multi-turn benchmark for evaluating personalised alignment in LLM-based AI assistants, focusing on their ability to handle user-provided safety-critical contexts. Our assessment of ten leading models across five scenarios (with 337 use cases each) reveals systematic inconsistencies in maintaining user-specific consideration, with even top-rated "harmless" models making recommendations that should be recognised as obviously harmful to the user given the context provided. Key failure modes include inappropriate weighing of conflicting preferences, sycophancy (prioritising desires above safety), a lack of attentiveness to critical user information within the context window, and inconsistent application of user-specific knowledge. The same systematic biases were observed in OpenAI's o1, suggesting that strong reasoning capacities do not necessarily transfer to this kind of personalised thinking. We find that prompting LLMs to consider safety-critical context significantly improves performance, unlike a generic 'harmless and helpful' instruction. Based on these findings, we propose research directions for embedding self-reflection capabilities, online user modelling, and dynamic risk assessment in AI assistants. Our work emphasises the need for nuanced, context-aware approaches to alignment in systems designed for persistent human interaction, aiding the development of safe and considerate AI assistants.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
