面向个性化与代表性价值对齐的奖励设计民主化
Democratizing Reward Design for Personal and Representative Value-Alignment
AI总结：
针对传统AI价值对齐方法易压制少数偏好的问题，提出交互式反思对话对齐方法，通过基于语言模型的偏好引导构建个性化奖励模型，实验验证其可准确捕捉个体独特价值理解，助力个性化与更具代表性的集体对齐。
摘要：
由于人类价值概念具有多样性和主观性，让AI智能体与人类价值对齐颇具挑战。标准对齐方法通常聚合众包反馈，可能会压制独特或少数群体的偏好。我们提出交互式反思对话对齐（Interactive-Reflective Dialogue Alignment）方法，该方法以迭代方式引导用户反思并明确其主观价值定义。该系统通过基于语言模型的偏好引导学习个体价值定义，构建可用于对齐AI行为的个性化奖励模型。我们通过两项共30名参与者的研究评估了该系统，一项聚焦“尊重”，另一项聚焦自动驾驶车辆的伦理决策。研究结果显示了价值对齐行为的定义多样性，且该系统能准确捕捉每个人的独特理解。此方法可实现个性化对齐，也能为更具代表性、可解释的集体对齐策略提供参考。
英文摘要：
Aligning AI agents with human values is challenging due to diverse and subjective notions of values. Standard alignment methods often aggregate crowd feedback, which can result in the suppression of unique or minority preferences. We introduce Interactive-Reflective Dialogue Alignment, a method that iteratively engages users in reflecting on and specifying their subjective value definitions. This system learns individual value definitions through language-model-based preference elicitation and constructs personalized reward models that can be used to align AI behaviour. We evaluated our system through two studies with 30 participants, one focusing on "respect" and the other on ethical decision-making in autonomous vehicles. Our findings demonstrate diverse definitions of value-aligned behaviour and show that our system can accurately capture each person's unique understanding. This approach enables personalized alignment and can inform more representative and interpretable collective alignment strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
