RobotValues: 当人类价值观冲突时评估家用机器人
RobotValues: Evaluating Household Robots When Human Values Conflict
AI总结：
提出RobotValues基准，通过10K个价值冲突场景评估家用机器人规划器，发现视觉语言模型存在默认价值偏好且难以覆盖，表明评估需考虑价值冲突下的行动选择。
摘要：
虽然家用机器人通常基于任务完成度进行评估，但日常家庭环境涉及价值冲突情境，其中机器人应选择优先考虑其他价值观（如人类自主性、效率或社会适宜性）而非任务成功的行动。然而，目前尚无评估机器人在此类场景中价值偏好的基准。我们引入RobotValues，一个在10K个价值冲突场景中评估家用机器人规划器的基准。每个实例包含一个逼真的家庭图像和多个优先考虑不同人类价值观的合理机器人动作。我们通过LLM辅助场景生成、利益相关者基于价值观提取、图像生成和自动质量控制构建RobotValues。使用RobotValues评估机器人领域使用的视觉语言模型，发现模型表现出默认价值偏好，包括安全性和适应性，而低估了隐私优先的行动。当模型被指示优先考虑与其自身偏好冲突的特定价值观时，它们通常无法覆盖默认行动，80%的时间选择了错误行动。这些发现表明，家用机器人评估不仅应衡量任务完成度或安全性合规性，还应衡量当人类价值观冲突时机器人是否能在合理行动中做出选择。
英文摘要：
While household robots are often evaluated based on task completion, everyday domestic environments involve value-conflicting situations where robots are expected to choose actions that prioritize diverse values such as human autonomy, efficiency, or social appropriateness. Yet, there are no benchmarks for evaluating robots' value preferences in such scenarios. We introduce RobotValues, a benchmark to evaluate household robot planners in 8K value-conflict scenarios. Each instance consists of a realistic, synthetically generated household image with multiple plausible robot actions that prioritize different human values. We construct ROBOTVALUES through LLM-assisted scenario generation, stakeholder-grounded value extraction, image generation and automatic quality control. We evaluate 10 VLMs used in robotics and find that models show default value preferences, including safety and accommodation, while underselecting privacy-prioritizing actions. When models are prompted to prioritize values that conflict with their preferences, models often fail to override the default actions, choosing incorrect actions 80% of the time on average across models. These findings highlight the need to go beyond task completion or safety evaluations and assess robots' decision-making capability when human values conflict.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
