CARE：面向文化意识的多语言人类偏好学习
CARE: Multilingual Human Preference Learning for Cultural Awareness
AI总结：
本文提出多语言人类偏好资源CARE，通过纳入原住民文化偏好数据训练语言模型，显著提升了模型的文化意识，并揭示了初始文化性能更强的模型从对齐中获益更多的现象。
摘要：
语言模型（LMs）通常通过人类偏好进行微调以生成有用的回复，但偏好微调对处理文化多样性查询能力的影响仍缺乏研究。本文系统分析了如何将原住民人类文化偏好纳入偏好学习过程，以训练更具文化意识的LMs。我们引入了CARE，这是一个多语言资源，包含3,490个文化特定问题和31.7k个带有人类判断的回复。我们展示了少量高质量原住民偏好如何提升各种LMs的文化意识，其表现优于更大规模的通用偏好数据。分析表明，具有更强初始文化性能的模型从对齐中获益更多，导致在不同地区开发、获取文化相关数据能力不同的模型之间产生差距。CARE已在https://github.com/Guochry/CARE公开发布。
英文摘要：
Language Models (LMs) are typically tuned with human preferences to produce helpful responses, but the impact of preference tuning on the ability to handle culturally diverse queries remains understudied. In this paper, we systematically analyze how native human cultural preferences can be incorporated into the preference learning process to train more culturally aware LMs. We introduce \textbf{CARE}, a multilingual resource containing 3,490 culturally specific questions and 31.7k responses with human judgments. We demonstrate how a modest amount of high-quality native preferences improves cultural awareness across various LMs, outperforming larger generic preference data. Our analyses reveal that models with stronger initial cultural performance benefit more from alignment, leading to gaps among models developed in different regions with varying access to culturally relevant data. CARE is publicly available at https://github.com/Guochry/CARE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
