arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.12971cs.CL
收藏

大型语言模型的自复数化文化对齐

Self-Pluralising Culture Alignment for Large Language Models

School of New Media and Communication, Tianjin University(天津大学新媒体与传播学院);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部)

Shaoyang Xu, Yongqi Leng, Linhao Yu, Deyi Xiong

AI总结:

提出CultureSPA框架,通过生成文化问题、比较文化感知与无文化感知输出收集实例并微调模型,实现LLMs对多元文化的对齐,显著提升文化对齐能力且不损害一般能力。

摘要:

随着大型语言模型(LLMs)在许多国家日益普及,将它们对齐以服务于跨文化的多元人类价值观至关重要。然而,LLMs中的多元文化对齐仍是一个开放问题。在本文中,我们提出了CultureSPA,这是一种自复数化文化对齐框架,允许LLMs同时对齐多元文化。该框架首先生成关于各种文化主题的问题,然后在文化感知和无文化感知设置下,针对这些生成的问题产生LLM输出。通过比较文化感知/无文化感知输出,我们能够检测并收集与文化相关的实例。这些实例被用于微调LLMs,以文化联合或文化特定的方式服务于多元文化。大量实验表明,CultureSPA显著提高了LLMs对多元文化的对齐能力,且不损害其一般能力。如果将CultureSPA与先进的提示工程技术相结合,可以取得进一步的改进。对文化联合和文化特定调优策略的比较,以及数据质量和数量的变化,说明了我们方法的鲁棒性。我们还探讨了CultureSPA背后的机制及其反映的不同文化之间的关系。

英文摘要:

As large language models (LLMs) become increasingly accessible in many countries, it is essential to align them to serve pluralistic human values across cultures. However, pluralistic culture alignment in LLMs remain an open problem. In this paper, we propose CultureSPA, a Self-Pluralising Culture Alignment framework that allows LLMs to simultaneously align to pluralistic cultures. The framework first generates questions on various culture topics, then yields LLM outputs in response to these generated questions under both culture-aware and culture-unaware settings. By comparing culture-aware/unaware outputs, we are able to detect and collect culture-related instances. These instances are employed to fine-tune LLMs to serve pluralistic cultures in either a culture-joint or culture-specific way. Extensive experiments demonstrate that CultureSPA significantly improves the alignment of LLMs to diverse cultures without compromising general abilities. And further improvements can be achieved if CultureSPA is combined with advanced prompt engineering techniques. Comparisons between culture-joint and culture-specific tuning strategies, along with variations in data quality and quantity, illustrate the robustness of our method. We also explore the mechanisms underlying CultureSPA and the relations between different cultures it reflects.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑