arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.18192cs.CLcs.AI
收藏

面向特定文化语境的大型英语语言模型适配方法

Methodology of Adapting Large English Language Models for Specific Cultural Contexts

AI Innovation Center, China Unicom(中国联通AI创新中心);Unicom Digital Technology, China Unicom(中国联通数字科技)

Wenjing Zhang, Siqi Xiao, Xuejiao Lei, Ning Wang, Huazheng Zhang, Meijuan An, Bikun Yang, Zhaoxiang Liu, Kai Wang, Shiguo Lian

AI总结:

本文提出一种基于特定文化知识和安全价值观指令微调的快速适配方法,解决了英语LLMs在特定文化领域应用的知识缺失与价值观误解问题,在LLaMA3-8B上验证了其能增强领域能力并保持原有优势。

摘要:

大型语言模型(LLMs)的快速发展已成为人工智能领域的显著趋势。然而,当前最先进的LLMs主要以英语为基础。由于缺乏特定领域知识以及文化价值观差异导致的误解,将其直接应用于特定文化领域的任务时会遇到局限性。为应对这一挑战,本文提出了一种在特定文化语境下大型模型的快速适配方法,该方法利用基于特定文化知识和安全价值观数据的指令微调。以中文作为特定文化语境,并利用LLaMA3-8B作为实验性英语LLM,评估结果表明,适配后的LLM显著增强了其在特定领域知识的能力和对安全价值观的适应性,同时保持了其原有的专业优势。

英文摘要:

The rapid growth of large language models(LLMs) has emerged as a prominent trend in the field of artificial intelligence. However, current state-of-the-art LLMs are predominantly based on English. They encounter limitations when directly applied to tasks in specific cultural domains, due to deficiencies in domain-specific knowledge and misunderstandings caused by differences in cultural values. To address this challenge, our paper proposes a rapid adaptation method for large models in specific cultural contexts, which leverages instruction-tuning based on specific cultural knowledge and safety values data. Taking Chinese as the specific cultural context and utilizing the LLaMA3-8B as the experimental English LLM, the evaluation results demonstrate that the adapted LLM significantly enhances its capabilities in domain-specific knowledge and adaptability to safety values, while maintaining its original expertise advantages.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑