arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2404.15238cs.CLcs.AI

CultureBank:面向具有文化意识的语言技术的在线社区驱动知识库

CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies

Stanford University(斯坦福大学);IBM Research(IBM研究院)

Weiyan Shi, Ryan Li, Yutong Zhang, Caleb Ziems, Chunhua yu, Raya Horesh, Rogério Abreu de Paula, Diyi Yang

AI总结:

本文提出可泛化流程构建基于在线社区自述的文化知识库CultureBank,包含多元观点与情境化场景,评估并微调LLM以提升其在零样本下游文化任务的表现。

摘要:

为了增强语言模型的文化意识,我们设计了一个可泛化的流程,以大规模地从不同在线社区构建文化知识库。借助该流程,我们构建了CultureBank,这是一个基于用户自述构建的知识库,包含源自TikTok的12K和Reddit的11K文化描述符。与以往的文化知识资源不同,CultureBank包含对文化描述符的多元观点,以允许对文化知识进行灵活解释,并提供情境化的文化场景以助于扎实的评估。利用CultureBank,我们评估了不同LLM的文化意识,并找出了有待改进的领域。我们还在CultureBank上微调了一个语言模型:实验表明,它在零样本设置下的两个下游文化任务中取得了更好的性能。最后,我们基于发现为未来的具有文化意识的语言技术提供了建议。项目页面为 https://culturebank.github.io 。代码和模型位于 https://github.com/SALT-NLP/CultureBank 。发布的CultureBank数据集位于 https://huggingface.co/datasets/SALT-NLP/CultureBank 。

英文摘要:

To enhance language models' cultural awareness, we design a generalizable pipeline to construct cultural knowledge bases from different online communities on a massive scale. With the pipeline, we construct CultureBank, a knowledge base built upon users' self-narratives with 12K cultural descriptors sourced from TikTok and 11K from Reddit. Unlike previous cultural knowledge resources, CultureBank contains diverse views on cultural descriptors to allow flexible interpretation of cultural knowledge, and contextualized cultural scenarios to help grounded evaluation. With CultureBank, we evaluate different LLMs' cultural awareness, and identify areas for improvement. We also fine-tune a language model on CultureBank: experiments show that it achieves better performances on two downstream cultural tasks in a zero-shot setting. Finally, we offer recommendations based on our findings for future culturally aware language technologies. The project page is https://culturebank.github.io . The code and model is at https://github.com/SALT-NLP/CultureBank . The released CultureBank dataset is at https://huggingface.co/datasets/SALT-NLP/CultureBank .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑