arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2512.01077cs.CLcs.HC

ELR-1000:一个社区生成的濒危印度-原住民语言数据集

ELR-1000: A Community-Generated Dataset for Endangered Indic Indigenous Languages

Karya;UC San Diego;Microsoft Corporation(微软公司)

Neha Joshi, Pamir Gogoi, Aasim Mirza, Aayush Jansari, Aditya Yadavalli, Ayushi Pandey, Arunima Shukla, Deepthi Sudharsan, Kalika Bali, Vivek Seshadri

AI总结:

ELR-1000数据集通过众包方式收集10种濒危印度-原住民语言的传统食谱,评估LLMs翻译表现,发现提供文化背景信息可显著提升翻译质量,推动公平的文化敏感语言技术发展。

摘要:

我们呈现了一个由1,060种传统食谱组成的文化导向型多模态数据集,这些食谱是来自印度东部偏远地区农村社区的众包数据,涵盖10种濒危语言。这些食谱富含语言和文化细微差别,是通过为数字素养较低的贡献者设计的移动界面收集的。Endangered Language Recipes (ELR)-1000不仅捕捉了烹饪实践,还捕捉了原住民食物传统中嵌入的社会文化背景。我们评估了几种最先进的大型语言模型(LLMs)在将这些食谱翻译成英语方面的表现,并发现:尽管模型具备一定的能力,但它们在低资源、文化特定语言方面表现不佳。然而,我们观察到,提供针对性的上下文——包括关于语言的背景信息、翻译示例以及文化保护的指南——能够显著提高翻译质量。我们的结果强调了需要针对未被代表的语言和领域开发基准测试,以推动公平和文化敏感的语言技术发展。作为这项工作的组成部分,我们向自然语言处理社区发布了ELR-1000数据集,希望它能激励开发用于濒危语言的语言技术。

英文摘要:

We present a culturally-grounded multimodal dataset of 1,060 traditional recipes crowdsourced from rural communities across remote regions of Eastern India, spanning 10 endangered languages. These recipes, rich in linguistic and cultural nuance, were collected using a mobile interface designed for contributors with low digital literacy. Endangered Language Recipes (ELR)-1000 -- captures not only culinary practices but also the socio-cultural context embedded in indigenous food traditions. We evaluate the performance of several state-of-the-art large language models (LLMs) on translating these recipes into English and find the following: despite the models' capabilities, they struggle with low-resource, culturally-specific language. However, we observe that providing targeted context -- including background information about the languages, translation examples, and guidelines for cultural preservation -- leads to significant improvements in translation quality. Our results underscore the need for benchmarks that cater to underrepresented languages and domains to advance equitable and culturally-aware language technologies. As part of this work, we release the ELR-1000 dataset to the NLP community, hoping it motivates the development of language technologies for endangered languages.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑