BhashaVerse：面向印度次大陆语言的翻译生态系统
BhashaVerse : Translation Ecosystem for Indian Subcontinent Languages
AI总结：
本文提出BhashaVerse翻译生态系统，面向36种印度语言构建语料库并综合评估机器翻译，以提升低资源与多文字场景下的跨语言沟通质量。
摘要：
本文聚焦于为36种印度语言开发翻译模型及相关应用，这些语言包括阿萨姆语、阿瓦迪语、孟加拉语、博杰普尔语、布拉吉语、博多语、多格拉语、英语、孔卡尼语、贡德语、古吉拉特语、印地语、印式英语、霍语、卡纳达语、坎格里语、克什米尔语（阿拉伯文和天城文）、卡西语、米佐语、摩揭陀语、迈蒂利语、马拉雅拉姆语、马拉地语、曼尼普尔语（孟加拉文和梅泰文）、尼泊尔语、奥里亚语、旁遮普语、梵语、桑塔利语、僧伽罗语、信德语（阿拉伯文和天城文）、泰米尔语、图鲁语、泰卢固语和乌尔都语。实现这一目标需要为所有36×36个语言对构建平行语料库及其他类型语料库，以应对文字变体、语音差异和句法多样性等挑战。例如，克什米尔语和信德语等使用多种文字的语言需要进行文字规范化以实现对齐，而卡西语和桑塔利语等低资源语言则需要通过合成数据增强来保证足够的覆盖范围和质量。为应对这些挑战，本文提出了语料库构建策略，包括利用现有资源、开发平行数据集、生成特定领域语料库以及运用合成数据技术。此外，本文还从多个维度评估机器翻译，包括标准翻译和篇章级翻译、特定领域翻译、基于参考和免参考的评估、错误分析以及自动后编辑。通过整合这些要素，本研究建立了一个综合框架，以提升机器翻译质量，并在印度语言多样化的生态系统中实现更好的跨语言交流。
英文摘要：
This paper focuses on developing translation models and related applications for 36 Indian languages, including Assamese, Awadhi, Bengali, Bhojpuri, Braj, Bodo, Dogri, English, Konkani, Gondi, Gujarati, Hindi, Hinglish, Ho, Kannada, Kangri, Kashmiri (Arabic and Devanagari), Khasi, Mizo, Magahi, Maithili, Malayalam, Marathi, Manipuri (Bengali and Meitei), Nepali, Oriya, Punjabi, Sanskrit, Santali, Sinhala, Sindhi (Arabic and Devanagari), Tamil, Tulu, Telugu, and Urdu. Achieving this requires parallel and other types of corpora for all 36 * 36 language pairs, addressing challenges like script variations, phonetic differences, and syntactic diversity. For instance, languages like Kashmiri and Sindhi, which use multiple scripts, demand script normalization for alignment, while low-resource languages such as Khasi and Santali require synthetic data augmentation to ensure sufficient coverage and quality. To address these challenges, this work proposes strategies for corpus creation by leveraging existing resources, developing parallel datasets, generating domain-specific corpora, and utilizing synthetic data techniques. Additionally, it evaluates machine translation across various dimensions, including standard and discourse-level translation, domain-specific translation, reference-based and reference-free evaluation, error analysis, and automatic post-editing. By integrating these elements, the study establishes a comprehensive framework to improve machine translation quality and enable better cross-lingual communication in India's linguistically diverse ecosystem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
