首页/论文/2003.08925

arXiv 2003.089252020-03-19cs.CL

Utilizing Language Relatedness to improve Machine Translation: A Case Study on Languages of the Indian Subcontinent

Microsoft India（微软印度）；Indian Institute of Technology Bombay（印度理工学院孟买分校）

Anoop Kunchukuttan, Pushpak Bhattacharyya

备注：: This work was done in 2017-2018 as part of the first author's thesis

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

In this work, we present an extensive study of statistical machine translation involving languages of the Indian subcontinent. These languages are related by genetic and contact relationships. We describe the similarities between Indic languages arising from these relationships. We explore how lexical and orthographic similarity among these languages can be utilized to improve translation quality between Indic languages when limited parallel corpora is available. We also explore how the structural correspondence between Indic languages can be utilized to re-use linguistic resources for English to Indic language translation. Our observations span 90 language pairs from 9 Indic languages and English. To the best of our knowledge, this is the first large-scale study specifically devoted to utilizing language relatedness to improve translation between related languages.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

BhashaVerse：面向印度次大陆语言的翻译生态系统

BhashaVerse : Translation Ecosystem for Indian Subcontinent Languages

Language Technologies Research Centre, IIIT-Hyderabad（印度信息技术研究所海得拉巴分校语言技术研究中心）

本文提出BhashaVerse翻译生态系统，面向36种印度语言构建语料库并综合评估机器翻译，以提升低资源与多文字场景下的跨语言沟通质量。

2025-01-02cs.AIcs.CL

Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study

Indian Institute of Technology Bombay（孟买印度理工学院）；Google Research（谷歌研究院）

Recent research in multilingual language models (LM) has demonstrated their ability to effectively handle multiple languages in a single model. This holds promise for low web-resource languages (LRL) as multilingual models can enable transfer of supervision from high resource languages to LRLs. However, incorporating a new language in an LM still remains a challenge, particularly for languages with limited corpora and in unseen scripts. In this paper we argue that relatedness among languages in a language family may be exploited to overcome some of the corpora limitations of LRLs, and propose …

2021-06-09cs.CL

印度次大陆的日常言语

Everyday Speech in the Indian Subcontinent

Indian Institute of Technology Madras（印度马德拉斯理工学院）；Shiv Nadar University（希夫纳达尔大学）

针对印度多语言多文字下E2E多语言合成单元词汇量大的问题，研究提出基于语音学的通用标签集CLS，实现13种印度语言与英语的无缝语码转换，适配印度次大陆日常多语言使用场景。

2025-02-21cs.CLcs.SDeess.AS

语音中的亲缘关系：利用语言相关性实现印度语言的零样本语音合成

Kinship in Speech: Leveraging Linguistic Relatedness for Zero-Shot TTS in Indian Languages

Indian Institute of Technology, Madras（印度理工学院马德拉斯分校）

本文提出通过增强共享音素表示并修改文本解析规则以匹配目标语言音系规则，实现印度低资源语言的零样本语音合成，并在梵语、孔卡尼语等五种语言上验证了方法的有效性。

2025-06-04cs.CLcs.CV

Towards Building ASR Systems for the Next Billion Users

IIT Madras（印度理工学院马德拉斯分校）；AI4Bharat（AI4Bharat（印度人工智能研究机构））；RBCDSAI（RBCDSAI（数据科学与人工智能研究中心））；Microsoft（微软）

Recent methods in speech and language technology pretrain very LARGE models which are fine-tuned for specific tasks. However, the benefits of such LARGE models are often limited to a few resource rich languages of the world. In this work, we make multiple contributions towards building ASR systems for low resource languages from the Indian subcontinent. First, we curate 17,000 hours of raw speech data for 40 Indian languages from a wide variety of domains including education, news, technology, and finance. Second, using this raw speech data we pretrain several variants of wav2vec style models …

2021-12-22cs.CLcs.SDeess.AS

IndicMMLU-Pro：在多任务语言理解上对印度语系大语言模型进行基准测试

IndicMMLU-Pro: Benchmarking Indic Large Language Models on Multi-Task Language Understanding

Artificial Intelligence Institute, University of South Carolina（南卡罗来纳大学人工智能学院）；Rochester Institute of Technology（罗切斯特理工学院）；Meta AI；Amazon Gen AI（亚马逊生成式人工智能部门）

本文提出了IndicMMLU-Pro基准，基于MMLU Pro框架评估大语言模型在9种印度次大陆主要语言的多任务理解能力，涵盖语言理解、推理与生成任务，并提供了最先进多语言模型的基线结果。

2025-01-28cs.AIcs.CL

更多相关论文收起更多论文

通过印度语言的视角看多语言分词：挑战与洞见

Multilingual Tokenization through the Lens of Indian Languages: Challenges and Insights

Department of CSE, Indian Institute of Technology Hyderabad（印度理工学院海得拉巴分校计算机科学与工程系）；Department of CSE, Indian Institute of Technology Bombay（印度理工学院孟买分校计算机科学与工程系）；SCEE, Indian Institute of Technology Mandi（印度理工学院曼迪分校软件工程系）；Adobe Research（Adobe研究）

本研究针对17种印度语言，系统评估了子词算法、标准化方法、词汇量及词汇构建策略对多语言分词的影响，发现脚本特定标准化、Unigram LM保留形态边界及聚类词汇构建可提升性能。

2026-06-23cs.CL

从文化遗产保护视角重新思考印度人工智能

Rethinking Indic AI from a Lens of Cultural Heritage Preservation

International Institute of Information Technology, Bengaluru（班加罗尔国际信息技术学院）

本文从文化遗产保护角度探讨人工智能对印度语言文化的影响，通过纵向调查自然语言处理技术演变、分析印度语言特征及基础模型作用，提出‘文化感知’方向，为印度自然语言处理下一阶段研究及基础模型发展提供指引。

2026-07-08cs.AIcs.CL

基于多语言编码器与知识蒸馏的低资源神经机器翻译增强：一项案例研究

Enhancing Low-Resource NMT with a Multilingual Encoder and Knowledge Distillation: A Case Study

Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）；Vizzhy Inc（Vizzhy公司）

针对mBART-50等预训练多语言模型覆盖语种有限、扩展易出现灾难性遗忘的问题，提出结合多语言编码器seq2seq架构与互补知识蒸馏的低资源NMT框架，在印度低资源语言翻译任务上指标显著提升，代码已开源。

2024-07-09cs.CL

解码多样性：印度语系 AI 研究图景综述

Decoding the Diversity: A Review of the Indic AI Research Landscape

AI Institute, University of South Carolina（南卡罗来纳大学AI学院）；University of South Carolina（南卡罗来纳大学）；Stanford University（斯坦福大学）；DeepSig Inc.（DeepSig公司）；Amazon GenAI（亚马逊生成式AI部门）

本文综述印度语系语言的LLM研究，梳理84篇近期论文并提出涵盖模型开发、微调、语料、评测及应用的分类体系，指出数据稀缺、标准化不足和语言复杂性等核心挑战。

2024-06-13cs.AIcs.CLcs.LG