科米-亚兹瓦语-俄语平行语料库及零样本和少样本LLM翻译评估协议
A Komi-Yazva--Russian Parallel Corpus and Evaluation Protocol for Zero- and Few-Shot LLM Translation
AI总结：
构建首个科米-亚兹瓦语-俄语平行语料库，并提出显式评估协议，研究大语言模型在极度低资源濒危语言翻译中的零样本和检索增强少样本性能。
摘要：
我们提出了首个科米-亚兹瓦语-俄语平行语料库，以及用于研究LLM在濒危、极度低资源环境下翻译的显式评估协议。该数据集包含来自74篇叙事文本的457个对齐句子对，并附有文档化的来源、句子级对齐和故事标识符，支持泄漏感知评估。我们利用这一设置，在零样本和基于检索的少样本场景下，比较了现代大语言模型在科米-亚兹瓦语到俄语翻译中的表现，其中平行数据极度稀缺。协议包括故事级交叉验证、用于少样本提示的确定性检索、生成输出的严格验证、互补的基于参考和基于评判的指标，以及故事级不确定性估计。跨模型而言，LLM产生了有意义的翻译，但性能因模型家族和提示方式而异。基于检索的少样本提示始终优于零样本提示，而超出小检索上下文的增益仍然有限。结果表明，该设置下的评估结论在很大程度上取决于指标选择和失败处理方式，因此本文将该语料库既作为数据集贡献，也作为濒危语言机器翻译的可复现评估测试平台。
英文摘要：
We present the first Komi-Yazva--Russian parallel corpus together with an explicit evaluation protocol for studying LLM translation in an endangered, extremely low-resource setting. The dataset contains 457 aligned sentence pairs from 74 narrative texts and is accompanied by documented provenance, sentence-level alignment, and story identifiers that enable leakage-aware evaluation. We use this setup to compare modern large language models on Komi-Yazva-to-Russian translation under severe parallel-data scarcity in zero-shot and retrieval-based few-shot regimes. The protocol includes story-level cross-validation, deterministic retrieval for few-shot prompting, strict validation of generated outputs, complementary reference-based and judge-based metrics, and story-level uncertainty estimates. Across models, LLMs produce non-trivial translations, but performance varies strongly by model family and prompting regime. Retrieval-based few-shot prompting consistently improves over zero-shot prompting, while gains beyond a small retrieved context remain limited. The results show that evaluative conclusions in this setting depend materially on metric choice and failure handling, so the paper frames the corpus as both a dataset contribution and a reproducible evaluation testbed for endangered-language machine translation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
