共享遗产,迥异写作:重新审视东亚历史文件的资源选择
Shared Heritage, Distinct Writing: Rethinking Resource Selection for East Asian Historical Documents
AI总结:
本文质疑通过古典汉语资源进行跨语言迁移在处理韩日古文历史文件的有效性,实验显示古典汉语数据对模型性能影响有限,强调需谨慎验证而非盲目依赖跨语言迁移。
摘要:
东亚历史文件在形式和实践中共享共同点,尤其在由宫廷史官编纂的实录中。这种共同的语言遗产促使研究者在处理韩日历史文件时使用古典汉语资源进行跨语言迁移,但这些文件仍属低资源语言。本文质疑从古典汉语到韩语汉文和日语 Kanbun 的跨语言迁移假设。在机器翻译、命名实体识别和标点恢复任务中,实验显示古典汉语数据对古韩文汉文文档的语言模型性能影响很小,序列标注任务的性能差异在±0.0068 F1分数内,翻译任务的性能差异最高可达+0.84 BLEU分数。这些限制在各种模型大小、架构和领域特定数据集上保持一致。分析显示,当汉文本地语言数据增加时,古典汉语资源的益处迅速减弱,而在极低资源情况下,韩日历史文件的性能显著提升。这些发现强调了需要仔细的实证验证,而非盲目依赖跨语言迁移。
英文摘要:
Historical documents in the Sinosphere are known to share common formats and practices, particularly in veritable records compiled by court historians. This shared linguistic heritage has led researchers to use Classical Chinese resources for cross-lingual transfer when processing historical documents from Korea and Japan, which remain relatively low-resource. In this paper, we question the assumption of cross-lingual transferability from Classical Chinese to Hanja and Kanbun, the ancient written languages of Korea and Japan, respectively. Our experiments across machine translation, named entity recognition, and punctuation restoration tasks show minimal impact of Classical Chinese datasets on language model performance for ancient Korean documents written in Hanja, with performance differences within $\pm{}0.0068$ F1-score for sequence labeling tasks and up to $+0.84$ BLEU score for translation. These limitations persist consistently across various model sizes, architectures, and domain-specific datasets. Our analysis reveals that the benefits of Classical Chinese resources diminish rapidly as local language data increases for Hanja, while showing substantial improvements only in extremely low-resource scenarios for both Korean and Japanese historical documents. These findings emphasize the need for careful empirical validation rather than assuming benefits from indiscriminate cross-lingual transfer.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。