填补古代阿卡德语文本中的空白：一种掩蔽语言建模方法
Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach
摘要：
我们提出的模型，以完成缺失的文本，给出了古代美索不达米亚文件的音译，最初写在楔形文字粘土片（公元前2500年-公元100年）。由于泥板的损坏，学者们往往依靠上下文线索来手动填充文本中的缺失部分，这是一个主观而耗时的过程。我们确定，这一挑战可以制定为一个掩蔽的语言建模任务，主要用作上下文语言模型的预训练目标。接下来，我们开发了几个专注于Akkadian语言的架构，Akkadian语言是当时的通用语言。我们发现，尽管数据稀缺（100万个令牌），我们可以使用贪心解码方案和来自其他语言和不同时间段的数据进行预训练，在丢失令牌预测（89% hit@5）上实现最先进的性能。最后，我们进行人类评估，显示我们的模型在协助专家用灭绝的语言转录文本方面的适用性。
英文摘要：
We present models which complete missing text given transliterations of ancient Mesopotamian documents, originally written on cuneiform clay tablets (2500 BCE - 100 CE). Due to the tablets' deterioration, scholars often rely on contextual cues to manually fill in missing parts in the text in a subjective and time-consuming process. We identify that this challenge can be formulated as a masked language modelling task, used mostly as a pretraining objective for contextualized language models. Following, we develop several architectures focusing on the Akkadian language, the lingua franca of the time. We find that despite data scarcity (1M tokens) we can achieve state of the art performance on missing tokens prediction (89% hit@5) using a greedy decoding scheme and pretraining on data from other languages and different time periods. Finally, we conduct human evaluations showing the applicability of our models in assisting experts to transcribe texts in extinct languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
