SumTablets: 一则苏美尔泥板的转写数据集
SumTablets: A Transliteration Dataset of Sumerian Tablets
AI总结：
SumTablets数据集通过将苏美尔楔形文字泥板的Unicode表示与转写配对，为NLP方法应用于苏美尔转写提供了支持，展示了基于转换器的模型在快速验证转写方面的潜力。
摘要：
苏美尔转写是一种传统系统，用于用拉丁字母表示学者对泥板的解读。得益于诸如ETCSL、CDLI和Oracc等远见的数字楔形文字学项目，大量苏美尔转写已被在线发布，这些数据对于各种搜索和分析任务都是结构良好的。然而，缺乏一个全面、易得的数据集，将转写与泥板楔形文字的数字表示配对，阻碍了现代自然语言处理（NLP）方法在苏美尔转写任务中的应用。为了解决这一缺口，我们提出了SumTablets数据集，该数据集将Unicode表示的91,606块苏美尔楔形文字泥板（总计6,970,407个字符）与Oracc发布的相关转写配对。我们通过首先预处理和标准化Oracc转写，然后将每个解读映射回源字符的Unicode表示来构建SumTablets。此外，我们通过使用特殊标记保留平行结构信息（例如表面、新行、断裂段）。我们发布SumTablets作为Hugging Face数据集（CC BY 4.0）并通过GitHub开放数据准备代码。此外，我们利用SumTablets实现了并评估了两个转写基线：（1）从字符的可能解读中进行加权采样，（2）微调自回归语言模型。我们的微调语言模型在字符层面的转写F分数（chrF）达到97.55，证明了基于转换器的转写模型在允许专家快速验证生成的转写而不是手动逐个转写泥板方面的即时潜力。
英文摘要：
Sumerian transliteration is a conventional system for representing a scholar's interpretation of a tablet in the Latin script. Thanks to visionary digital Assyriology projects such as ETCSL, CDLI, and Oracc, a large number of Sumerian transliterations have been published online, and these data are well-structured for a variety of search and analysis tasks. However, the absence of a comprehensive, accessible dataset pairing transliterations with a digital representation of the tablet's cuneiform glyphs has prevented the application of modern Natural Language Processing (NLP) methods to the task of Sumerian transliteration. To address this gap, we present SumTablets, a dataset pairing Unicode representations of 91,606 Sumerian cuneiform tablets (totaling 6,970,407 glyphs) with the associated transliterations published by Oracc. We construct SumTablets by first preprocessing and standardizing the Oracc transliterations before mapping each reading back to the Unicode representation of the source glyph. Further, we retain parallel structural information (e.g., surfaces, newlines, broken segments) through the use of special tokens. We release SumTablets as a Hugging Face Dataset (CC BY 4.0) and open source data preparation code via GitHub. Additionally, we leverage SumTablets to implement and evaluate two transliteration baselines: (1) weighted sampling from a glyph's possible readings, and (2) fine-tuning an autoregressive language model. Our fine-tuned language model achieves an average transliteration character-level F-score (chrF) of 97.55, demonstrating the immediate potential of transformer-based transliteration models in allowing experts to rapidly verify generated transliterations rather than manually transliterating tablets one-by-one.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
