Apollo Restore：面向历史希腊语的基础大语言模型，针对古希腊文本的中间填充修复进行优化
Apollo Restore: A Foundation LLM for Historical Greek Optimized for Fill-in-the-Middle Restoration of Ancient Greek Texts
浏览论文内容
中文总结 AI 辅助
Apollo Restore 是首个面向历史希腊语的 240 亿参数大模型，通过中间填充目标修复残损文本，在多项基准上超越最强模型，并获专家认可。
中文摘要 AI 辅助
我们提出 Apollo Restore，一个 240 亿参数的大语言模型，用于修复残损古希腊文本中的 lacunae（物理缺口）。该模型基于 Mistral Small 进行微调，采用中间填充（fill-in-the-middle）目标，无需知道缺失片段的长度即可重建缺失部分。据我们所知，这是首个面向历史希腊语的大规模解码器模型，也是首个面向任何古代地中海语言的此类模型。按照先前工作的评估方法，在长达十个字符的短缺口上，Apollo Restore 将正确修复结果置于其前二十个候选中的比例，对于文献纸莎草、文学纸莎草和石碑铭文分别为 80.6%/54.6%/61.0%，超过最强已发表模型 $1.6\ imes$/$2.6\ imes$/$1.4\ imes$。然而，先前的评估协议因偏向于琐碎的短缺口而虚增了分数；在长度平衡的指标下，Apollo Restore 相对于最强已发表模型优势扩大至 $2.3\ imes$/$3.5\ imes$/$1.6\ imes$，并且即使在长度提示错误的情况下也能优雅地退化。在一项盲测中，20 位资深纸莎草学家、铭文学家和语文学家强烈倾向于 Apollo Restore 而非最强基线，并在 77% 的情况下认为其表现至少与人类修复相当。Apollo Restore 还改进了此 http URL 1667 的已发表解读——这是一卷在公元 79 年维苏威火山喷发中碳化、并在 Apollo Restore 训练数据编制后经数字展开和编辑的纸莎草卷。Apollo Restore 是 Decoding Antiquity 计划的成果，该计划由奥地利科学院牵头，旨在为历史语言和手稿构建专门的 LLM。
英文摘要
We present Apollo Restore, a 24-billion-parameter large language model for restoring lacunae---physical gaps---in fragmentary Ancient Greek texts. Fine-tuned from Mistral Small with a fill-in-the-middle objective, Apollo Restore reconstructs missing spans without requiring oracle knowledge of their length. To our knowledge, it is the first large-scale decoder model for historical Greek, and the first for any ancient Mediterranean language. Evaluated as in prior work, on short gaps of up to ten characters, Apollo Restore places the correct restoration among its top twenty candidates for 80.6%/54.6%/61.0% of documentary-papyrus, literary-papyrus, and stone-inscription lacunae, exceeding the strongest published models by $1.6\times$/$2.6\times$/$1.4\times$. Prior evaluation protocols, however, inflate scores through a bias toward trivially short gaps; under a length-balanced metric Apollo Restore's advantage over the strongest published models grows to $2.3\times$/$3.5\times$/$1.6\times$ and degrades gracefully, even given incorrect length hints. In a blind study, 20 expert papyrologists, epigraphists, and philologists strongly preferred Apollo Restore to the strongest baseline and judged its performance at least as good as human restorations in 77% of cases. Apollo Restore also improves the published reading of PHerc. 1667---a papyrus roll carbonised in the eruption of Vesuvius in 79 CE and digitally unrolled and edited after Apollo Restore's training data was compiled. Apollo Restore is an output of the Decoding Antiquity initiative to build specialized LLMs for historical languages and manuscripts, led by the Austrian Academy of Sciences.
发表机构
- Mistral AI
- Reply
- Northeastern University（东北大学）
- Austrian Academy of Sciences（奥地利科学院）
机构由 AI 辅助整理，请以论文原文为准。
