Cryptonite：针对语言极端模糊性的神秘填字游戏基准
Cryptonite: A Cryptic Crossword Benchmark for Extreme Ambiguity in Language
摘要：
目前针对歧义的NLP数据集可以由母语者相对容易地解决。我们介绍了Cryptonite，一个基于神秘填字游戏的大规模数据集，它既具有语言复杂性，又具有自然来源。Cryptonite中的每个例子都是一个神秘的线索，一个简短的短语或句子，具有误导性的表面阅读，其解决需要消除语义，句法和语音文字游戏的歧义，以及世界知识。即使对于经验丰富的解决者来说，隐藏的线索也是一个挑战，尽管顶级专家可以以几乎100%的准确率解决它们。Cryptonite对于当前的模型来说是一项具有挑战性的任务;在470 k个神秘线索上微调T5-Large只能达到7.6%的准确率，与基于规则的线索求解器的准确率（8.6%）相当。
英文摘要：
Current NLP datasets targeting ambiguity can be solved by a native speaker with relative ease. We present Cryptonite, a large-scale dataset based on cryptic crosswords, which is both linguistically complex and naturally sourced. Each example in Cryptonite is a cryptic clue, a short phrase or sentence with a misleading surface reading, whose solving requires disambiguating semantic, syntactic, and phonetic wordplays, as well as world knowledge. Cryptic clues pose a challenge even for experienced solvers, though top-tier experts can solve them with almost 100% accuracy. Cryptonite is a challenging task for current models; fine-tuning T5-Large on 470k cryptic clues achieves only 7.6% accuracy, on par with the accuracy of a rule-based clue solver (8.6%).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
