叙事中角色的位置：Andersen和Persuasion数据集
Locations of Characters in Narratives: Andersen and Persuasion Datasets
AI总结：
为测试AI理解叙事中角色与位置关系的能力，研究构建Andersen和Persuasion两个数据集并评估5个LLM，发现最佳模型位置识别准确率为61.85%和56.06%。
摘要：
机器在叙事语境中掌握空间理解的能力是阅读理解中一个持续被研究的有趣方面。为了测试AI理解叙事中角色与其各自位置之间关系的能力，我们引入了两个新数据集：Andersen和Persuasion。对于Andersen数据集，我们从安徒生的《安徒生童话》中选取了15个儿童故事，并手动标注了每个故事中的角色及其各自的位置。类似地，对于Persuasion数据集，也手动标注了简·奥斯汀的小说《Persuasion》中的角色及其位置。我们使用这些数据集来提示大型语言模型（LLMs）。提示的创建方式是从故事或小说中提取摘录，并将其与询问该摘录中提到的某个角色位置的问题相结合。在我们测试的5个LLM中，在Andersen数据集上表现最好的模型在61.85%的样本中准确识别了位置，而在Persuasion数据集上表现最好的模型在56.06%的案例中做到了这一点。
英文摘要：
The ability of machines to grasp spatial understanding within narrative contexts is an intriguing aspect of reading comprehension that continues to be studied. Motivated by the goal to test the AI's competence in understanding the relationship between characters and their respective locations in narratives, we introduce two new datasets: Andersen and Persuasion. For the Andersen dataset, we selected fifteen children's stories from "Andersen's Fairy Tales" by Hans Christian Andersen and manually annotated the characters and their respective locations throughout each story. Similarly, for the Persuasion dataset, characters and their locations in the novel "Persuasion" by Jane Austen were also manually annotated. We used these datasets to prompt Large Language Models (LLMs). The prompts are created by extracting excerpts from the stories or the novel and combining them with a question asking the location of a character mentioned in that excerpt. Out of the five LLMs we tested, the best-performing one for the Andersen dataset accurately identified the location in 61.85% of the examples, while for the Persuasion dataset, the best-performing one did so in 56.06% of the cases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
