Narrative-of-Thought：通过复述叙事改进大型语言模型的时间推理
Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narratives
AI总结：
本文提出了一种名为Narrative-of-Thought（NoT）的新提示技术，通过将事件集合转换为Python类并引导小模型生成时间叙事，从而在无需微调的预算约束下显著提升了大型语言模型的时间推理能力，其F1分数和结构相似性甚至可媲美GPT-3.5/4。
摘要：
对时间和时间关系的推理是人类认知的重要组成部分，对于感知世界和引导我们的经验至关重要。尽管大型语言模型（LLMs）在许多推理任务中表现出色，但由于其内在复杂性，时间推理仍然具有挑战性。在这项工作中，我们首先研究了时间推理的一个基本任务——时间图生成，以揭示LLMs固有的全局推理能力。我们表明，即使对于最强大的LLMs（如GPT-3.5/4），该任务也面临巨大挑战。我们还注意到，小模型（<10B）存在显著的性能差距，落后于LLMs 50%。接下来，我们研究了如何在预算约束下（例如不使用模型微调）缩小这一差距。我们提出了一种专为时间推理量身定制的新提示技术，即Narrative-of-Thought（NoT），该技术首先将事件集合转换为Python类，然后提示小模型生成具有时间基础的叙事，从而引导最终生成时间图。大量实验表明NoT在改进各项指标方面的有效性。值得注意的是，NoT在Schema-11评估集上获得了最高的F1分数，同时总体F1分数与GPT-3.5持平。NoT还在所有情况下实现了最佳的结构相似性，甚至与GPT-3.5/4相比也是如此。我们的代码可在https://github.com/launchnlp/NoT获取。
英文摘要：
Reasoning about time and temporal relations is an integral aspect of human cognition, essential for perceiving the world and navigating our experiences. Though large language models (LLMs) have demonstrated impressive performance in many reasoning tasks, temporal reasoning remains challenging due to its intrinsic complexity. In this work, we first study an essential task of temporal reasoning -- temporal graph generation, to unveil LLMs' inherent, global reasoning capabilities. We show that this task presents great challenges even for the most powerful LLMs, such as GPT-3.5/4. We also notice a significant performance gap by small models (<10B) that lag behind LLMs by 50%. Next, we study how to close this gap with a budget constraint, e.g., not using model finetuning. We propose a new prompting technique tailored for temporal reasoning, Narrative-of-Thought (NoT), that first converts the events set to a Python class, then prompts a small model to generate a temporally grounded narrative, guiding the final generation of a temporal graph. Extensive experiments showcase the efficacy of NoT in improving various metrics. Notably, NoT attains the highest F1 on the Schema-11 evaluation set, while securing an overall F1 on par with GPT-3.5. NoT also achieves the best structural similarity across the board, even compared with GPT-3.5/4. Our code is available at https://github.com/launchnlp/NoT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
