ChronoSense：利用事件时间区间探索大语言模型的时间理解能力
ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Events
AI总结：
本文提出ChronoSense基准，通过16项任务评估大语言模型对Allen时间区间关系和时间算术的理解能力，揭示现有模型表现不佳且可能依赖记忆回答问题。
摘要：
大语言模型（LLMs）在各种自然语言处理任务中取得了显著成功，但在推理和算术方面仍面临重大挑战。时间推理作为自然语言理解的关键组成部分，已引起越来越多的研究关注。然而，对Allen区间关系（如before、after、during）——时间关系的一个基本框架——的全面测试仍未得到充分探索。为填补这一空白，我们提出了ChronoSense，一个用于评估大语言模型时间理解能力的新基准。它包含16项任务，重点识别两个时间事件之间的Allen关系以及时间算术，同时使用抽象事件和来自Wikidata的真实世界数据。我们使用该基准评估了七个近期大语言模型的性能，结果表明模型处理Allen关系（甚至对称关系）的方式存在很大差异。此外，研究结果表明，模型可能依赖记忆来回答与时间相关的问题。总体而言，模型的低性能凸显了改进大语言模型时间理解能力的必要性，而ChronoSense为该领域的未来研究提供了一个稳健的框架。我们的数据集和源代码可在https://github.com/duyguislakoglu/chronosense获取。
英文摘要：
Large Language Models (LLMs) have achieved remarkable success in various NLP tasks, yet they still face significant challenges in reasoning and arithmetic. Temporal reasoning, a critical component of natural language understanding, has raised increasing research attention. However, comprehensive testing of Allen's interval relations (e.g., before, after, during) -- a fundamental framework for temporal relationships -- remains underexplored. To fill this gap, we present ChronoSense, a new benchmark for evaluating LLMs' temporal understanding. It includes 16 tasks, focusing on identifying the Allen relation between two temporal events and temporal arithmetic, using both abstract events and real-world data from Wikidata. We assess the performance of seven recent LLMs using this benchmark and the results indicate that models handle Allen relations, even symmetrical ones, quite differently. Moreover, the findings suggest that the models may rely on memorization to answer time-related questions. Overall, the models' low performance highlights the need for improved temporal understanding in LLMs and ChronoSense offers a robust framework for future research in this area. Our dataset and the source code are available at https://github.com/duyguislakoglu/chronosense.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
