通过多跳问答数据集与伪指令调优实现大语言模型的鲁棒时序推理
Towards Robust Temporal Reasoning of Large Language Models via a Multi-Hop QA Dataset and Pseudo-Instruction Tuning
AI总结：
针对现有时序问答研究未关注多答案多跳推理的问题，本文构建了复杂时序推理数据集Complex-TR并提出新型数据增强策略，显著提升了大语言模型的时序推理性能与鲁棒性。
摘要：
现实世界中的知识在不断更新，然而频繁更新大语言模型（LLMs）的成本高昂，因此LLMs理解时序知识的概念至关重要。但现有关于时序问答（TQA）的研究并未强调多答案与多跳类型的时序推理。本文提出了一个聚焦于多答案、多跳时序推理的复杂时序问答数据集Complex-TR。此外，我们还提出了一种新颖的数据增强策略，以提升LLMs的复杂时序推理能力与鲁棒性。我们在多个时序QA数据集上开展了实验，结果表明，该方法能大幅提升LLMs在时序QA基准上的表现。我们的代码与数据已发布于：https://github.com/nusnlp/complex-tr。
英文摘要：
Knowledge in the real world is being updated constantly. However, it is costly to frequently update large language models (LLMs). Therefore, it is crucial for LLMs to understand the concept of temporal knowledge. However, prior works on temporal question answering (TQA) did not emphasize multi-answer and multi-hop types of temporal reasoning. In this paper, we propose a complex temporal question-answering dataset Complex-TR that focuses on multi-answer and multi-hop temporal reasoning. Besides, we also propose a novel data augmentation strategy to improve the complex temporal reasoning capability and robustness of LLMs. We conducted experiments on multiple temporal QA datasets. Experimental results show that our method is able to improve LLMs' performance on temporal QA benchmarks by significant margins. Our code and data are released at: https://github.com/nusnlp/complex-tr.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
