如果我能让时光倒流:将时间重构作为大语言模型的历史推理任务
If I Could Turn Back Time: Temporal Reframing as a Historical Reasoning Task for LLMs
AI总结:
本文通过让大语言模型以1940年的视角回答挪威语常识问答,评估其时间推理能力,发现英语提示优于挪威语,且更大模型表现更好。
摘要:
在本研究中,我们实验了大语言模型进行时间推理的能力。我们使用一本1940年的挪威语书籍,其中包含常识问答,并提示大语言模型以1940年的视角回答问题。我们还用英语和挪威语两种语言提出这些问题。正确答案通常以句子的形式呈现,评分通过LLM-as-judge(大语言模型作为评判者)完成,并由母语者进行抽样检查。用英语提示始终比用挪威语提示取得更好的结果,这是一个出乎意料的结果。相比之下,使用更大的大语言模型改善了结果。我们测试了DeepSeek-R1、Gemma3、Qwen3和Llama3.1模型系列,以及专门为挪威语精心打造的最大可用大语言模型。
英文摘要:
In this study, we experiment with the ability of LLMs to do temporal reasoning. Using a Norwegian book from 1940 containing trivia questions, we prompt the LLMs to answer the questions as if it were 1940. We also pose the questions in both English and Norwegian. Correct answers are often presented as sentences, and grading is done by means of LLM-as-judge, with sampled checks by a native speaker. Prompting in English consistently gave better results than in Norwegian, an unexpected result. In contrast, using larger LLMs improved results. We tested the DeepSeek-R1, Gemma3, Qwen3, and Llama3.1 model families, and also the largest available LLM especially crafted for Norwegian.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。