Time-Stamped Language Model：教授语言模型以理解事件流
Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Events
摘要：
由于过程中描述的世界的动态性，在文本中描述的整个过程中跟踪实体具有挑战性。首先，我们建议制定这个任务作为一个问答问题。这使我们能够在其他QA基准测试中使用预先训练的基于transformer的语言模型，使其适应过程性文本理解。其次，由于基于transformer的语言模型不能单独对事件流进行编码，本文提出了一种基于时间戳的语言模型（TSLM模型），通过引入时间戳编码来对LM体系结构中的事件信息进行编码。我们的模型在Propara数据集上进行了评估，结果显示出对已发表的最新结果的改进，F1评分增加了$3.1\%$。此外，我们的模型在NPN-Cooking数据集上的位置预测任务上产生了更好的结果。这一结果表明，我们的方法是有效的程序性文本的理解一般。
英文摘要：
Tracking entities throughout a procedure described in a text is challenging due to the dynamic nature of the world described in the process. Firstly, we propose to formulate this task as a question answering problem. This enables us to use pre-trained transformer-based language models on other QA benchmarks by adapting those to the procedural text understanding. Secondly, since the transformer-based language models cannot encode the flow of events by themselves, we propose a Time-Stamped Language Model~(TSLM model) to encode event information in LMs architecture by introducing the timestamp encoding. Our model evaluated on the Propara dataset shows improvements on the published state-of-the-art results with a $3.1\%$ increase in F1 score. Moreover, our model yields better results on the location prediction task on the NPN-Cooking dataset. This result indicates that our approach is effective for procedural text understanding in general.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
