arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.02162cs.CL
收藏

时间感知的文本生成算法

Time-aware Prompting for Text Generation

University of Michigan(密歇根大学)

Shuyang Cao, Lu Wang

摘要:

在本文中,我们研究了将时间戳,如文件创建日期,到生成系统的影响。研究了两种类型的时间感知提示:(1)在自然语言句子中编码文档时间戳的文本提示;以及(2)将时间戳转换为连续向量的线性提示。为了探索对未来数据点的外推,我们进一步引入了一个新的数据到文本生成数据集TempWikiBio,它包含了来自英文维基百科的超过400万个按时间顺序排列的传记文章修订版,每个修订版都与结构化的个人资料配对。通过TempWikiBio上的数据到文本生成,内容传输数据集上的文本到文本生成以及XSum上的摘要,我们表明编码器上的线性提示和文本提示提高了所有数据集的生成质量。根据人类评估和敏感性分析,尽管在测试从较晚时间提取的数据时性能下降较少,但线性提示更多地关注非时间信息,并且对给定的时间戳不太敏感。同时,文本提示建立了给定时间戳和输出日期之间的关联,从而在输出中产生更多的实际时间信息。

英文摘要:

In this paper, we study the effects of incorporating timestamps, such as document creation dates, into generation systems. Two types of time-aware prompts are investigated: (1) textual prompts that encode document timestamps in natural language sentences; and (2) linear prompts that convert timestamps into continuous vectors. To explore extrapolation to future data points, we further introduce a new data-to-text generation dataset, TempWikiBio, containing more than 4 millions of chronologically ordered revisions of biographical articles from English Wikipedia, each paired with structured personal profiles. Through data-to-text generation on TempWikiBio, text-to-text generation on the content transfer dataset, and summarization on XSum, we show that linear prompts on encoder and textual prompts improve the generation quality on all datasets. Despite having less performance drop when testing on data drawn from a later time, linear prompts focus more on non-temporal information and are less sensitive to the given timestamps, according to human evaluations and sensitivity analyses. Meanwhile, textual prompts establish the association between the given timestamps and the output dates, yielding more factual temporal information in the output.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑