arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.13664cs.CLcs.AI
收藏

利用隐式反馈改进问题生成

Using Implicit Feedback to Improve Question Generation

Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院);INESC-ID(葡萄牙信息与通信技术研究院);Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所)

Hugo Rodrigues, Eric Nyberg, Luisa Coheur

AI总结:

本文提出 GEN 系统,利用用户选择与修正两类隐式反馈迭代生成、评分并排序问题模式,实验表明其在 top 5、10、20 设置下较无学习版本提升至少 10%。

摘要:

问题生成(Question Generation,QG)是自然语言处理(Natural Language Processing,NLP)中的一项任务,旨在从文本中自动生成问题。许多应用都可以从自动生成的问题中受益,但通常有必要对这些问题进行筛选整理,方式包括选择或编辑。这项任务本身具有信息价值,但它通常是在生成之后完成的,因此相关投入被浪费了。此外,大多数现有系统无法轻松地将这种反馈重新纳入自身。在本文中,我们提出了一个名为 GEN 的系统,它能够从此类(隐式)反馈中学习。该系统采用基于模式的方法,以少量句子/问题对作为输入,创建模式,随后将这些模式应用于未见过的新句子。每个生成的问题在经过用户修正后,都会在下一次迭代中作为新的种子使用,因此每次都会创建更多模式。我们还利用用户所做的修正来为模式评分,从而对生成的问题进行排序。结果表明,在考虑排名前 5、前 10 和前 20 的问题时,与不具备学习能力的版本相比,GEN 能够通过从两个层面的隐式反馈中学习而获得改进。根据所使用的指标和策略,提升幅度从 10% 起步,最高可达更高水平。

英文摘要:

Question Generation (QG) is a task of Natural Language Processing (NLP) that aims at automatically generating questions from text. Many applications can benefit from automatically generated questions, but often it is necessary to curate those questions, either by selecting or editing them. This task is informative on its own, but it is typically done post-generation, and, thus, the effort is wasted. In addition, most existing systems cannot incorporate this feedback back into them easily. In this work, we present a system, GEN, that learns from such (implicit) feedback. Following a pattern-based approach, it takes as input a small set of sentence/question pairs and creates patterns which are then applied to new unseen sentences. Each generated question, after being corrected by the user, is used as a new seed in the next iteration, so more patterns are created each time. We also take advantage of the corrections made by the user to score the patterns and therefore rank the generated questions. Results show that GEN is able to improve by learning from both levels of implicit feedback when compared to the version with no learning, considering the top 5, 10, and 20 questions. Improvements go up from 10%, depending on the metric and strategy used.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑