arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.15040cs.CL
收藏

自然语言生成的主动学习

Active Learning for Natural Language Generation

IBM Research AI(IBM研究院人工智能部门)

Yotam Perlitz, Ariel Gera, Michal Shmueli-Scheuer, Dafna Sheinwald, Noam Slonim, Liat Ein-Dor

AI总结:

针对自然语言生成标注数据稀缺问题,首次系统研究主动学习在生成任务中的应用,发现现有策略性能不稳定,并分析了分类与生成场景差异,为探索新方法提供了动机。

摘要:

自然语言生成(NLG)领域由于人工标注过程极其昂贵且耗时,面临着标注数据严重短缺的问题。应对这一问题的一种自然方法是主动学习(AL),这是一种众所周知的机器学习技术,通过选择性地挑选最具信息量的样本进行标注来提高标注效率。然而,尽管主动学习在文本分类领域已得到充分研究,其在自然语言生成中的应用在很大程度上仍未被探索。在本文中,我们首次对自然语言生成的主动学习进行了系统性研究,考虑了多种不同的任务和多种领先的选择策略,并利用了一个强大的指令微调模型。我们的结果表明,现有主动学习策略的性能并不一致,在某些情况下超过了随机选择样本的基线,但在其他情况下则不然。我们强调了分类与生成场景之间的一些显著差异,并分析了现有主动学习策略的选择行为。我们的发现激励了探索将主动学习应用于生成任务的新方法。

英文摘要:

The field of Natural Language Generation (NLG) suffers from a severe shortage of labeled data due to the extremely expensive and time-consuming process involved in manual annotation. A natural approach for coping with this problem is active learning (AL), a well-known machine learning technique for improving annotation efficiency by selectively choosing the most informative examples to label. However, while AL has been well-researched in the context of text classification, its application to NLG remains largely unexplored. In this paper, we present a first systematic study of active learning for NLG, considering a diverse set of tasks and multiple leading selection strategies, and harnessing a strong instruction-tuned model. Our results indicate that the performance of existing AL strategies is inconsistent, surpassing the baseline of random example selection in some cases but not in others. We highlight some notable differences between the classification and generation scenarios, and analyze the selection behaviors of existing AL strategies. Our findings motivate exploring novel approaches for applying AL to generation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑