arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.04771cs.CL
收藏

QASE增强的预训练语言模型:面向机器阅读理解的文本生成可控性提升

QASE Enhanced PLMs: Improved Control in Text Generation for MRC

Columbia University(哥伦比亚大学)

Lin Ai, Zheng Hui, Zizhou Liu, Julia Hirschberg

AI总结:

针对机器阅读理解生成式模型生成失控问题,提出QASE模块并集成到生成式预训练语言模型微调中,在不显著增加计算成本的前提下,使模型性能追平最优抽取式方法且超越GPT-4等大模型。

摘要:

为解决机器阅读理解(MRC)生成式模型中生成失控的难题,我们提出问题感知跨度提取(Question-Attended Span Extraction, QASE)模块。该模块在生成式预训练语言模型(PLMs)微调阶段集成,可使模型在MRC任务上达到与最优抽取式方法相当的性能,且优于GPT-4等主流大语言模型,同时计算成本无显著增加。

英文摘要:

To address the challenges of out-of-control generation in generative models for machine reading comprehension (MRC), we introduce the Question-Attended Span Extraction (QASE) module. Integrated during the fine-tuning of pre-trained generative language models (PLMs), QASE enables these PLMs to match SOTA extractive methods and outperform leading LLMs like GPT-4 in MRC tasks, without significant increases in computational costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑