arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.09205cs.CLcs.AI
收藏

ReadCtrl:通过可读性控制的指令学习实现个性化文本生成

ReadCtrl: Personalizing text generation with readability-controlled instruction learning

Manning College of Information and Computer Sciences, University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校曼宁信息与计算机科学学院);Department of Medicine, University of Massachusetts Medical School(马萨诸塞大学医学院医学系);Miner School of Computer and Information Sciences, University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校迈纳计算机与信息科学学院);Center for Healthcare Organization and Implementation Research, VA Bedford Health Care(弗吉尼亚州贝德福德医疗保健中心医疗保健组织与实施研究中心)

Hieu Tran, Zonghai Yao, Lingxi Li, Hong Yu

AI总结:

本文提出ReadCtrl方法,通过可读性控制指令学习微调大语言模型,实现接近连续级别的个性化文本复杂度生成,并在人类和自动评估中显著优于GPT-4等基线模型。

摘要:

以用户可读性为条件的内容生成是个性化的重要应用。在大语言模型(LLMs)时代,基于LLMs的可读性控制文本生成变得越来越重要。本文提出了一种名为“可读性控制指令学习(ReadCtrl)”的新方法,旨在对LLMs进行指令微调,以适应用户的可读性水平。与传统方法主要关注通常划分为高、中、低或专家与外行等级别的分类式可读性调整且效果有限不同,ReadCtrl引入了一个动态框架,使LLMs能够在各种(接近连续水平的)复杂度级别上生成内容,从而增强其在不同应用中的通用性。我们的结果表明,ReadCtrl-Mistral-7B模型显著优于GPT-4和Claude-3等强基线模型,在人类评估中对GPT-4的胜率为52.1%:35.7%。此外,ReadCtrl在自动评估中也表现出显著改进,更好的可读性指标(例如FOG、FKGL)和生成质量指标(例如BLEU、SARI、SummaC-Factuality、UniEval-Consistency和Coherence)证明了这一点。这些结果凸显了ReadCtrl在生成高质量、上下文适当且与目标可读性水平高度契合的输出方面的有效性和稳健性,标志着使用LLMs进行个性化内容生成的重大进步。

英文摘要:

Content generation conditioning on users's readability is an important application for personalization. In an era of large language models (LLMs), readability-controlled text generation based on LLMs has become increasingly important. This paper introduces a novel methodology called "Readability-Controlled Instruction Learning (ReadCtrl)," which aims to instruction-tune LLMs to tailor users' readability levels. Unlike the traditional methods, which primarily focused on categorical readability adjustments typically classified as high, medium, and low or expert and layperson levels with limited success, ReadCtrl introduces a dynamic framework that enables LLMs to generate content at various (near continuous level) complexity levels, thereby enhancing their versatility across different applications. Our results show that the ReadCtrl-Mistral-7B models significantly outperformed strong baseline models such as GPT-4 and Claude-3, with a win rate of 52.1%:35.7% against GPT-4 in human evaluations. Furthermore, Read-Ctrl has shown significant improvements in automatic evaluations, as evidenced by better readability metrics (e.g., FOG, FKGL) and generation quality metrics (e.g., BLEU, SARI, SummaC-Factuality, UniEval-Consistency and Coherence). These results underscore Read-Ctrl's effectiveness and tenacity in producing high-quality, contextually appropriate outputs that closely align with targeted readability levels, marking a significant advancement in personalized content generation using LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑