Dr Genre：基于解耦 LLM 反馈的强化学习用于通用文本改写
Dr Genre: Reinforcement Learning from Decoupled LLM Feedback for Generic Text Rewriting
AI总结：
本文提出 Dr Genre 通用文本改写框架，通过构建 ChatRewrite 对话改写数据集并结合解耦奖励学习与任务特定加权，在事实性、风格性和对话式改写任务上同时提升指令遵循、连贯性与简洁性。
摘要：
通用文本改写是一种流行的大语言模型（LLM）应用，涵盖多种现实世界任务，如风格迁移、事实修正和邮件编辑。这些任务在改写目标上各不相同（例如事实一致性 vs. 语义保留），因此开发一个在所有维度上都表现出色的统一模型具有挑战性。现有方法通常专注于单一任务或特定目标，限制了其泛化能力。在本工作中，我们引入了一个精通事实性、风格性和对话式改写任务的通用模型。为了模拟现实世界用户的改写请求，我们构建了一个对话式改写数据集 ChatRewrite，该数据集利用 LLM 从原始邮件中生成听起来“自然”的指令。结合其他流行的改写数据集，包括用于事实性改写任务的 LongFact 和用于风格性改写任务的 RewriteLM，形成了一个用于训练和评估通用改写模型的广泛基准。为了与任务特定目标对齐，我们提出了 Dr Genre，一个用于通用改写的解耦奖励学习框架，该框架利用面向目标的奖励模型并进行任务特定加权。评估表明，该方法在所有目标任务上都能提供更高质量的改写，改善了包括指令遵循（一致性）、内部一致性（连贯性）和最少不必要编辑（简洁性）在内的目标。
英文摘要：
Generic text rewriting is a prevalent large language model (LLM) application that covers diverse real-world tasks, such as style transfer, fact correction, and email editing. These tasks vary in rewriting objectives (e.g., factual consistency vs. semantic preservation), making it challenging to develop a unified model that excels across all dimensions. Existing methods often specialize in either a single task or a specific objective, limiting their generalizability. In this work, we introduce a generic model proficient in factuality, stylistic, and conversational rewriting tasks. To simulate real-world user rewrite requests, we construct a conversational rewrite dataset, ChatRewrite, that presents ``natural''-sounding instructions, from raw emails using LLMs. Combined with other popular rewrite datasets, including LongFact for the factuality rewrite task and RewriteLM for the stylistic rewrite task, this forms a broad benchmark for training and evaluating generic rewrite models. To align with task-specific objectives, we propose Dr Genre, a Decoupled-reward learning framework for Generic rewriting, that utilizes objective-oriented reward models with a task-specific weighting. Evaluation shows that \approach delivers higher-quality rewrites across all targeted tasks, improving objectives including instruction following (agreement), internal consistency (coherence), and minimal unnecessary edits (conciseness).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
