RuleR：通过基于规则的数据回收提升 LLM 可控性
RuleR: Improving LLM Controllability by Rule-based Data Recycling
AI总结：
本文提出 RuleR 数据增强方法，通过按预定义规则改写现有样本回复并追加约束指令来低成本构造 SFT 数据，从而在保持通用指令遵循能力的同时提升 LLM 可控性。
摘要：
大型语言模型（LLM）在对其回复进行精细可控性方面仍然存在不足，而这种可控性对于提升模型性能和用户体验至关重要。然而，为提升 LLM 可控性而构建监督微调（SFT）数据集通常依赖人类专家或专有 LLM，这会带来额外成本。为弥合这一差距，我们提出了 Rule-based Data Recycling（RuleR），这是一种数据增强方法，根据预定义规则将多种约束纳入原始数据样本，从而创建新的训练任务以巩固 LLM 的可控性。RuleR 并非从头创建新数据，而是“回收”现有数据：对其回复进行简单的基于规则的编辑，并在原始指令中追加规则指令。实验结果表明，RuleR 能有效提升 LLM 的可控性，同时保持其通用指令遵循能力。
英文摘要：
Large language models (LLMs) still lack delicate controllability over their responses, which is critical to enhancing their performance and the user experience. However, curating supervised fine-tuning (SFT) datasets to improve LLM controllability usually relies on human experts or proprietary LLMs, which requires additional costs. To bridge this gap, we propose Rule-based Data Recycling (RuleR), a data augmentation method incorporating multiple constraints into the original data samples according to predefined rules, which creates new training tasks to consolidate the controllability of LLMs. Instead of creating new data from scratch, RuleR "recycles" existing data by simply applying rule-based edits to their responses and appending the rule-instructions in their original instructions. Experimental results demonstrate RuleR's effectiveness in improving LLM controllability while maintaining general instruction-following capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
