RIDE：通过重构上下文学习示范样本来增强大型语言模型对齐
RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplars
AI总结：
本文提出RIDE，一种低成本免微调方法，通过重构上下文学习示范样本的风格，平衡事实性与安全性，显著提升了大型语言模型的对齐能力。
摘要：
对齐微调对于确保大型语言模型（LLM）行为符合道德且具有帮助性至关重要。当前的对齐方法需要高质量的标注和大量的训练资源。本文提出了一种低成本的、免微调的方法，利用上下文学习（ICL）来增强LLM对齐。通过对高质量ICL示范的分析，我们发现风格是影响LLM对齐能力的关键因素，并基于这一风格框架明确地重构了ICL样本。此外，我们将重构后的示范结合起来，以在LLM对齐的两个冲突方面——事实性和安全性——之间实现平衡。我们将重构后的样本打包作为提示来触发少样本学习，从而改善LLM对齐。与最佳基线方法相比，在最高平均分为5.00的情况下，我们的方法在Alpaca任务上实现了最高0.10的提升（从4.50到4.60），在Just-eval基准上取得了0.22的增强（从4.34到4.56），并在MT-Bench数据集上实现了最高0.32的改进（从3.53到3.85）。我们在 https://github.com/AnonymousCode-ComputerScience/RIDE 发布了代码和数据。
英文摘要：
Alignment tuning is crucial for ensuring large language models (LLMs) behave ethically and helpfully. Current alignment approaches require high-quality annotations and significant training resources. This paper proposes a low-cost, tuning-free method using in-context learning (ICL) to enhance LLM alignment. Through an analysis of high-quality ICL demos, we identified style as a key factor influencing LLM alignment capabilities and explicitly restyled ICL exemplars based on this stylistic framework. Additionally, we combined the restyled demos to achieve a balance between the two conflicting aspects of LLM alignment--factuality and safety. We packaged the restyled examples as prompts to trigger few-shot learning, improving LLM alignment. Compared to the best baseline approach, with an average score of 5.00 as the maximum, our method achieves a maximum 0.10 increase on the Alpaca task (from 4.50 to 4.60), a 0.22 enhancement on the Just-eval benchmark (from 4.34 to 4.56), and a maximum improvement of 0.32 (from 3.53 to 3.85) on the MT-Bench dataset. We release the code and data at https://github.com/AnonymousCode-ComputerScience/RIDE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
