以少优化对齐：利用数据增强实现个性化评估
Optimizing Alignment with Less: Leveraging Data Augmentation for Personalized Evaluation
AI总结：
针对个性化评估中数据稀缺、开源LLM难适配人类偏好的问题，提出从有限数据中筛选有效样本的数据增强技术，在数学推理任务上显著提升与参考裁判的皮尔逊相关系数。
摘要：
大语言模型（LLM）的自动评估是当前的热门研究方向；然而，判断与评估任务往往具有主观性，且受多种因素影响，使得模型适配颇具挑战。尽管诸多研究已证明前沿专有LLM相较于人类评估者的能力，但这类模型通常难以随时间适配参考评估者的偏好，而这正是实现个性化判断的必要条件。此外，大量研究尝试将开源LLM用作裁判或评估器，但这些工作往往忽视了数据稀缺场景下的局限性。个性化判断本质上与有限数据场景相关，这类场景在诸多现实问题中十分常见。本研究提出一种数据增强技术，用于从有限数据中筛选更有效的样本，从而使开源LLM与人类偏好对齐。在数学推理评估任务中，本方法相较于基线模型，与参考裁判的皮尔逊相关系数提升约7%，相较于基础模型Llama3.1-8B-Instruct提升30%，证明通过增强筛选更有效的偏好数据，本方法的表现能够超越基线方法。
英文摘要：
Automatic evaluation by large language models (LLMs) is a prominent topic today; however, judgment and evaluation tasks are often subjective and influenced by various factors, making adaptation challenging. While many studies demonstrate the capabilities of state-of-the-art proprietary LLMs in comparison to human evaluators, they often struggle to adapt to reference evaluators over time, a requirement for achieving personalized judgment. Additionally, numerous works have attempted to apply open LLMs as judges or evaluators, but these efforts frequently overlook the limitations of working with scarce data. Personalized judgment is inherently associated with limited data scenarios, which are common in many real-world problems. Our work aims to present a data augmentation technique to select a more effective sample from limited data in order to align an open LLM with human preference. Our work achieves approximately 7% improvements in Pearson correlation with a reference judge over the baseline,and 30% improvement over the base model (Llama3.1-8B-Instruct) in the mathematical reasoning evaluation task. demonstrating that augmenting selecting more effective preference data enables our approach to surpass baseline methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
