用于大型语言模型数据标注的直觉模糊集：一种并排偏好标注的新方法
Intuitionistic Fuzzy Sets for Large Language Model Data Annotation: A Novel Approach to Side-by-Side Preference Labeling
AI总结：
本文提出基于直觉模糊集（IFS）的大模型数据标注框架，通过隶属度、非隶属度和犹豫度建模人类偏好与不确定性，显著提升了标注一致性与数据质量，使下游任务胜率提高12.3%。
摘要：
人类偏好数据的质量对于训练和评估大型语言模型（LLMs）至关重要，特别是在人类反馈强化学习（RLHF）和直接偏好优化（DPO）场景中。传统的并排（SBS）标注方法常常难以应对内在的不确定性、标注者分歧以及偏好判断的复杂性。本文引入了一种基于直觉模糊集（IFS）的新颖框架，用于建模和聚合LLM数据标注任务中的人类偏好。我们的方法不仅捕捉了偏好程度，还通过隶属度、非隶属度和犹豫度捕捉了人类判断中固有的不确定性和犹豫。我们提出了一种基于IFS的标注协议，能够实现更细致的偏好建模，开发了处理标注者分歧的聚合方法，并引入了用于偏好数据评估的质量指标。在多个数据集上的实验验证表明，与传统的二元和李克特量表方法相比，我们基于IFS的方法显著提高了标注一致性，减少了标注者疲劳，并生成了更高质量的偏好数据。所得的偏好数据集提升了下游任务中的模型性能，与基线模型相比胜率提高了12.3%，标注时间减少了15.7%。我们的框架为处理人类偏好标注中的不确定性提供了一种有原则的方法，并为大规模LLM训练提供了实际益处。
英文摘要：
The quality of human preference data is crucial for training and evaluating large language models (LLMs), particularly in reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO) scenarios. Traditional side-by-side (SBS) annotation approaches often struggle with inherent uncertainty, annotator disagreement, and the complexity of preference judgments. This paper introduces a novel framework based on intuitionistic fuzzy sets (IFS) for modeling and aggregating human preferences in LLM data annotation tasks. Our approach captures not only the degree of preference but also the uncertainty and hesitation inherent in human judgment through membership, non-membership, and hesitation degrees. We propose an IFS-based annotation protocol that enables more nuanced preference modeling, develops aggregation methods for handling annotator disagreement, and introduces quality metrics for preference data assessment. Experimental validation on multiple datasets demonstrates that our IFS-based approach significantly improves annotation consistency, reduces annotator fatigue, and produces higher-quality preference data compared to traditional binary and Likert-scale methods. The resulting preference datasets lead to improved model performance in downstream tasks, with 12.3\% improvement in win-rate against baseline models and 15.7\% reduction in annotation time. Our framework provides a principled approach to handling uncertainty in human preference annotation and offers practical benefits for large-scale LLM training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
