通过多智能体系统和组合融合增强大语言模型的价值对齐
Enhancing Value Alignment of LLMs with Multi-agent system and Combinatorial Fusion
AI总结：
本文提出VAS-CFA框架，通过多智能体和组合融合提升大语言模型的价值对齐，实验证明其在标准度量上优于现有方法。
摘要：
对齐大型语言模型（LLMs）与人类价值观是确保可信和安全部署的核心挑战。尽管现有的方法如基于人类反馈的强化学习（RLHF）及其变体已提高了对齐程度，但它们往往依赖单一评估者或狭义定义的奖励信号，限制了其捕捉伦理多元主义的能力。在本文中，我们提出了价值对齐系统使用组合融合分析（VAS-CFA），一种框架，它通过多智能体融合对齐来实现。它实例化多个道德代理，每个代理都经过微调以代表不同的规范视角，并使用CFA以基于排名和得分的方式聚合其输出。这种设计利用代理之间的认知多样性，以减轻多个代理之间的冲突和冗余，产生更符合人类价值观的响应。实证评估表明，VAS-CFA在标准度量上优于单个代理基线和先前聚合方法，显示多代理融合提供了一种稳健且有效的方法，以推动LLMs的价值对齐。
英文摘要：
Aligning large language models (LLMs) with human values is a central challenge for ensuring trustworthy and safe deployment. While existing methods such as Reinforcement Learning from Human Feedback (RLHF) and its variants have improved alignment, they often rely on a single evaluator or narrowly defined reward signals, limiting their ability to capture ethical pluralism. In this work, we propose the Value Alignment System using Combinatorial Fusion Analysis (VAS-CFA), a framework that operationalizes multi-agent fusion alignment. It instantiates multiple moral agents, each fine-tuned to represent a distinct normative perspective, and fuses their outputs using CFA with both rank- and score-based aggregation. This design leverages cognitive diversity, between agents, to mitigate conflicts and redundancies across multiple agents, producing responses that better reflect human values. Empirical evaluation demonstrates that VAS-CFA outperforms both single agent baselines and prior aggregation approaches on standard metrics, showing that multi-agent fusion provides a robust and effective mechanism for advancing value alignment in LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
