arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.00307cs.LGcs.AIcs.CL
收藏

ABC Align:面向安全性与准确性的大语言模型对齐

ABC Align: Large Language Model Alignment for Safety & Accuracy

Australian Broadcasting Corporation(澳大利亚广播公司)

Gareth Seneque, Lap-Hang Ho, Ariel Kuperman, Nafise Erfanian Saeedi, Jeffrey Molendijk

AI总结:

针对大语言模型难以整合组织标准与偏好的问题,本文提出 ABC Align,融合合成数据、偏好优化和训练后量化,在保持推理能力的同时提升安全性与准确性。

摘要:

大语言模型(LLM)的对齐仍是一个尚未解决的问题。人类偏好具有高度分布性,并且可以在从个体到多样化群体的多个抽象层级上被捕捉。以标准和原则表示的组织偏好,其制定目的是降低声誉风险或履行立法义务。本文提出 ABC Align,这是一种面向 LLM 的新型对齐方法,能够将某大型媒体组织的标准与偏好整合到 LLM 本身之中。我们结合了一组数据与方法,其基础是合成数据生成、偏好优化以及训练后模型量化方面的最新突破。我们的统一方法缓解了偏见并提高了准确性,同时按照标准基准测得,还保留了推理能力。

英文摘要:

Alignment of Large Language Models (LLMs) remains an unsolved problem. Human preferences are highly distributed and can be captured at multiple levels of abstraction, from the individual to diverse populations. Organisational preferences, represented by standards and principles, are defined to mitigate reputational risk or meet legislative obligations. In this paper, we present ABC Align, a novel alignment methodology for LLMs that enables integration of the standards and preferences of a large media organisation into the LLM itself. We combine a set of data and methods that build on recent breakthroughs in synthetic data generation, preference optimisation, and post-training model quantisation. Our unified approach mitigates bias and improves accuracy, while preserving reasoning capability, as measured against standard benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑