使大语言模型与人类对齐：一项综述
Aligning Large Language Models with Human: A Survey
AI总结：
该综述针对大语言模型易误解指令、生成偏见或幻觉内容的问题，从数据收集、训练方法、模型评估三方面系统梳理LLM与人类对齐的技术，并指明未来研究方向，为相关研究提供参考资源。
摘要：
在海量文本语料上训练的大语言模型（Large Language Models, LLMs）已成为应对各类自然语言处理（Natural Language Processing, NLP）任务的主流方案。尽管性能显著，这类模型仍存在若干局限，例如误解人类指令、生成存在潜在偏见的内容，或是出现事实错误（幻觉）信息。因此，让LLMs与人类预期对齐已成为研究界的热门方向。本综述全面梳理了这类对齐技术，涵盖以下方面：（1）数据收集：为LLM对齐有效收集高质量指令的方法，包括使用NLP基准、人工标注，以及借助能力更强的LLMs。（2）训练方法：详细回顾当前用于LLM对齐的主流训练方法，涵盖监督微调、在线与离线人类偏好训练，以及参数高效训练机制。（3）模型评估：评估这些经人类对齐的LLMs效果的方法，提出多维度的评估思路。最后，我们整理并提炼了研究发现，指明了该领域若干有前景的未来研究方向。因此，本综述可为所有致力于理解并推进LLMs对齐、使其更好地适配人类导向任务与预期的研究者提供宝贵资源。相关的最新论文汇总GitHub链接为：https://github.com/GaryYufei/AlignLLMHumanSurvey。
英文摘要：
Large Language Models (LLMs) trained on extensive textual corpora have emerged as leading solutions for a broad array of Natural Language Processing (NLP) tasks. Despite their notable performance, these models are prone to certain limitations such as misunderstanding human instructions, generating potentially biased content, or factually incorrect (hallucinated) information. Hence, aligning LLMs with human expectations has become an active area of interest within the research community. This survey presents a comprehensive overview of these alignment technologies, including the following aspects. (1) Data collection: the methods for effectively collecting high-quality instructions for LLM alignment, including the use of NLP benchmarks, human annotations, and leveraging strong LLMs. (2) Training methodologies: a detailed review of the prevailing training methods employed for LLM alignment. Our exploration encompasses Supervised Fine-tuning, both Online and Offline human preference training, along with parameter-efficient training mechanisms. (3) Model Evaluation: the methods for evaluating the effectiveness of these human-aligned LLMs, presenting a multifaceted approach towards their assessment. In conclusion, we collate and distill our findings, shedding light on several promising future research avenues in the field. This survey, therefore, serves as a valuable resource for anyone invested in understanding and advancing the alignment of LLMs to better suit human-oriented tasks and expectations. An associated GitHub link collecting the latest papers is available at https://github.com/GaryYufei/AlignLLMHumanSurvey.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
