关于大语言模型与对齐的校准
On the Calibration of Large Language Models and Alignment
AI总结：
本文系统研究大语言模型在预训练和对齐训练全过程中的置信度校准，探究参数规模和训练数据对生成、事实性和理解三方面的影响，揭示模型校准状态及训练过程的作用。
摘要：
随着大语言模型吸引越来越多的关注并得到广泛应用，可靠性的挑战也随之而来。置信度校准作为衡量深度模型可靠性的有效分析方法，是评估和提升其可靠性的关键工具。然而，相关研究相对较少。本文系统性地研究了已对齐语言模型在整个构建过程中的校准情况，包括预训练和对齐训练。在每个阶段，我们探究了参数规模和训练数据等不同训练设置对模型校准的影响。为全面评估模型校准，我们从最受关注的三个方面评估模型：生成、事实性和理解。本研究揭示了流行的大语言模型是否校准良好，以及训练过程如何影响模型校准。
英文摘要：
As large language models attract increasing attention and find widespread application, concurrent challenges of reliability also arise at the same time. Confidence calibration, an effective analysis method for gauging the reliability of deep models, serves as a crucial tool for assessing and improving their reliability. However, such investigation has been comparatively underexplored. In this work, we conduct a systematic examination of the calibration of aligned language models throughout the entire construction process, including pretraining and alignment training. At each stage, we investigate how different training settings, such as parameter scales and training data, affect model calibration. To thoroughly assess model calibration, we evaluate models on three most concerned aspects: generation, factuality and understanding. Our work sheds light on whether popular LLMs are well-calibrated and how the training process influences model calibration.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
