预训练中植根，微调中摇摆：关于LLMs认知偏差起源的案例研究
Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
AI总结：
本文提出一种两步因果实验方法，通过多次微调和交叉微调分离训练随机性、预训练与微调对LLMs认知偏差的影响，发现偏差主要由预训练塑造，为评估和缓解偏差提供了新视角。
摘要：
大型语言模型（LLMs）表现出认知偏差——系统性的非理性决策倾向，与人类中看到的类似。先前的工作发现这些偏差在不同模型间存在差异，并且可以通过指令微调被放大。然而，目前尚不清楚这些偏差差异是源于预训练、微调，还是由于训练随机性导致的随机噪声。我们提出了一种两步因果实验方法来分离这些因素。首先，我们使用不同的随机种子多次微调模型，以研究训练随机性如何影响超过30种认知偏差。其次，我们引入了“交叉微调”——在模型之间交换指令数据集以隔离偏差来源。这种交换使用了导致不同偏差模式的数据集，直接测试偏差是否依赖于数据集。我们的发现表明，尽管训练随机性引入了一些变异性，但偏差主要由预训练塑造：具有相同预训练骨干的模型比仅共享微调数据的模型表现出更相似的偏差模式。这些见解表明，理解微调模型中的偏差需要考虑其预训练起源，而不仅仅是微调效果。这一视角可以指导未来的努力，以开发有原则的策略来评估和缓解LLMs中的偏差。
英文摘要：
Large language models (LLMs) exhibit cognitive biases -- systematic tendencies of irrational decision-making, similar to those seen in humans. Prior work has found that these biases vary across models and can be amplified by instruction tuning. However, it remains unclear if these differences in biases stem from pretraining, finetuning, or even random noise due to training stochasticity. We propose a two-step causal experimental approach to disentangle these factors. First, we finetune models multiple times using different random seeds to study how training randomness affects over $30$ cognitive biases. Second, we introduce \emph{cross-tuning} -- swapping instruction datasets between models to isolate bias sources. This swap uses datasets that led to different bias patterns, directly testing whether biases are dataset-dependent. Our findings reveal that while training randomness introduces some variability, biases are mainly shaped by pretraining: models with the same pretrained backbone exhibit more similar bias patterns than those sharing only finetuning data. These insights suggest that understanding biases in finetuned models requires considering their pretraining origins beyond finetuning effects. This perspective can guide future efforts to develop principled strategies for evaluating and mitigating bias in LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
