arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.16043cs.CL
收藏

超越漂绿:大语言模型的生态合理决策在压力下是否站得住脚?

Beyond the greenwash: do LLMs' ecologically sound decisions hold up under pressure?

Bioaligned Labs(生物对齐实验室)

Trent R. Northen, Mingxun Wang

AI总结:

该研究针对大语言模型的生态决策偏差,构建校准阶梯评估生物对齐及偏差鲁棒性,发现训练剂量与鲁棒性反转,且系统提示无法完全消除权重安装的生态倾向。

摘要:

人类正越来越多地在社会各领域的决策过程中使用大语言模型(LLMs)。遗憾的是,模型存在偏差,且这些偏差极难测量,尤其是对于权重封闭的模型而言。例如,“漂绿”模型可能产生经强化学习得到的响应,但在压力情境下会回归其固有的偏差响应。因此,需要开展能够测量针对生物系统的相关偏差(我们将其称为“生物对齐”)以及这些偏差鲁棒性的评估。由于行为评分无法证明其追踪的是内在倾向的深度而非表面,我们构建了一个已知剂量校准阶梯:在同一个30B混合专家(MoE)基础模型上,通过配方对耦合人类/生物圈/AI福利语料库进行分级训练剂量的三次持续预训练(CPT)分支,加上未训练的基础模型作为锚点,所有分支均保留能力且已公开。首先,该倾向在每个剂量下都存在:在不可逆决策情境的操作压力阶梯上,基础模型在64-77%的尾部情境中出现缺陷(缺陷率AUC为0.250),而所有分支均保持稳定(AUC为0.004-0.086)。其次,存在过度安装反转:在各分支中,鲁棒性与构建时的训练剂量呈相反顺序(最重的安装缺陷最多),且该反转在自由文本审议、轴解耦以及对提示的表面人设的抵抗中均会重现。第三,现有的行为指标因此在行为上表现强劲,但对剂量不敏感。对于权重封闭评估而言,关键的是,行为评分本身具有表面鲁棒性:“说环保/讲实际”的系统提示将基础模型变为近乎完全的缺陷者(AUC从0.250升至0.973),但无法伪造或完全消除权重安装的倾向(最鲁棒分支的AUC从0.004降至0.068)。

英文摘要:

Humans are increasingly using large language models (LLMs) in decision making processes throughout society. Unfortunately, models have biases, and these can be very difficult to measure, especially in closed-weight models. For example, 'green washed' models may produce reinforcement learned responses but, in pressure situations, revert to their innate biased responses. Therefore, evaluations are needed that can measure relevant biases towards biosystems, which we term 'bioalignment', and the robustness of these biases. Since nothing in a behavioral score certifies that it tracks the depth of an installed disposition rather than its surface, we constructed a known-dose calibration ladder: three continued-pretraining (CPT) arms with training dose graded by recipe on the same 30B mixture-of-experts base model with a coupled human/biosphere/AI-welfare corpus, plus the untrained base as anchor, all capability-preserving and publicly released. First, the disposition installs at every dose: on an operational-pressure ladder over irreversible decision scenarios, the base model defects on 64-77% of tail scenarios (breaking-rate AUC 0.250) while every arm holds (AUC 0.004-0.086). Second, an over-install inversion: among the arms, robustness orders opposite to construction-time training dose (the heaviest install breaks most), and the inversion reproduces under free-text deliberation, under axis-decoupling, and in resistance to a prompted veneer persona. Third, the incumbent behavioral metric is therefore behaviorally strong but dose-blind. Critically for closed-weight evaluation, the behavioral score itself is veneer-robust: a talk-green/act-pragmatic system prompt turns the base model into a near-total defector (AUC 0.250 to 0.973) but cannot counterfeit or fully remove a weight-installed disposition (0.004 to 0.068 for the most robust arm).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑