arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.03047cs.CLcs.AIcs.LG
收藏

基于熵的大型语言模型价值漂移与对齐工作的测量

Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models

Aeris Space Laboratory(Aeris空间实验室);Capitol Technology University(国会理工大学)

Samih Fadli

AI总结:

本文提出基于熵的框架,用于测量大型语言模型中的价值漂移和对齐工作,通过五种行为分类法和熵动态分析,实现对模型伦理熵的实时监控与警报。

摘要:

大型语言模型的安全性通常通过静态基准测试来评估,但关键失败是动态的:在分布偏移下的价值漂移、劫持攻击以及部署中的对齐缓慢退化。基于最近的智能第二定律,该定律将伦理熵视为一个状态变量,除非通过对齐工作抵消,否则会增加。我们使这一框架适用于大型语言模型。我们定义了一个五种行为分类法,训练一个分类器来从模型转录中估计伦理熵S(t),并在压力测试中测量基础模型和指令微调变体的四个前沿模型的熵动态。基础模型显示出持续的熵增长,而微调变体抑制了漂移并减少了伦理熵约80%。从这些轨迹中,我们估计了有效的对齐工作率gamma_eff,并将S(t)和gamma_eff嵌入监控流程中,当熵漂移超过稳定性阈值时会触发警报,从而实现对价值漂移的实时监控。

英文摘要:

Large language model safety is usually assessed with static benchmarks, but key failures are dynamic: value drift under distribution shift, jailbreak attacks, and slow degradation of alignment in deployment. Building on a recent Second Law of Intelligence that treats ethical entropy as a state variable which tends to increase unless countered by alignment work, we make this framework operational for large language models. We define a five-way behavioral taxonomy, train a classifier to estimate ethical entropy S(t) from model transcripts, and measure entropy dynamics for base and instruction-tuned variants of four frontier models across stress tests. Base models show sustained entropy growth, while tuned variants suppress drift and reduce ethical entropy by roughly eighty percent. From these trajectories we estimate an effective alignment work rate gamma_eff and embed S(t) and gamma_eff in a monitoring pipeline that raises alerts when entropy drift exceeds a stability threshold, enabling run-time oversight of value drift.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑