发表机构
Rice University; University of North Carolina at Chapel Hill; Rutgers University（莱斯大学; 北卡罗来纳大学教堂山分校; 罗格斯大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究通过评估3个LLM、11种压缩方法，发现LLM压缩会引发知识保留、置信度及偏见的非对称变化，聚合指标无法捕捉，需对压缩模型开展细粒度评估。
AI 中文摘要
大语言模型（LLM）压缩可降低部署成本，但困惑度、准确率等标准聚合指标常掩盖潜在的行为变化。本研究系统评估3个LLM、11种压缩方法，探究压缩对知识保留、模型置信度和社会偏见的影响。研究发现，压缩会不成比例地降低头部知识的相对保留率，使其低于尾部知识；压缩模型对新丢失的知识仍对错误答案保持相当高的置信度；稳定的聚合偏见分数会掩盖不同人口统计亚群中刻板印象偏好的显著相反变化。这些发现揭示了聚合性能指标无法捕捉的非对称行为变化，强调部署前需对压缩模型进行细粒度评估。
英文摘要
Large language models (LLMs) compression reduces deployment costs, but standard aggregate metrics like perplexity and accuracy often mask underlying behavioral shifts. In this work, we systematically evaluate 3 LLMs across 11 compression methods to investigate the effects of compression on knowledge retention, model confidence, and social bias. We find that compression disproportionately reduces the relative retention of head knowledge compared to tail knowledge. Furthermore, compressed models often remain substantially confident in their incorrect answers on newly lost knowledge. Finally, we demonstrate that stable aggregate bias scores can conceal substantial, opposing shifts in stereotypical preferences across demographic subgroups. Together, these findings reveal asymmetric behavioral changes that aggregate performance measures fail to capture, highlighting the need for granular evaluation of compressed models before deployment.
