arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.02569cs.CLcs.AIcs.CYcs.HC
收藏

多即是多:大型语言模型中的加法偏差

More is More: Addition Bias in Large Language Models

University of Trento(特伦托大学);SISSA/ICTP(国际高等研究学院/国际理论物理中心);Pi School(派学校)

Luca Santagata, Cristiano De Nobili

AI总结:

本文通过受控实验发现GPT-3.5 Turbo、Llama 3.1等多种大型语言模型普遍存在类似人类的加法偏差,在修改任务中显著偏好添加而非删除,并指出该偏差可能带来资源浪费和经济成本,需在模型开发与应用中加以考虑。

摘要:

在本文中,我们研究了大型语言模型(LLMs)中加法偏差的存在,并将其与人类观察到的认知偏差进行类比,即个体倾向于偏好加法改变而非减法改变。通过一系列受控实验,我们测试了多种大型语言模型,包括GPT-3.5 Turbo、Claude 3.5 Sonnet、Mistral、MathΣtral和Llama 3.1,在旨在衡量其加法修改与减法修改倾向的任务上的表现。我们的研究结果表明,所有测试模型都显著偏好加法改变。例如,在回文创建任务中,Llama 3.1在97.85%的情况下选择添加字母而非删除字母。同样,在乐高塔平衡任务中,GPT-3.5 Turbo在76.38%的情况下选择添加一块积木而非移除一块。在文本摘要任务中,当被要求改进自己或他人的写作时,Mistral 7B在59.40%至75.10%的情况下生成了更长的摘要。这些结果表明,与人类类似,大型语言模型表现出明显的加法偏差,这可能在大型语言模型大规模应用时产生影响。加法偏差可能增加资源使用和环境影响,由于过度消耗和浪费而导致更高的经济成本。在大型语言模型的开发和应用中应考虑这种偏差,以确保平衡且高效的问题解决方法。

英文摘要:

In this paper, we investigate the presence of additive bias in Large Language Models (LLMs), drawing a parallel to the cognitive bias observed in humans where individuals tend to favor additive over subtractive changes. Using a series of controlled experiments, we tested various LLMs, including GPT-3.5 Turbo, Claude 3.5 Sonnet, Mistral, Math$Σ$tral, and Llama 3.1, on tasks designed to measure their propensity for additive versus subtractive modifications. Our findings demonstrate a significant preference for additive changes across all tested models. For example, in a palindrome creation task, Llama 3.1 favored adding letters 97.85% of the time over removing them. Similarly, in a Lego tower balancing task, GPT-3.5 Turbo chose to add a brick 76.38% of the time rather than remove one. In a text summarization task, Mistral 7B produced longer summaries in 59.40% to 75.10% of cases when asked to improve its own or others' writing. These results indicate that, similar to humans, LLMs exhibit a marked additive bias, which might have implications when LLMs are used on a large scale. Addittive bias might increase resource use and environmental impact, leading to higher economic costs due to overconsumption and waste. This bias should be considered in the development and application of LLMs to ensure balanced and efficient problem-solving approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑