arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.18320cs.CYcs.AIcs.CL
收藏

认知网络科学揭示GPT-3、ChatGPT和GPT-4中存在反映高中生数学焦虑的偏见

Cognitive network science reveals bias in GPT-3, ChatGPT, and GPT-4 mirroring math anxiety in high-school students

University of Pisa(比萨大学);ISTI - CNR(意大利国家研究委员会信息科学与技术研究所);University of Trento(特伦托大学)

Katherine Abramski, Salvatore Citraro, Luigi Lombardi, Giulio Rossetti, Massimo Stella

AI总结:

本文利用认知网络科学方法分析GPT-3、ChatGPT和GPT-4对数学及STEM学科的感知偏见,发现LLM总体持负面态度且数学最负面,但GPT-4比旧版本和高中生偏见更少、感知更丰富,表明架构进步有望减少有害刻板印象。

摘要:

大语言模型正日益融入我们的生活。因此,理解其输出中存在的偏见至关重要,以避免延续源于我们自身有缺陷思维模式的有害刻板印象。这一挑战要求开发新的基准和方法来量化情感与语义偏见,同时要牢记LLM充当着心理社会镜子,反映社会中普遍存在的观点和倾向。其中一种具有有害负面影响的倾向是全球范围内对数学和STEM学科的焦虑现象。在此,我们通过应用网络科学和认知心理学的方法,研究前沿语言模型(即GPT-3、ChatGPT和GPT-4)对数学和STEM领域的感知。具体而言,我们使用行为forma mentis网络(BFMNs)来理解这些LLM如何将数学和STEM学科与其他概念相关联地构建框架。我们使用通过在一个此前已应用于人类的语言生成任务中探测这三个LLM所获得的数据。我们的研究结果表明,LLM对数学和STEM领域总体持负面感知,其中数学被感知得最为负面。我们观察到三个LLM之间存在显著差异。我们观察到,与较旧版本以及N=159名高中生相比,较新版本(即GPT-4)产生更丰富、更复杂的感知以及更少的负面感知。这些发现表明,LLM架构的进步可能会带来偏见日益减少的模型,甚至或许有朝一日能够帮助减少社会中的有害刻板印象,而非延续它们。

英文摘要:

Large language models are becoming increasingly integrated into our lives. Hence, it is important to understand the biases present in their outputs in order to avoid perpetuating harmful stereotypes, which originate in our own flawed ways of thinking. This challenge requires developing new benchmarks and methods for quantifying affective and semantic bias, keeping in mind that LLMs act as psycho-social mirrors that reflect the views and tendencies that are prevalent in society. One such tendency that has harmful negative effects is the global phenomenon of anxiety toward math and STEM subjects. Here, we investigate perceptions of math and STEM fields provided by cutting-edge language models, namely GPT-3, Chat-GPT, and GPT-4, by applying an approach from network science and cognitive psychology. Specifically, we use behavioral forma mentis networks (BFMNs) to understand how these LLMs frame math and STEM disciplines in relation to other concepts. We use data obtained by probing the three LLMs in a language generation task that has previously been applied to humans. Our findings indicate that LLMs have an overall negative perception of math and STEM fields, with math being perceived most negatively. We observe significant differences across the three LLMs. We observe that newer versions (i.e. GPT-4) produce richer, more complex perceptions as well as less negative perceptions compared to older versions and N=159 high-school students. These findings suggest that advances in the architecture of LLMs may lead to increasingly less biased models that could even perhaps someday aid in reducing harmful stereotypes in society rather than perpetuating them.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑