arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.09397cs.CLcs.AIcs.CYcs.LGcs.NE
收藏

大型 GPT 模型能否在语言表征中发现道德维度?句子嵌入的拓扑学研究

Do Large GPT Models Discover Moral Dimensions in Language Representations? A Topological Study Of Sentence Embeddings

Stephen Fitz

AI总结:

本文通过拓扑学方法分析 GPT-3.5 句子嵌入,发现其表征空间可分解为公平与不公平两个子流形,证明语言模型在训练中形成了道德维度。

摘要:

随着大型语言模型被部署到日益与人类社会融合的人工智能系统中,研究其内部结构变得比以往任何时候都更加重要。GPT-3.5 等大型语言模型的高层次能力在很大程度上源于它们在数万亿词元的预训练过程中从原始文本数据中归纳出的信息丰富的语言表征。这些嵌入存在于数千维的向量空间中,其处理涉及多个向量空间之间的映射,参数总量达到万亿级。此外,这些语言表征是通过梯度优化归纳得到的,形成了一个难以解释的黑箱系统。本文考察了 Chat-GPT 基础语言模型“大脑”中神经元活动的拓扑结构,并参照表示公平性概念的度量对其进行分析。我们开发了一种新颖的方法来可视化 GPT 的道德维度。我们首先计算一个受社会心理学文献启发的公平性度量,以识别通常影响人类公平性评估的因素,如合法性、需求和责任。随后,我们使用一个低维单纯复形来概括流形的形状,该复形的拓扑结构由该度量导出。我们用与该公平性度量相关的热图为其着色,从而生成高维句子流形的人类可读可视化。我们的结果表明,基于 GPT-3.5 的句子嵌入可以分解为两个子流形,分别对应公平和不公平的道德判断。这表明基于 GPT 的语言模型在其表征空间中发展出了道德维度,并在训练过程中归纳出了对公平性的理解。

英文摘要:

As Large Language Models are deployed within Artificial Intelligence systems, that are increasingly integrated with human society, it becomes more important than ever to study their internal structures. Higher level abilities of LLMs such as GPT-3.5 emerge in large part due to informative language representations they induce from raw text data during pre-training on trillions of words. These embeddings exist in vector spaces of several thousand dimensions, and their processing involves mapping between multiple vector spaces, with total number of parameters on the order of trillions. Furthermore, these language representations are induced by gradient optimization, resulting in a black box system that is hard to interpret. In this paper, we take a look at the topological structure of neuronal activity in the "brain" of Chat-GPT's foundation language model, and analyze it with respect to a metric representing the notion of fairness. We develop a novel approach to visualize GPT's moral dimensions. We first compute a fairness metric, inspired by social psychology literature, to identify factors that typically influence fairness assessments in humans, such as legitimacy, need, and responsibility. Subsequently, we summarize the manifold's shape using a lower-dimensional simplicial complex, whose topology is derived from this metric. We color it with a heat map associated with this fairness metric, producing human-readable visualizations of the high-dimensional sentence manifold. Our results show that sentence embeddings based on GPT-3.5 can be decomposed into two submanifolds corresponding to fair and unfair moral judgments. This indicates that GPT-based language models develop a moral dimension within their representation spaces and induce an understanding of fairness during their training process.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑