arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.17671cs.CLcs.AIq-bio.NC
收藏

大语言模型与大脑中的上下文特征提取层级趋于一致

Contextual Feature Extraction Hierarchies Converge in Large Language Models and the Brain

Columbia University(哥伦比亚大学);The Feinstein Institutes for Medical Research(范斯坦医学研究所);Northwell Health(诺斯韦尔健康中心)

Gavin Mischler, Yinghao Aaron Li, Stephan Bickel, Ashesh D. Mehta, Nima Mesgarani

AI总结:

本研究通过对比多组参数相近的高性能大语言模型,发现模型性能提升时其特征提取层级与大脑更趋同、编码效率更高,证实了上下文信息的关键作用,为类脑语言模型开发提供新方向。

摘要:

人工智能的最新进展引发了人们对大语言模型(LLM)与人类神经处理(尤其是语言理解方面)之间相似性的研究兴趣。尽管已有研究证实LLM与大脑的表征存在相似性,但导致这种趋同的底层计算原理——尤其是在LLM不断演进的背景下——仍不明确。本文中,我们选取了一组参数规模相近、性能各异的高性能LLM,以探究促使其与大脑语言处理机制对齐的因素。我们发现,随着LLM在基准任务上取得更高性能,不仅其“类脑性”会提升(衡量标准为用LLM嵌入预测神经响应时的准确率更高),而且它们的层级特征提取通路与大脑的映射也更紧密,同时完成相同编码所需的层数更少。我们还对比了不同LLM的特征提取通路,发现了高性能模型向相似层级处理机制趋同的新表现。最后,我们证明了上下文信息对于提升模型性能和类脑相似度的重要性。我们的发现揭示了大脑与LLM在语言处理方面的趋同点,为开发更符合人类认知处理的模型提供了新方向。

英文摘要:

Recent advancements in artificial intelligence have sparked interest in the parallels between large language models (LLMs) and human neural processing, particularly in language comprehension. While prior research has established similarities in the representation of LLMs and the brain, the underlying computational principles that cause this convergence, especially in the context of evolving LLMs, remain elusive. Here, we examined a diverse selection of high-performance LLMs with similar parameter sizes to investigate the factors contributing to their alignment with the brain's language processing mechanisms. We find that as LLMs achieve higher performance on benchmark tasks, they not only become more brain-like as measured by higher performance when predicting neural responses from LLM embeddings, but also their hierarchical feature extraction pathways map more closely onto the brain's while using fewer layers to do the same encoding. We also compare the feature extraction pathways of the LLMs to each other and identify new ways in which high-performing models have converged toward similar hierarchical processing mechanisms. Finally, we show the importance of contextual information in improving model performance and brain similarity. Our findings reveal the converging aspects of language processing in the brain and LLMs and offer new directions for developing models that align more closely with human cognitive processing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑