TopoNets：具有类脑拓扑结构的高性能视觉与语言模型
TopoNets: High Performing Vision and Language Models with Brain-Like Topography
AI总结：
本文提出TopoLoss损失函数，在保持高性能的同时为视觉与语言模型引入类脑拓扑结构，构建了迄今性能最佳的有监督拓扑模型TopoNets，并验证其具备类脑特性及预测大脑反应的能力。
摘要：
大脑中的神经元组织使得邻近细胞倾向于共享相似功能。AI模型缺乏这种组织，且过去引入拓扑结构的尝试往往导致拓扑与任务性能之间的权衡。在本研究中，我们提出了TopoLoss，这是一种新的损失函数，能够在不显著牺牲任务性能的情况下，促进AI模型中空间组织的拓扑表征。TopoLoss具有高度适应性，可无缝集成到领先模型架构的训练中。我们在视觉模型（ResNet-18、ResNet-50、ViT）和语言模型（GPT-Neo-125M、NanoGPT）上验证了该方法，统称为TopoNets。TopoNets是迄今为止性能最高的有监督拓扑模型，表现出局部特征处理、较低维度和更高效率等类脑特性。TopoNets还能预测大脑中的反应，并复制在大脑视觉和语言皮层中观察到的关键拓扑特征。总之，这项工作建立了一个稳健且可推广的框架，将拓扑结构集成到领先的模型架构中，推动了更紧密模拟人脑计算策略的高性能模型的发展。
英文摘要：
Neurons in the brain are organized such that nearby cells tend to share similar functions. AI models lack this organization, and past efforts to introduce topography have often led to trade-offs between topography and task performance. In this work, we present TopoLoss, a new loss function that promotes spatially organized topographic representations in AI models without significantly sacrificing task performance. TopoLoss is highly adaptable and can be seamlessly integrated into the training of leading model architectures. We validate our method on both vision (ResNet-18, ResNet-50, ViT) and language models (GPT-Neo-125M, NanoGPT), collectively TopoNets. TopoNets are the highest-performing supervised topographic models to date, exhibiting brain-like properties such as localized feature processing, lower dimensionality, and increased efficiency. TopoNets also predict responses in the brain and replicate the key topographic signatures observed in the brain's visual and language cortices. Together, this work establishes a robust and generalizable framework for integrating topography into leading model architectures, advancing the development of high-performing models that more closely emulate the computational strategies of the human brain.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
