arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.14561cs.LGcs.AI
收藏

NNsight与NDIF:开放权重基础模型内部机制的民主化访问

NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals

Northeastern University(东北大学);TU Clausthal(克劳斯塔尔工业大学);University of Hamburg(汉堡大学)
Jaden Fiotto-Kaufman, Alexander R. Loftus, Eric Todd, Jannik Brinkmann, Koyena Pal, Dmitrii Troitskii, Michael Ripa, Adam Belfki, Can Rager, Caden Juang, Aaron … 展开作者

Jaden Fiotto-Kaufman, Alexander R. Loftus, Eric Todd, Jannik Brinkmann, Koyena Pal, Dmitrii Troitskii, Michael Ripa, Adam Belfki, Can Rager, Caden Juang, Aaron Mueller, Samuel Marks, Arnab Sen Sharma, Francesca Lucchetti, Nikhil Prakash, Carla Brodley, Arjun Guha, Jonathan Bell, Byron C. Wallace, David Bau

AI总结:

本文提出协同工作的NNsight与NDIF技术,通过Intervention Graph架构解耦实验设计与模型运行时,为研究者提供透明高效访问超大语言模型内部机制的途径,降低了研究成本与复杂性。

摘要:

我们介绍了NNsight和NDIF,这两种技术协同工作,使得对超大神经网络学习到的表征和计算进行科学研究成为可能。NNsight是一个开源系统,扩展了PyTorch以引入延迟远程执行。National Deep Inference Fabric(NDIF)是一个可扩展的推理服务,它执行NNsight请求,允许用户共享GPU资源和预训练模型。这些技术由Intervention Graph(干预图)实现,这是一种为将实验设计与模型运行时解耦而开发的架构。结合起来,该框架为深度神经网络(如超大语言模型,LLMs)的内部机制提供了透明且高效的访问途径,而无需单独承担托管定制模型的成本或复杂性。我们对机器学习文献进行了定量调查,揭示了大规模AI内部机制研究中日益扩大的差距。我们展示了该框架的设计和使用,通过在巨大模型上支持多种研究方法来弥补这一差距。最后,我们进行了基准测试,以将性能与以前的方法进行比较。代码、文档和教程可在https://nnsight.net/ 获取。

英文摘要:

We introduce NNsight and NDIF, technologies that work in tandem to enable scientific study of the representations and computations learned by very large neural networks. NNsight is an open-source system that extends PyTorch to introduce deferred remote execution. The National Deep Inference Fabric (NDIF) is a scalable inference service that executes NNsight requests, allowing users to share GPU resources and pretrained models. These technologies are enabled by the Intervention Graph, an architecture developed to decouple experimental design from model runtime. Together, this framework provides transparent and efficient access to the internals of deep neural networks such as very large language models (LLMs) without imposing the cost or complexity of hosting customized models individually. We conduct a quantitative survey of the machine learning literature that reveals a growing gap in the study of the internals of large-scale AI. We demonstrate the design and use of our framework to address this gap by enabling a range of research methods on huge models. Finally, we conduct benchmarks to compare performance with previous approaches. Code, documentation, and tutorials are available at https://nnsight.net/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑