arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2508.13633cs.LG
收藏

Text2Weight:桥接自然语言与神经网络权重空间

Text2Weight: Bridging Natural Language and Neural Network Weight Spaces

Deep Interdisciplinary Intelligence Lab(深度跨学科智能实验室);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The University of Cambridge(剑桥大学);Zhejiang University(浙江大学);Institute of Deep Perception Technology(深度感知技术研究所);Jiangsu Industrial Technology Research Institute(江苏省工业技术研究所)

Bowen Tian, Wenshuo Chen, Zexi Li, Songning Lai, Jiemin Wu, Yutao Yue

AI总结:

本文提出 T2W 扩散 Transformer 框架,以自然语言描述为条件生成任务特定神经网络权重,通过分层参数处理、CLIP 文本嵌入先验注意力及权重空间对抗训练提升泛化能力,并开源文本-权重对数据集。

摘要:

我们距离自动生成神经网络究竟还有多远?尽管神经网络权重生成展现出前景,当前方法在泛化到未见任务和实际应用探索方面仍面临困难。为解决这一问题,我们提出 T2W,一个以自然语言描述为条件生成任务特定权重的扩散 Transformer 框架。T2W 将网络参数分层处理为统一块,通过先验注意力机制整合来自 CLIP 的文本嵌入,并采用带权重空间增强的对抗训练来提升泛化能力。在 Cifar100、Caltech256 和 TinyImageNet 上的实验表明,T2W 能够为未见任务生成高质量权重,优于基于优化的初始化方法,并支持权重增强和文本引导模型融合等新应用。我们的工作将文本语义与权重空间动态连接起来,并提供一个开源的文本-权重对数据集,推动了生成模型在神经网络参数合成中的实用性。代码已在 Github 上公开。

英文摘要:

How far are we really from automatically generating neural networks? While neural network weight generation shows promise, current approaches struggle with generalization to unseen tasks and practical application exploration. To address this, we propose T2W, a diffusion transformer framework that generates task-specific weights conditioned on natural language descriptions. T2W hierarchically processes network parameters into uniform blocks, integrates text embeddings from CLIP via a prior attention mechanism, and employs adversarial training with weight-space augmentation to enhance generalization. Experiments on Cifar100, Caltech256, and TinyImageNet demonstrate T2W's ability to produce high-quality weights for unseen tasks, outperforming optimization-based initialization and enabling novel applications such as weight enhancement and text-guided model fusion. Our work bridges textual semantics with weight-space dynamics, supported by an open-source dataset of text-weight pairs, advancing the practicality of generative models in neural network parameter synthesis. Our code is available on Github.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑