发表机构
University of Pisa（比萨大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文从动力系统视角揭示图Transformer去噪器随深度增加而耗散导致梯度消失与表示坍缩，提出稳定非耗散置换等变GT及阻尼机制，实验证明非耗散动力学对深度图生成性能的关键作用。
AI 中文摘要
图生成模型日益依赖图Transformer（GT）来捕捉节点和边之间的复杂依赖关系。虽然更深的架构应提供更强的表达能力和更广的感受野，但其有效性可能随深度增加而下降：重复的自注意力会逐步压缩节点表示，阻碍信息流动和梯度传播。我们从动力系统的视角分析这一现象，重点关注去噪器的谱动力学如何影响图生成。我们表明，标准GT去噪器随着深度增加而变得越来越耗散，导致梯度消失和表示坍缩。为了隔离这些动力学的影响，我们构建了一个置换等变的GT，其具有固有稳定、非耗散的传输特性。我们还引入了一种阻尼机制，可在非耗散与逐渐收缩的机制之间连续插值，从而能够直接评估耗散对生成的影响。在合成和分子图生成基准上的实验表明，这两种机制之间的差距随深度增加而扩大：非耗散动力学保持了表示多样性和梯度流动，维持了强大的生成性能，而更大的收缩则逐渐损害性能。这些发现将去噪器的动力学机制确定为深度图生成模型的关键设计因素。
英文摘要
Graph generative models increasingly rely on Graph Transformers (GT) to capture complex dependencies among nodes and edges. While deeper architectures should provide greater expressive capacity and a broader receptive field, their effectiveness can decline with depth: repeated self-attention progressively contracts node representations, impeding information flow and gradient propagation. We analyse this phenomenon from a dynamical systems perspective, focusing on how the denoiser's spectral dynamics affect graph generation. We show that standard GT denoisers become increasingly dissipative as depth grows, leading to vanishing gradients and representation collapse. To isolate the effect of these dynamics, we construct a permutation-equivariant GT with inherently stable, non-dissipative transport. We also introduce a damping mechanism that continuously interpolates between non-dissipative and increasingly contractive regimes, enabling a direct assessment of how dissipation influences generation. Experiments on synthetic and molecular graph generation benchmarks show that the gap between these regimes widens with depth: non-dissipative dynamics preserve representation diversity and gradient flow, sustaining strong generative performance, whereas greater contraction progressively impairs it. These findings identify the denoiser's dynamical regime as a key design factor for deep graph generative models.
