远距离的幽灵作用:归一化层实现侧信道空间通信
Spooky Action at a Distance: Normalization Layers Enable Side-Channel Spatial Communication
AI总结:
本文发现归一化层可在卷积网络中实现跨空间维度的侧信道通信,使信息超出局部感受野聚合,并提醒在扩散轨迹生成等需限制感受野的任务中谨慎使用。
摘要:
本工作表明,归一化层可以在输入张量的空间维度上促成令人惊讶的通信程度。我们研究了一个采用卷积架构的玩具定位任务,并证明归一化层能够实现一种迭代式消息传递过程,从而允许从局部感受野之外很远的地方聚合信息。我们的结果表明,在基于扩散的轨迹生成等应用中,应谨慎使用归一化层,因为在这些应用中保持空间受限的感受野至关重要。
英文摘要:
This work shows that normalization layers can facilitate a surprising degree of communication across the spatial dimensions of an input tensor. We study a toy localization task with a convolutional architecture and show that normalization layers enable an iterative message passing procedure, allowing information aggregation from well outside the local receptive field. Our results suggest that normalization layers should be employed with caution in applications such as diffusion-based trajectory generation, where maintaining a spatially limited receptive field is crucial.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。