注意力归一化影响槽注意力中的基数泛化
Attention Normalization Impacts Cardinality Generalization in Slot Attention
AI总结：
本文发现槽注意力模块中注意力归一化方式显著影响其对槽和对象数量的泛化能力，提出将加权平均改为缩放加权求和的简单修改，提升无监督图像分割性能。
摘要：
对象中心的场景分解是计算机视觉和机器人等领域下游任务的重要表示。最近提出的槽注意力模块是一种深度学习组件，它对输入图像执行无监督的对象中心场景分解，该模块已被多个衍生工作用于图像分割和视频中的对象跟踪。它基于一种注意力架构，其中持有对象压缩信息的潜在槽向量关注输入图像中的局部感知特征。在本文中，我们证明了注意力架构中聚合值归一化的设计决策对槽注意力泛化到训练中未见过的更多槽和对象数量的能力有显著影响。我们提出并研究了原始归一化方案的替代方案，这些方案增强了槽注意力对不同槽和对象数量的泛化能力，从而在无监督图像分割任务上带来了性能提升。新提出的归一化是对通常的槽注意力模块的最小且易于实现的修改，将值聚合机制从加权平均操作改为缩放加权求和操作。
英文摘要：
Object-centric scene decompositions are important representations for downstream tasks in fields such as computer vision and robotics. The recently proposed Slot Attention module, already leveraged by several derivative works for image segmentation and object tracking in videos, is a deep learning component which performs unsupervised object-centric scene decomposition on input images. It is based on an attention architecture, in which latent slot vectors, which hold compressed information on objects, attend to localized perceptual features from the input image. In this paper, we demonstrate that design decisions on normalizing the aggregated values in the attention architecture have considerable impact on the capabilities of Slot Attention to generalize to a higher number of slots and objects as seen during training. We propose and investigate alternatives to the original normalization scheme which increase the generalization capabilities of Slot Attention to varying slot and object counts, resulting in performance gains on the task of unsupervised image segmentation. The newly proposed normalizations represent minimal and easy to implement modifications of the usual Slot Attention module, changing the value aggregation mechanism from a weighted mean operation to a scaled weighted sum operation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
