增强特征多样性可提升通道自适应 Vision Transformers
Enhancing Feature Diversity Boosts Channel-Adaptive Vision Transformers
AI总结：
针对多通道成像中通道配置变化与特征冗余问题，提出 DiChaViT，通过差异化通道采样、正则化和初始化增强特征多样性，在三类数据集上较 SOTA 提升 1.5–5.0%。
摘要：
多通道成像（Multi-Channel Imaging，MCI）在编码有用特征表示方面包含一系列传统图像中不存在的挑战。例如，来自两颗不同卫星的图像可能都包含 RGB 通道，但每个成像来源的其余通道可能不同。因此，MCI 模型必须在测试时支持多种通道配置。近期工作通过用表示通道配置的编码补充像素信息，将传统视觉编码器扩展到 MCI，例如 Vision Transformers（ViT）。然而，这些方法平等对待每个通道，即它们没有考虑每种通道类型的独特属性，这可能导致学习到的特征中出现不必要且可能有害的冗余。例如，如果 RGB 通道始终存在，其他通道就可以专注于提取 RGB 通道无法捕获的信息。为此，我们提出了 DiChaViT，旨在增强 MCI-ViT 模型学习特征的多样性。这是通过一种新颖的通道采样策略实现的，该策略鼓励在训练中选择更具差异性的通道集合。此外，我们采用正则化和初始化技术，以提高从每个通道学习新信息的可能性。我们的许多改进与架构无关，并且可以在新架构开发时整合进去。在卫星和细胞显微镜数据集 CHAMMI、JUMP-CP 和 So2Sat 上的实验表明，DiChaViT 相比最先进方法带来了 1.5–5.0% 的提升。我们的代码公开发布于 https://github.com/chaudatascience/diverse_channel_vit。
英文摘要：
Multi-Channel Imaging (MCI) contains an array of challenges for encoding useful feature representations not present in traditional images. For example, images from two different satellites may both contain RGB channels, but the remaining channels can be different for each imaging source. Thus, MCI models must support a variety of channel configurations at test time. Recent work has extended traditional visual encoders for MCI, such as Vision Transformers (ViT), by supplementing pixel information with an encoding representing the channel configuration. However, these methods treat each channel equally, i.e., they do not consider the unique properties of each channel type, which can result in needless and potentially harmful redundancies in the learned features. For example, if RGB channels are always present, the other channels can focus on extracting information that cannot be captured by the RGB channels. To this end, we propose DiChaViT, which aims to enhance the diversity in the learned features of MCI-ViT models. This is achieved through a novel channel sampling strategy that encourages the selection of more distinct channel sets for training. Additionally, we employ regularization and initialization techniques to increase the likelihood that new information is learned from each channel. Many of our improvements are architecture agnostic and can be incorporated into new architectures as they are developed. Experiments on both satellite and cell microscopy datasets, CHAMMI, JUMP-CP, and So2Sat, report DiChaViT yields a 1.5 - 5.0% gain over the state-of-the-art. Our code is publicly available at https://github.com/chaudatascience/diverse_channel_vit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
