用于噪声图像分类的处理学习因果Transformer
Treatment Learning Causal Transformer for Noisy Image Classification
AI总结:
本文提出处理学习因果Transformer(TLT),将噪声存在性作为处理因素,通过因果变分推断和潜在生成模型估计鲁棒特征,从而提升噪声图像分类精度,并构建了含多种噪声因素的新基准数据集。
摘要:
当前顶尖的基于深度学习(DL)的视觉模型主要依赖于探索和利用训练数据样本与其关联标签之间的内在相关性。然而,一个已知的实际挑战是它们在“噪声”数据上的性能会下降,这些噪声数据由虚假相关、无关上下文、领域偏移和对抗攻击等不同情况引起。在这项工作中,我们将“噪声存在性”这一二元信息作为处理因素引入图像分类任务,通过联合估计其处理效应来提高预测精度。受因果变分推断的启发,我们提出了一种基于Transformer的架构,即处理学习因果Transformer(TLT),它使用潜在生成模型从当前观测输入中估计鲁棒特征表示,用于噪声图像分类。根据估计的噪声水平(建模为二元处理因素),TLT分配由设计的因果损失训练的相应推断网络进行预测。我们还创建了新的噪声图像数据集,其中包含广泛的噪声因素(例如,物体掩蔽、风格迁移和对抗扰动),用于性能基准测试。TLT在噪声图像分类中的优越性能进一步通过若干反驳评估指标得到验证。作为副产品,TLT还改进了用于感知噪声图像的视觉显著性方法。
英文摘要:
Current top-notch deep learning (DL) based vision models are primarily based on exploring and exploiting the inherent correlations between training data samples and their associated labels. However, a known practical challenge is their degraded performance against "noisy" data, induced by different circumstances such as spurious correlations, irrelevant contexts, domain shift, and adversarial attacks. In this work, we incorporate this binary information of "existence of noise" as treatment into image classification tasks to improve prediction accuracy by jointly estimating their treatment effects. Motivated from causal variational inference, we propose a transformer-based architecture, Treatment Learning Causal Transformer (TLT), that uses a latent generative model to estimate robust feature representations from current observational input for noise image classification. Depending on the estimated noise level (modeled as a binary treatment factor), TLT assigns the corresponding inference network trained by the designed causal loss for prediction. We also create new noisy image datasets incorporating a wide range of noise factors (e.g., object masking, style transfer, and adversarial perturbation) for performance benchmarking. The superior performance of TLT in noisy image classification is further validated by several refutation evaluation metrics. As a by-product, TLT also improves visual salience methods for perceiving noisy images.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。