arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.09789cs.LGcs.CV
收藏

深度学习模型语义连续性的实验研究

An Experimental Study of Semantic Continuity for Deep Learning Models

Beijing Jiaotong University(北京交通大学);University of International Business and Economics(对外经济贸易大学)

Shangxi Wu, Dongyuan Lu, Xian Zhao, Lizhang Chen, Jitao Sang

AI总结:

本文针对深度学习模型的语义不连续问题,分析其证据与成因,并提出简单的语义连续性约束,使模型获得平滑梯度并学习语义特征,从而提升鲁棒性、可解释性、迁移性并减少机器偏见。

摘要:

深度学习模型存在语义不连续问题:输入空间中的微小扰动往往会对模型输出造成语义层面的干扰。我们认为,语义不连续源于这些不恰当的训练目标,并导致了对抗鲁棒性、可解释性等众所周知的问题。我们首先进行数据分析,为现有深度学习模型中的语义不连续提供证据;随后设计一种简单的语义连续性约束,该约束在理论上使模型能够获得平滑梯度,并学习面向语义的特征。定性与定量实验证明,语义连续的模型成功减少了对非语义信息的使用,进而促进了对抗鲁棒性、可解释性、模型迁移和机器偏见方面的改善。

英文摘要:

Deep learning models suffer from the problem of semantic discontinuity: small perturbations in the input space tend to cause semantic-level interference to the model output. We argue that the semantic discontinuity results from these inappropriate training targets and contributes to notorious issues such as adversarial robustness, interpretability, etc. We first conduct data analysis to provide evidence of semantic discontinuity in existing deep learning models, and then design a simple semantic continuity constraint which theoretically enables models to obtain smooth gradients and learn semantic-oriented features. Qualitative and quantitative experiments prove that semantically continuous models successfully reduce the use of non-semantic information, which further contributes to the improvement in adversarial robustness, interpretability, model transfer, and machine bias.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑