arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.02639cs.CVcs.MM
收藏

可视化对话框的对抗鲁棒性

Adversarial Robustness of Visual Dialog

Tianjin University of Technology(天津理工大学);Heriot Watt University(赫瑞瓦特大学)

Lu Yu, Verena Rieser

摘要:

对抗鲁棒性评估机器学习模型的最坏情况性能,以确保其安全性和可靠性。这项研究是第一个调查的鲁棒性视觉接地对话模型对文本攻击。这些攻击代表了一种最坏的情况,即输入问题包含一个同义词,导致之前正确的模型返回错误的答案。使用这种情况下,我们首先要了解多模态输入组件如何有助于模型的鲁棒性。我们的研究结果表明,对对话历史进行编码的模型更鲁棒,并且当对历史发起攻击时,模型预测变得更加不确定。这与先前的工作相反,该工作发现对话历史对于该任务的模型性能可以忽略不计。我们还评估了如何生成对抗性测试示例,这些示例成功地欺骗了模型,但用户/软件设计者仍然无法检测到。我们发现,文本,以及视觉环境是很重要的,以产生合理的最坏情况。

英文摘要:

Adversarial robustness evaluates the worst-case performance scenario of a machine learning model to ensure its safety and reliability. This study is the first to investigate the robustness of visually grounded dialog models towards textual attacks. These attacks represent a worst-case scenario where the input question contains a synonym which causes the previously correct model to return a wrong answer. Using this scenario, we first aim to understand how multimodal input components contribute to model robustness. Our results show that models which encode dialog history are more robust, and when launching an attack on history, model prediction becomes more uncertain. This is in contrast to prior work which finds that dialog history is negligible for model performance on this task. We also evaluate how to generate adversarial test examples which successfully fool the model but remain undetected by the user/software designer. We find that the textual, as well as the visual context are important to generate plausible worst-case scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑