面向视障人群安全辅助的质量无关图像描述
Quality-agnostic Image Captioning to Safely Assist People with Vision Impairment
AI总结：
针对视障人群拍摄图像噪声多导致图像描述模型预测不准、有安全风险的问题，从数据、模型、评估三方面提出质量无关框架，提升了模型性能、抗噪鲁棒性与安全场景可靠性，已用于合作开发的视障辅助生活应用。
摘要：
自动化图像描述有望成为视障人群的实用工具。该用户群体拍摄的图像通常存在噪声，会导致模型预测错误甚至产生安全风险。本文提出一种质量无关框架，用于提升面向视障人群的图像描述模型的性能与鲁棒性，我们从数据、模型、评估三个维度解决该问题。首先，我们展示了用于生成合成噪声的数据增强技术如何解决该领域的数据稀疏问题。其次，我们将当前最优模型扩展为双网络架构，结合增强数据并利用多种一致性损失，增强模型的鲁棒性。实验结果表明，与当前最优的图像描述网络相比，所提方法性能有所提升，例如CIDEr指标绝对提升2.15；同时抗噪声鲁棒性更强，在噪声更高的设置下，CIDEr指标提升最高可达3分。最后，我们通过对不同难度/噪声水平的图像进行置信度校准，评估预测可靠性，结果显示我们的模型在安全关键场景下表现更可靠。该改进模型是一款辅助生活应用的组成部分，我们与英国皇家国立盲人研究所（Royal National Institute of Blind People）合作开发了该应用。
英文摘要：
Automated image captioning has the potential to be a useful tool for people with vision impairments. Images taken by this user group are often noisy, which leads to incorrect and even unsafe model predictions. In this paper, we propose a quality-agnostic framework to improve the performance and robustness of image captioning models for visually impaired people. We address this problem from three angles: data, model, and evaluation. First, we show how data augmentation techniques for generating synthetic noise can address data sparsity in this domain. Second, we enhance the robustness of the model by expanding a state-of-the-art model to a dual network architecture, using the augmented data and leveraging different consistency losses. Our results demonstrate increased performance, e.g. an absolute improvement of 2.15 on CIDEr, compared to state-of-the-art image captioning networks, as well as increased robustness to noise with up to 3 points improvement on CIDEr in more noisy settings. Finally, we evaluate the prediction reliability using confidence calibration on images with different difficulty/noise levels, showing that our models perform more reliably in safety-critical situations. The improved model is part of an assisted living application, which we develop in partnership with the Royal National Institute of Blind People.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
