arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.00211cs.CV
收藏

Trustworthy Self-Attention:使网络仅关注最相关的参考点

Trustworthy Self-Attention: Enabling the Network to Focus Only on the Most Relevant References

Chongqing University(重庆大学)

Yu Jing, Tan Yujuan, Ren Ao, Liu Duo

AI总结:

针对遮挡点光流预测中自注意力易受错误参考干扰的问题,提出利用在线遮挡识别信息构建扩展特征与强约束的轻量化方法,无需遮挡标签即可使网络聚焦最相关参考,在 Sintel 基准上达到 SOTA 性能。

摘要:

遮挡点的光流预测仍然是一个尚未解决的难题。近期方法利用自注意力机制寻找相关的非遮挡点作为参考,基于自相似性假设估计遮挡点的光流。然而,这些方法依赖单图像的视觉特征和弱约束,这不足以约束训练后的网络避免关注错误和弱相关的参考点。我们充分利用在线遮挡识别信息,构建遮挡扩展视觉特征和两项强约束,使网络学会仅关注最相关的参考点,且无需遮挡真实标签参与网络训练。我们的方法在原始框架上仅增加极少的网络参数,保持轻量化。大量实验表明,我们的模型具有最优的跨数据集泛化能力。在 Sintel Albedo pass 上,相较于基于 GMA 的最先进方法 MATCHFlow(GMA),我们的方法实现了更大的误差降低,所有点、非遮挡点和遮挡点的误差分别降低了 18.6%、16.2% 和 20.1%。此外,我们的模型在 Sintel 基准上达到最先进性能,在 Sintel clean pass 的所有已发表方法中排名第一。代码将开源。

英文摘要:

The prediction of optical flow for occluded points is still a difficult problem that has not yet been solved. Recent methods use self-attention to find relevant non-occluded points as references for estimating the optical flow of occluded points based on the assumption of self-similarity. However, they rely on visual features of a single image and weak constraints, which are not sufficient to constrain the trained network to focus on erroneous and weakly relevant reference points. We make full use of online occlusion recognition information to construct occlusion extended visual features and two strong constraints, allowing the network to learn to focus only on the most relevant references without requiring occlusion ground truth to participate in the training of the network. Our method adds very few network parameters to the original framework, making it very lightweight. Extensive experiments show that our model has the greatest cross-dataset generalization. Our method achieves much greater error reduction, 18.6%, 16.2%, and 20.1% for all points, non-occluded points, and occluded points respectively from the state-of-the-art GMA-base method, MATCHFlow(GMA), on Sintel Albedo pass. Furthermore, our model achieves state-of-the-art performance on the Sintel bench-marks, ranking \#1 among all published methods on Sintel clean pass. The code will be open-source.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑