arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2111.07534cs.CV
收藏

序列观察场景的概率硬注意模型

A Probabilistic Hard Attention Model For Sequentially Observed Scenes

McGill University(麦吉尔大学)

Samrudhdhi B. Rangrej, James J. Clark

摘要:

视觉硬注意模型主动选择并观察图像中的子区域序列以进行预测。大多数硬注意模型通过首先分析完整的图像来确定值得注意的区域。然而,可能的情况是,整个图像最初不可用,而是通过一系列部分观察逐渐感测。在本文中,我们设计了一个有效的硬注意力模型分类这样的顺序观察的场景。所提出的模型从未完全观察到图像。为了选择部分可观测的信息区域,该模型使用贝叶斯最优实验设计。该方法首先在已观测区域的基础上综合未观测区域的特征。然后,它使用预测的功能来估计所达到的预期信息增益(EIG),应参加各个区域。最后,该模型关注的实际内容的位置上,上述EIG是最大的。该模型使用a)一个递归特征聚合器来维持一个递归状态,b)一个线性分类器来预测类标签,c)一个部分变分自编码器来预测未观察区域的特征。我们在Partial VAE中使用归一化流来处理特征合成问题中的多模态。我们使用可微目标训练我们的模型,并在五个数据集上对其进行测试。我们的模型比基线模型的准确度高2-10%,当两者都只看到几个一瞥时。

英文摘要:

A visual hard attention model actively selects and observes a sequence of subregions in an image to make a prediction. The majority of hard attention models determine the attention-worthy regions by first analyzing a complete image. However, it may be the case that the entire image is not available initially but instead sensed gradually through a series of partial observations. In this paper, we design an efficient hard attention model for classifying such sequentially observed scenes. The presented model never observes an image completely. To select informative regions under partial observability, the model uses Bayesian Optimal Experiment Design. First, it synthesizes the features of the unobserved regions based on the already observed regions. Then, it uses the predicted features to estimate the expected information gain (EIG) attained, should various regions be attended. Finally, the model attends to the actual content on the location where the EIG mentioned above is maximum. The model uses a) a recurrent feature aggregator to maintain a recurrent state, b) a linear classifier to predict the class label, c) a Partial variational autoencoder to predict the features of unobserved regions. We use normalizing flows in Partial VAE to handle multi-modality in the feature-synthesis problem. We train our model using a differentiable objective and test it on five datasets. Our model gains 2-10% higher accuracy than the baseline models when both have seen only a couple of glimpses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑