arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2105.02802cs.CV
收藏

用于联合视觉表示学习的多视角LSTM

Multi-Perspective LSTM for Joint Visual Representation Learning

Instituto de Telecomunicações(电信研究所);Instituto Superior Técnico - Universidade de Lisboa(里斯本大学高等技术学院)

Alireza Sepas-Moghaddam, Fernando Pereira, Paulo Lobato Correia, Ali Etemad

摘要:

我们提出了一种新的LSTM细胞架构,能够学习从多个视角捕获的视觉序列中的视角内和视角间关系。我们的架构采用了一种新的经常性的联合学习策略,使用额外的门和记忆在细胞水平。我们证明,通过使用所提出的细胞来创建一个网络,更有效和更丰富的视觉表示学习识别任务。我们验证了我们提出的架构的性能的上下文中的两个多视角的视觉识别任务,即唇读和人脸识别。我们考虑了三个相关的数据集,并将结果与融合策略、其他现有的多输入LSTM架构和替代识别解决方案进行了比较。实验结果表明,我们的解决方案的性能优越的考虑基准,无论是在识别精度和复杂性。我们在https://github.com/arsm/MPLSTM上公开发布我们的代码。

英文摘要:

We present a novel LSTM cell architecture capable of learning both intra- and inter-perspective relationships available in visual sequences captured from multiple perspectives. Our architecture adopts a novel recurrent joint learning strategy that uses additional gates and memories at the cell level. We demonstrate that by using the proposed cell to create a network, more effective and richer visual representations are learned for recognition tasks. We validate the performance of our proposed architecture in the context of two multi-perspective visual recognition tasks namely lip reading and face recognition. Three relevant datasets are considered and the results are compared against fusion strategies, other existing multi-input LSTM architectures, and alternative recognition solutions. The experiments show the superior performance of our solution over the considered benchmarks, both in terms of recognition accuracy and complexity. We make our code publicly available at https://github.com/arsm/MPLSTM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑