We present a novel LSTM cell architecture capable of learning both intra- and inter-perspective relationships available in visual sequences captured from multiple perspectives. Our architecture adopts a novel recurrent joint learning strategy that uses additional gates and memories at the cell level. We demonstrate that by using the proposed cell to create a network, more effective and richer visual representations are learned for recognition tasks. We validate the performance of our proposed architecture in the context of two multi-perspective visual recognition tasks namely lip reading and face recognition. Three relevant datasets are considered and the results are compared against fusion strategies, other existing multi-input LSTM architectures, and alternative recognition solutions. The experiments show the superior performance of our solution over the considered benchmarks, both in terms of recognition accuracy and complexity. We make our code publicly available at https://github.com/arsm/MPLSTM.
Attentioned Convolutional LSTM InpaintingNetwork for Anomaly Detection in Videos
Intel(英特尔)
我们提出了一种半监督模型,用于检测视频像素网络启发的视频中的异常[van den Oord et al.,2016年]。VPN是一种基于深度神经网络的概率生成模型,用于估计视频帧中原始像素的离散联合分布。我们的模型扩展了VPN的卷积-LSTM视频编码器部分,具有一种新的基于卷积的注意力机制。我们还将VPN的Pixel-CNN解码器部分修改为帧修复任务,其中要预测的帧的部分掩蔽版本作为输入。帧重构误差被用作异常指示符。我们在移动mnist数据集的修改版本上测试我们的模型[Srivastava et al.,2015年]。我们的模型被证明是有效的检测视频中的异常。这种方法可以作为需要视觉常识的应用程序的一个组件。
Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks
University of California, Irvine(加州大学欧文分校);Microsoft Research Asia(微软亚洲研究院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Peking University(北京大学);University of Chinese Academy of Sciences(中国科学院大学)
ARN-LSTM: A Multi-Stream Fusion Model for Skeleton-based Action Recognition
Universiti Sains Malaysia(马来西亚理科大学);College of Technology and Business(科技与商学院);Shenzhen Polytechnic University(深圳职业技术大学);Institute of Applied Artificial Intelligence of the Guangdong-Hong Kong-Macao Greater Bay Area(粤港澳大湾区应用人工智能研究院)
MS-LSTM: Exploring Spatiotemporal Multiscale Representations in Video Prediction Domain
Faculty of Computing, Harbin Institute of Technology(哈尔滨工业大学计算机学院);International Research Institute for Artificial Intelligence, Harbin Institute of Technology(哈尔滨工业大学国际人工智能研究院)
An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition
Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);University of Chinese Academy of Sciences (UCAS)(中国科学院大学);University of Science and Technology of China (USTC)(中国科学技术大学)
Beyond Weak Perspective for Monocular 3D Human Pose Estimation
Amazon Lab126(亚马逊 Lab126)
我们考虑的任务的三维关节的位置和方向预测从单目视频与皮肤多人线性(SMPL)模型。我们首先推断2D关节位置与现成的姿态估计算法。我们使用SPIN算法,并从深度回归神经网络中估计身体姿势、形状和相机参数的初始预测。然后,我们坚持SMPLify算法,该算法接收这些初始参数,并对其进行优化,以便从SMPL模型推断的3D关节将适合2D关节位置。该算法涉及3D关节到2D图像平面的投影步骤。传统的方法是遵循弱透视假设,使用特设焦距。通过在3D Poses in the Wild(3DPW)数据集上的实验,我们证明了使用全透视投影,正确的相机中心和近似的焦距,可以提供良好的结果。我们的算法已经在3DPW挑战赛中获胜,在关节定位精度方面达到第一名。
What and Where: Modeling Skeletons from Semantic and Spatial Perspectives for Action Recognition
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);CAS Center for Excellence in Brain Science and Intelligence Technology(中国科学院脑科学与智能技术卓越创新中心)
SAR Target Recognition Using the Multi-aspect-aware Bidirectional LSTM Recurrent Neural Networks
College of Information Science & Technology, Beijing University of Chemical Technology(北京化工大学信息科学与技术学院);National Research Center for High-Performance Computing Engineering Technology, Sugon Information Industry Co., Ltd(曙光信息产业股份有限公司高性能计算工程技术国家研究中心);Sichuan Provincial Key Laboratory of Information Coding and Transmission, Southwest Jiaotong University(西南交通大学信息编码与传输四川省重点实验室);Institute of Electronics, Chinese Academy of Sciences(中国科学院电子学研究所)