基于时空Fisher向量概率积分的动作联合识别与分割
Joint Recognition and Segmentation of Actions via Probabilistic Integration of Spatio-Temporal Fisher Vectors
摘要：
我们提出了一个分层的方法来进行联合分类和分割的多动作识别。一个给定的视频（包含几个连续的动作）通过一系列重叠的时间窗口进行处理。时间窗口中的每个帧通过选择性的低级别时空特征来表示，这些特征有效地捕获相关的局部动态。来自每个窗口的特征被表示为Fisher向量，其捕获一阶和二阶统计量。不是直接对每个Fisher向量进行分类，而是将其转换为类别概率的向量。然后，通过在帧级整合类概率来获得每个帧的最终分类决策，这利用了时间窗口的重叠。实验在两个数据集上进行：s-KTH（KTH数据集的拼接版本，用于模拟多动作）和具有挑战性的CMU-MMAC数据集。在s-KTH上，所提出的方法实现了85.0%的准确率，显著优于最近基于Gestival和Hestival的两种方法，分别获得78.3%和71.2%。在CMU-MMAC上，该方法的准确率为40.9%，优于GMM和HMM方法，分别为33.7%和38.4%。此外，所提出的系统平均比基于GMM的方法快40倍。
英文摘要：
We propose a hierarchical approach to multi-action recognition that performs joint classification and segmentation. A given video (containing several consecutive actions) is processed via a sequence of overlapping temporal windows. Each frame in a temporal window is represented through selective low-level spatio-temporal features which efficiently capture relevant local dynamics. Features from each window are represented as a Fisher vector, which captures first and second order statistics. Instead of directly classifying each Fisher vector, it is converted into a vector of class probabilities. The final classification decision for each frame is then obtained by integrating the class probabilities at the frame level, which exploits the overlapping of the temporal windows. Experiments were performed on two datasets: s-KTH (a stitched version of the KTH dataset to simulate multi-actions), and the challenging CMU-MMAC dataset. On s-KTH, the proposed approach achieves an accuracy of 85.0%, significantly outperforming two recent approaches based on GMMs and HMMs which obtained 78.3% and 71.2%, respectively. On CMU-MMAC, the proposed approach achieves an accuracy of 40.9%, outperforming the GMM and HMM approaches which obtained 33.7% and 38.4%, respectively. Furthermore, the proposed system is on average 40 times faster than the GMM based approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
