基于可微分频率的解纠缠方法用于 aerial 视频动作识别
Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition
AI总结：
针对无人机视频中人类活动识别问题，提出一种结合频域表示与数据驱动神经网络的方法，通过可微分静态-动态频率掩码先验学习解纠缠特征，并设计成本函数采样关键帧，在两个数据集上取得显著性能提升。
摘要：
我们提出了一种用于视频中人类活动识别的学习算法。我们的方法专为无人机（UAV）视频设计，这类视频主要由倾斜放置的动态摄像机拍摄，包含人类演员和背景运动。通常，人类演员占据的空间分辨率不到十分之一。我们的方法同时利用了频域表示（信号处理中的经典分析工具）和数据驱动神经网络的优势。我们构建了一个可微分的静态-动态频率掩码先验，以对视频中的显著静态和动态像素进行建模，这对动作识别的基础任务至关重要。我们使用这个可微分掩码先验，使神经网络能够通过身份损失函数内在地学习解纠缠的特征表示。我们的公式使网络能够在其层内固有地计算解纠缠的显著特征。此外，我们提出了一个包含时间相关性和空间内容的成本函数，以在均匀间隔的视频片段中采样最重要的帧。我们在 UAV Human 数据集和 NEC Drone 数据集上进行了广泛的实验，结果表明，与最先进的方法相比，相对改进了 5.72% - 13.00%，与相应的基线模型相比，相对改进了 14.28% - 38.05%。
英文摘要：
We present a learning algorithm for human activity recognition in videos. Our approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras that contain a human actor along with background motion. Typically, the human actors occupy less than one-tenth of the spatial resolution. Our approach simultaneously harnesses the benefits of frequency domain representations, a classical analysis tool in signal processing, and data driven neural networks. We build a differentiable static-dynamic frequency mask prior to model the salient static and dynamic pixels in the video, crucial for the underlying task of action recognition. We use this differentiable mask prior to enable the neural network to intrinsically learn disentangled feature representations via an identity loss function. Our formulation empowers the network to inherently compute disentangled salient features within its layers. Further, we propose a cost-function encapsulating temporal relevance and spatial content to sample the most important frame within uniformly spaced video segments. We conduct extensive experiments on the UAV Human dataset and the NEC Drone dataset and demonstrate relative improvements of 5.72% - 13.00% over the state-of-the-art and 14.28% - 38.05% over the corresponding baseline model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
