Ego-Only：无需外心转移的自我中心动作检测
Ego-Only: Egocentric Action Detection without Exocentric Transferring
- Meta AI
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出Ego-Only方法，通过使用针对时间分割微调的掩码自编码器训练视频表示，无需外心转移即可在自我中心视频上实现最先进的动作检测和识别。
AI中文摘要：
我们提出了Ego-Only，这是第一种在无需任何形式的外心（第三人称）转移的情况下，能够在自我中心（第一人称）视频上实现最先进动作检测的方法。尽管两个领域在内容和外观上存在差距，但大规模的外心转移一直是自我中心动作检测的默认选择。这是因为先前的研究发现，从头开始训练自我中心模型很困难，而且从外心表示进行转移可以提高准确性。然而，在本文中，我们重新审视了这一普遍看法。受两个领域之间巨大差距的启发，我们提出了一种无需外心转移即可有效训练自我中心模型的策略。我们的Ego-Only方法很简单。它使用针对时间分割进行微调的掩码自编码器来训练视频表示。然后将学习到的特征输入到现成的时间动作定位方法中以检测动作。我们发现，通过这种简单的Ego-Only方法在三个已建立的自我中心视频数据集：Ego4D、EPIC-Kitchens-100和Charades-Ego上取得的显著强劲结果，使得外心转移变得不必要。在动作检测和动作识别方面，Ego-Only都优于以前使用数量级更多标签的最佳外心转移方法。Ego-Only在没有外心数据的情况下，在这些数据集和基准上创造了新的最先进结果。
英文摘要：
We present Ego-Only, the first approach that enables state-of-the-art action detection on egocentric (first-person) videos without any form of exocentric (third-person) transferring. Despite the content and appearance gap separating the two domains, large-scale exocentric transferring has been the default choice for egocentric action detection. This is because prior works found that egocentric models are difficult to train from scratch and that transferring from exocentric representations leads to improved accuracy. However, in this paper, we revisit this common belief. Motivated by the large gap separating the two domains, we propose a strategy that enables effective training of egocentric models without exocentric transferring. Our Ego-Only approach is simple. It trains the video representation with a masked autoencoder finetuned for temporal segmentation. The learned features are then fed to an off-the-shelf temporal action localization method to detect actions. We find that this renders exocentric transferring unnecessary by showing remarkably strong results achieved by this simple Ego-Only approach on three established egocentric video datasets: Ego4D, EPIC-Kitchens-100, and Charades-Ego. On both action detection and action recognition, Ego-Only outperforms previous best exocentric transferring methods that use orders of magnitude more labels. Ego-Only sets new state-of-the-art results on these datasets and benchmarks without exocentric data.
