通过高阶运动流持续学习共轭视觉表示
Continual Learning of Conjugated Visual Representations through Higher-order Motion Flows
AI总结：
针对连续视觉流非独立同分布带来的持续学习挑战，提出基于自主学习多阶运动流的运动共轭特征表示方法，通过空间感知自监督对比损失避免平凡解，在合成与真实视频任务上性能优于现有无监督及预训练模型。
摘要：
由于数据的非独立同分布（non-i.i.d.）特性，用神经网络从连续视觉信息流中学习存在若干挑战，但这也为开发与信息流一致的表示提供了新机遇。本文研究了受多重运动诱导约束的像素级特征的无监督持续学习场景，这类特征因此被称为运动共轭特征表示。与现有方法不同，运动并非给定信号（既不是真实标注也不是由外部模块估计得到），而是在特征层级的不同层级上发生的渐进式自主学习过程的产物。我们用神经网络估计多条运动流，这些运动流具有不同的抽象层级，涵盖从传统光流到源自更高层特征的其他潜在信号，因此被称为高阶运动。持续学习以构建一致的多阶流和表示容易出现平凡解，我们通过引入一种基于流诱导相似性的空间感知自监督对比损失来应对这一问题。我们在真实感合成流和真实世界视频上评估了模型，与预训练的最先进特征提取器（也基于Transformer）以及近期的无监督学习模型进行了对比，结果表明所提方法的性能显著优于这些备选方案。
英文摘要：
Learning with neural networks from a continuous stream of visual information presents several challenges due to the non-i.i.d. nature of the data. However, it also offers novel opportunities to develop representations that are consistent with the information flow. In this paper we investigate the case of unsupervised continual learning of pixel-wise features subject to multiple motion-induced constraints, therefore named motion-conjugated feature representations. Differently from existing approaches, motion is not a given signal (either ground-truth or estimated by external modules), but is the outcome of a progressive and autonomous learning process, occurring at various levels of the feature hierarchy. Multiple motion flows are estimated with neural networks and characterized by different levels of abstractions, spanning from traditional optical flow to other latent signals originating from higher-level features, hence called higher-order motions. Continuously learning to develop consistent multi-order flows and representations is prone to trivial solutions, which we counteract by introducing a self-supervised contrastive loss, spatially-aware and based on flow-induced similarity. We assess our model on photorealistic synthetic streams and real-world videos, comparing to pre-trained state-of-the art feature extractors (also based on Transformers) and to recent unsupervised learning models, significantly outperforming these alternatives.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
