CE$^4$L：持续自我、外部及自我-外部学习
CE$^4$L: Continual Ego, Exo, and Ego-Exo Learning
- Tsinghua University（清华大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出CE$^4$L多视角持续学习基准及VISTA参数高效方法，通过子空间路由适配器应对任务与视角耦合变化，实现整体最优性能。
AI中文摘要：
具身智能体的感知基于视频，通常为多视角（自我视角、外部视角或两者兼具），并且本质上是持续性的，同时面临任务和视角的转变。然而，持续学习（CL）仍主要被仅外部视角的识别任务所主导，这掩盖了在现实世界中这些耦合转变下的行为表现。我们引入了持续自我、外部及自我-外部学习（CE$^4$L），这是一个统一的多视角持续学习基准，涵盖四个代表性任务：跨视角参考技能评估、时间动作分割、跨视角关联以及动作预测与规划。CE$^4$L 突出了先前持续学习基准中基本不存在的挑战，包括跨视角对应、视角依赖的异步性以及异构语义目标。为此，我们提出了视频增量子空间路由任务适配器（VISTA），这是一种参数高效的基线方法，将任务特定的更新存储在轻量级适配器中，并通过残差距离到基于二阶统计量估计的任务特定白化子空间来进行免训练路由。大量实验表明，代表性持续学习方法在 CE$^4$L 设置中的有效性差异显著，而 VISTA 始终具有竞争力，并达到了整体最先进的性能。我们用于基准和方法的源代码可在以下网址获取：此 https URL。
英文摘要：
Perception for embodied agents is video-based, often multi-view (ego, exo, or both), and inherently continual, with simultaneous task and viewpoint shifts. Yet continual learning (CL) remains dominated by exo-only recognition tasks, obscuring behavior under these real-world coupled shifts. We introduce Continual Ego, E}xo, and Ego-Exo Learning (CE$^4$L), a unified multi-view CL benchmark spanning four representative tasks: cross-view referenced skill assessment, temporal action segmentation, cross-view association, and action anticipation & planning. CE$^4$L highlights challenges largely absent in prior CL benchmarks, including cross-view correspondence, view-dependent asynchrony, and heterogeneous semantic objectives. To this end, we propose Video Incremental Subspace-routed Task Adapters (VISTA), a parameter-efficient baseline method that stores task-specific updates in lightweight adapters and performs training-free routing via residual distance to task-specific whitened subspaces estimated from second-order statistics. Extensive experiments demonstrate the significantly varied efficacy of representative CL methods across CE$^4$L settings, while VISTA is consistently competitive and achieves state-of-the-art overall performance. Our source code for benchmarks and methods is available at https://github.com/AnAppleCore/CE4L .
