arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.04085cs.CV
收藏

独特生活,共享世界:从单个人生视频中学习

Unique Lives, Shared World: Learning from Single-Life Videos

Google DeepMind(谷歌DeepMind)

Tengda Han, Sayna Ebrahimi, Dilara Gokay, Li Yang Ku, Maks Ovsjanikov, Iva Babukova, Daniel Zoran, Viorica Patraucean, Joao Carreira, Andrew Zisserman, Dima Damen

AI总结:

提出“单个人生”学习范式,利用单个人拍摄的自我中心视频通过多视角自监督学习视觉编码器,发现不同人生训练的模型具有高度对齐的几何理解,且学到的表示可泛化到下游任务,与大量网络数据性能相当。

摘要:

我们引入了“单个人生”学习范式,其中我们仅针对一个人拍摄的自我中心视频训练一个独特的视觉模型。我们利用单个人生中自然捕获的多个视角,以自监督方式学习视觉编码器。我们的实验展示了三个关键发现。首先,独立在不同人生上训练的模型发展出高度对齐的几何理解。我们通过在捕获不同人生(包括室内和室外)的不同数据集上训练视觉编码器,并引入一种新的基于交叉注意力的度量来量化不同模型发展的内部表示的功能对齐,来证明这一点。其次,我们展示了单个人生模型学习到可泛化的几何表示,这些表示能有效迁移到下游任务,如未见环境中的深度估计。第三,我们证明,对同一个人一周内最多30小时的数据进行训练,其性能与在30小时多样化网络数据上训练相当,突出了单个人生表示学习的优势。总体而言,我们的结果确立了世界的共享结构既导致了在个人人生上训练的模型的一致性,也为视觉表示学习提供了强大的信号。

英文摘要:

We introduce the "single-life" learning paradigm, where we train a distinct vision model exclusively on egocentric videos captured by one individual. We leverage the multiple viewpoints naturally captured within a single life to learn a visual encoder in a self-supervised manner. Our experiments demonstrate three key findings. First, models trained independently on different lives develop a highly aligned geometric understanding. We demonstrate this by training visual encoders on distinct datasets each capturing a different life, both indoors and outdoors, as well as introducing a novel cross-attention-based metric to quantify the functional alignment of the internal representations developed by different models. Second, we show that single-life models learn generalizable geometric representations that effectively transfer to downstream tasks, such as depth estimation, in unseen environments. Third, we demonstrate that training on up to 30 hours from one week of the same person's life leads to comparable performance to training on 30 hours of diverse web data, highlighting the strength of single-life representation learning. Overall, our results establish that the shared structure of the world, both leads to consistency in models trained on individual lives, and provides a powerful signal for visual representation learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑