视频模型是零样本学习者与推理者
Video models are zero-shot learners and reasoners
AI总结:
本文发现 Veo 3 等生成式视频模型具备零样本视觉任务处理与初步视觉推理能力,表明视频模型有望发展为统一的通才视觉基础模型。
摘要:
大型语言模型(LLM)卓越的零样本能力,推动自然语言处理从面向特定任务的模型转向统一的通才基础模型。这种转变源于简单的基本要素:在网络规模数据上训练的大型生成模型。令人好奇的是,同样的基本要素也适用于当今的生成式视频模型。视频模型是否会像 LLM 发展出通用语言理解那样,走向通用视觉理解?我们证明 Veo 3 可以解决其未曾明确训练过的多种任务:分割对象、检测边缘、编辑图像、理解物理属性、识别对象可供性、模拟工具使用等。这些感知、建模和操控视觉世界的能力,促成了迷宫求解与对称性求解等早期形式的视觉推理。Veo 涌现出的零样本能力表明,视频模型正走在成为统一通才视觉基础模型的道路上。
英文摘要:
The remarkable zero-shot capabilities of Large Language Models (LLMs) have propelled natural language processing from task-specific models to unified, generalist foundation models. This transformation emerged from simple primitives: large, generative models trained on web-scale data. Curiously, the same primitives apply to today's generative video models. Could video models be on a trajectory towards general-purpose vision understanding, much like LLMs developed general-purpose language understanding? We demonstrate that Veo 3 can solve a broad variety of tasks it wasn't explicitly trained for: segmenting objects, detecting edges, editing images, understanding physical properties, recognizing object affordances, simulating tool use, and more. These abilities to perceive, model, and manipulate the visual world enable early forms of visual reasoning like maze and symmetry solving. Veo's emergent zero-shot capabilities indicate that video models are on a path to becoming unified, generalist vision foundation models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。