PhysStream：基于结构化场景记忆与细粒度运动控制的流式物理驱动视频生成
PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control
- University of Pennsylvania（宾夕法尼亚大学）
- Snap Inc.（Snap公司）
- KAUST（阿卜杜拉国王科技大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
PhysStream提出一种自回归视频生成模型，通过结构化场景记忆和稀疏速度增量信号实现物理驱动的细粒度运动控制，支持交互式生成中控制，显著降低运动分布距离和轨迹误差。
AI中文摘要：
视频生成的交互式控制正从粗略的提示词转向对动态场景的细粒度、物理上有意义的操控。然而，现有的可控方法要么在生成开始前就需要完整的控制计划，要么使用像素空间信号来指定物体位置而非物理动态。为解决这些局限，我们提出了PhysStream，一种用于物理驱动的图像到视频合成的自回归模型，它结合了结构化场景记忆——从先前生成的帧中在线导出的位置图和物体跟踪图——并通过稀疏的速度增量信号支持细粒度运动控制，这些信号编码物理量，使模型能够学习底层动态。我们分两个阶段训练模型：首先用运动控制条件微调双向模型，然后训练因果自回归模型并附加结构化场景记忆，进一步提高物理一致性。PhysStream支持对多物体桌面刚体场景进行交互式、生成中控制——这是先前方法不支持的能力——在合成基准上将运动分布距离（FVMD）降低了33%，轨迹误差降低了12%，优于最强基线，并且在超过85%的真实场景比较中受到人类评估者的青睐。更多详情请访问我们的网站：此https URL。
英文摘要：
Interactive control for video generation is moving from coarse prompts toward fine-grained, physically meaningful manipulation of dynamic scenes. Yet existing controllable methods either require the full control schedule before generation starts, or use pixel-space signals that dictate object positions rather than physical dynamics. To address these limitations, we propose PhysStream, an autoregressive model for physics-grounded image-to-video synthesis that incorporates structured scene memory---positional maps and object tracking maps derived online from previously generated frames---and supports fine-grained motion control via sparse velocity-increment signals that encode physical quantities, letting the model learn the underlying dynamics. We train our model in two stages: a bidirectional model is first finetuned with motion-control conditioning, then a causal autoregressive model is trained with additional structured scene memory, further improving physical consistency. PhysStream enables interactive, mid-generation control over multi-object tabletop rigid-body scenes---a capability not supported by prior methods---reducing motion distribution distance (FVMD) by 33% and trajectory error by 12% over the strongest baselines on synthetic benchmarks, and is preferred by human evaluators in over 85% of in-the-wild comparisons. Please check our website for more details: https://czzzzh.github.io/PhysStream
