图像编辑指令具有异质性:颜色交换、对象插入和物理动作编辑需要不同的空间覆盖和推理深度,但现有基于推理的编辑器对每条指令应用单一固定推理配方。我们主张在空间和时间轴上都具有适应性是缺失的自由度,我们提出了PhysEdit,一种围绕这一原则构建的编辑框架。PhysEdit引入了两个推理时间模块,这些模块无需重新训练骨干网络即可组合。其核心是(1)复杂度自适应推理深度(CARD)直接从指令和参考图像预测编辑复杂度,并分配推理步骤数N_r和推理令牌长度r每样本--将之前固定的推理计划转换为条件计算问题。CARD由(2)空间推理掩码(SRM)支持,该掩码从交叉注意力中提取指令条件空间先验,将推理限制在语义上需要的区域。在完整的737个ImgEdit Basic-Edit套件上,PhysEdit在强推理基线上实现了1.18倍的墙钟加速(64.3秒 vs. 76.1秒每样本),同时略微提高了指令遵循度(CLIP-T 0.2283 vs. 0.2266,+0.7%)并在噪声中匹配了身份保持(CLIP-I 0.8246 vs. 0.8280)。加速是类别依赖的,达到外观级编辑时达到1.52倍,验证了CARD的自适应分配作为效率增益的主要来源。30个样本的试点与完整消融实验隔离了每个模块的贡献。
英文摘要:
Image editing instructions are heterogeneous: a color swap, an object insertion, and a physical-action edit all demand different spatial coverage and different reasoning depth, yet existing reasoning-based editors apply a single fixed inference recipe to every instruction. We argue that adaptivity along both the spatial and temporal axes is the missing degree of freedom, and we present PhysEdit, an editing framework built around this principle. PhysEdit introduces two inference-time modules that compose without retraining the backbone. At its core, (1) Complexity-Adaptive Reasoning Depth (CARD) predicts edit complexity directly from the instruction and reference image and allocates the reasoning step count N_r and reasoning-token length r per sample -- turning a previously fixed inference schedule into a conditional-computation problem. CARD is supported by (2) a Spatial Reasoning Mask (SRM) that extracts an instruction-conditioned spatial prior from cross-attention to confine reasoning to regions that semantically require it. On the full 737-case ImgEdit Basic-Edit Suite, PhysEdit delivers a 1.18x wall-clock speedup (64.3s vs. 76.1s per sample) over a strong reasoning baseline while slightly improving instruction adherence (CLIP-T 0.2283 vs. 0.2266, +0.7%) and matching identity preservation within noise (CLIP-I 0.8246 vs. 0.8280). The speedup is category-dependent and reaches 1.52x on appearance-level edits, validating CARD's adaptive allocation as the principal source of efficiency gain. A 30-sample pilot with full ablations isolates the contribution of each module.
Cut-and-Paste: Subject-Driven Video Editing with Attention Control
School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机与信息工程学院);School of Computer Science, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)计算机学院)
IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learning
School of Information and Communication Engineering, Communication University of China(中国通信大学信息与通信工程学院);Show Lab, National University of Singapore(新加坡国立大学Show实验室);Baidu Inc., Beijing, China(百度公司)
Google Research(谷歌研究院);The Hebrew University of Jerusalem(希伯来大学)
Text-driven image and video diffusion models have recently achieved unprecedented generation realism. While diffusion models have been successfully applied for image editing, very few works have done so for video editing. We present the first diffusion-based method that is able to perform text-based motion and appearance editing of general videos. Our approach uses a video diffusion model to combine, at inference time, the low-resolution spatio-temporal information from the original video with new, high resolution information that it synthesized to align with the guiding text prompt. As obtain…
VRWKV-Editor: Reducing quadratic complexity in transformer-based video editing
International Artificial Intelligence Center of Morocco University Mohammed VI Polytechnic Rabat, Morocco(摩洛哥国际人工智能中心摩洛哥穆莱·伊沙克·穆莱·阿卜杜勒阿齐兹 polytechnic 大学拉巴特分校);University Mohammed VI Polytechnic Rabat, Morocco(摩洛哥穆莱·伊沙克·穆莱·阿卜杜勒阿齐兹 polytechnic 大学拉巴特分校);Sorbonne University, LIP6 - UMR 7606 CNRS, France(索邦大学,LIP6 - UMR 7606 CNRS,法国)
Spatio-Temporal Garment Reconstruction Using Diffusion Mapping via Pattern Coordinates
CVLab, École Polytechnique Fédérale de Lausanne (EPFL)(瑞士联邦理工学院(EPFL)计算机视觉实验室);Mohamed bin Zayed University of Artificial Intelligence(马尔代夫人工智能大学);Department of Computer Science and Engineering, Southern University of Science and Technology(南方科技大学计算机科学与工程系);Pinscreen and Mohamed bin Zayed University of Artificial Intelligence(Pinscreen和马尔代夫人工智能大学)
Mining Multi-Modality Spatio-Temporal Cues for Video Important Person Identification
School of Computer Science and Technology, Wuhan University of Science and Technology(武汉科技大学计算机科学与技术学院);Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan University of Science and Technology(湖北省智能信息处理与实时工业系统重点实验室);School of Computer Science, National Engineering Research Center for Multimedia Software, Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University(计算机科学学院,国家多媒体软件工程技术研究中心,湖北省多媒体与网络通信工程重点实验室,武汉大学);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学)
Institute of Computing Technology(计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Huawei Technologies(华为技术有限公司);University of Science and Technology of China(中国科学技术大学)
In recent years, creative content generations like style transfer and neural photo editing have attracted more and more attention. Among these, cartoonization of real-world scenes has promising applications in entertainment and industry. Different from image translations focusing on improving the style effect of generated images, video cartoonization has additional requirements on the temporal consistency. In this paper, we propose a spatially-adaptive semantic alignment framework with perceptual motion consistency for coherent video cartoonization in an unsupervised manner. The semantic align…
ERMV: Editing 4D Robotic Multi-view images to enhance embodied agents
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University and Key Laboratory of System Control and Information Processing, Ministry of Education of China(自动化与智能感知学院,上海交通大学;系统控制与信息处理重点实验室,中华人民共和国教育部)
SimInsert: Seamless Video Object Insertion via Regional Sparse Attention Fusion
State Key Laboratory of Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学);School of Intelligence Science and Technology, Nanjing University(智能科学与技术学院,南京大学);JIUTIAN Research(中移九天);Xi’an Jiaotong-Liverpool University(西安交通大学利物浦大学);Zhejiang Sci-Tech University(浙江科技学院);The University of British Columbia(不列颠哥伦比亚大学)
The task of action recognition or action detection involves analyzing videos and determining what action or motion is being performed. The primary subject of these videos are predominantly humans performing some action. However, this requirement can be relaxed to generalize over other subjects such as animals or robots. The applications can range from anywhere between human-computer inter-action to automated video editing proposals. When we consider spatiotemporal action recognition, we deal with action localization. This task not only involves determining what action is being performed but al…
We introduce Emu3.5, a large-scale multimodal world model that natively predicts the next state across vision and language. Emu3.5 is pre-trained end-to-end with a unified next-token prediction objective on a corpus of vision-language interleaved data containing over 10 trillion tokens, primarily derived from sequential frames and transcripts of internet videos. The model naturally accepts interleaved vision-language inputs and generates interleaved vision-language outputs. Emu3.5 is further post-trained with large-scale reinforcement learning to enhance multimodal reasoning and generation. To…
We present an implicit neural representation to learn the spatio-temporal space of kinematic motions. Unlike previous work that represents motion as discrete sequential samples, we propose to express the vast motion space as a continuous function over time, hence the name Neural Motion Fields (NeMF). Specifically, we use a neural network to learn this function for miscellaneous sets of motions, which is designed to be a generative model conditioned on a temporal coordinate $t$ and a random vector $z$ for controlling the style. The model is then trained as a Variational Autoencoder (VAE) with m…