Despite significant advancements in video generation, inserting a given object into videos remains a challenging task. The difficulty lies in preserving the appearance details of the reference object and accurately modeling coherent motions at the same time. In this paper, we propose VideoAnydoor, a zero-shot video object insertion framework with high-fidelity detail preservation and precise motion control. Starting from a text-to-video model, we utilize an ID extractor to inject the global identity and leverage a box sequence to control the overall motion. To preserve the detailed appearance and meanwhile support fine-grained motion control, we design a pixel warper. It takes the reference image with arbitrary key-points and the corresponding key-point trajectories as inputs. It warps the pixel details according to the trajectories and fuses the warped features with the diffusion U-Net, thus improving detail preservation and supporting users in manipulating the motion trajectories. In addition, we propose a training strategy involving both videos and static images with a weighted loss to enhance insertion quality. VideoAnydoor demonstrates significant superiority over existing methods and naturally supports various downstream applications (e.g., talking head generation, video virtual try-on, multi-region editing) without task-specific fine-tuning.
SimInsert: Seamless Video Object Insertion via Regional Sparse Attention Fusion
State Key Laboratory of Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学);School of Intelligence Science and Technology, Nanjing University(智能科学与技术学院,南京大学);JIUTIAN Research(中移九天);Xi’an Jiaotong-Liverpool University(西安交通大学利物浦大学);Zhejiang Sci-Tech University(浙江科技学院);The University of British Columbia(不列颠哥伦比亚大学)
In-Context Brush: Zero-shot Customized Subject Insertion with Context-Aware Latent Space Manipulation
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);University of Konstanz(康斯坦茨大学);National Cheng-Kung University(国立成功大学)
DriveEditor: A Unified 3D Information-Guided Framework for Controllable Object Editing in Driving Scenes
Huazhong University of Science and Technology(华中科技大学);National Key Laboratory of Multispectral Information Intelligent Processing Technology(多光谱信息智能处理技术国家重点实验室);Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所)
Realistic and Controllable 3D Gaussian-Guided Object Editing for Driving Video Generation
School of Vehicle and Mobility, and State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University(车辆与移动系统学院,智能绿色车辆与移动系统国家重点实验室,清华大学);WUWEN AI;PhiGent Robotics
T2VAttack: Adversarial Attack on Text-to-Video Diffusion Models
Hangzhou Institute for Advanced Study, UCAS, school of Intelligent Science and Technology(杭州高等研究院,UCAS,智能科学与技术学院);State Key Laboratory of AI Safety, Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS)(人工智能安全国家重点实验室,计算技术研究所,中国科学院);University of Chinese Academy of Sciences (UCAS)(中国科学院大学)
ContextFlow: Training-Free Video Object Editing via Adaptive Context Enrichment
State Key Laboratory of General Artifical Intelligence, Peking University, Beijing, China(北京人工智能基础理论国家重点实验室,北京大学);The Hong Kong University of Science and Technology(香港科技大学)
ADELE - Adaptive Delaunay Grids for High-Fidelity Mesh-Native Reconstruction
ETH Zurich(苏黎世联邦理工学院);Beijing Institute for General Artificial Intelligence(北京通用人工智能研究院);University of Zurich(苏黎世大学);Balgrist University Hospital(巴尔格里斯特大学医院)
POCI-Diff: 3D-Layout Guided Diffusion for Controllable Synthetic Surveillance Data Generation
DISI, University of Trento(DISI,特伦托大学);Toretei S.r.l.(Toretei公司);Visual AI Lab, MIT-IBM Watson AI Lab(视觉人工智能实验室,MIT-IBM沃森人工智能实验室);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会)