Image-to-video (I2V) generation tasks always suffer from keeping high fidelity in the open domains. Traditional image animation techniques primarily focus on specific domains such as faces or human poses, making them difficult to generalize to open domains. Several recent I2V frameworks based on diffusion models can generate dynamic content for open domain images but fail to maintain fidelity. We found that two main factors of low fidelity are the loss of image details and the noise prediction biases during the denoising process. To this end, we propose an effective method that can be applied to mainstream video diffusion models. This method achieves high fidelity based on supplementing more precise image information and noise rectification. Specifically, given a specified image, our method first adds noise to the input image latent to keep more details, then denoises the noisy latent with proper rectification to alleviate the noise prediction biases. Our method is tuning-free and plug-and-play. The experimental results demonstrate the effectiveness of our approach in improving the fidelity of generated videos. For more image-to-video generated results, please refer to the project website: https://noise-rectification.github.io.
VideoAWR: Reinforcing Video Flow Models with Decoupled-Timestep Advantage-Weighted Rectification
Beihang University(北京航空航天大学);Zhongguancun Academy(中关村学院);Joy Future Academy(京东探索研究院);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学)
针对视频流模型扩展 AWR 时的冷启动与引导爆炸问题,提出 VideoAWR 解耦采样与优化锚点,结合解耦时间步校正,在 Wan2.1 上实现稳定优化与更优视频质量。
SNR-Edit: Structure-Aware Noise Rectification for Inversion-Free Flow-Based Editing
State Key lab of CAD\&CG, Zhejiang University(CAD与CG国家重点实验室,浙江大学);UniTTEC Co. Ltd.(UniTTEC有限公司);Research center, Ningbo Meidong Container Terminal Co.,Ltd.(宁波梅东集装箱码头有限公司研究中心)
Tuning-Free Image Editing with Fidelity and Editability via Unified Latent Diffusion Model
Communication University of China(中国传媒大学);National University of Singapore(新加坡国立大学);University of California at Merced(加州大学默塞德分校);Yonsei University(延世大学)
FaceSnap: Enhanced ID-fidelity Network for Tuning-free Portrait Customization
Vision AI System Lab, Nanjing University, Nanjing, China(南京大学视觉人工智能系统实验室);Research and Development Department, Wonxing Technology, Shanghai, China(Wonxing技术有限公司研发部)
TUNI: Unifying Pre-training and Fine-tuning with Modality-Aware Mutual Learning and Rectification for RGB-T Semantic Segmentation
School of Automation, Beijing Institute of Technology(自动化学院,北京理工大学);School of Computer Science, Wuhan University(计算机学院,武汉大学);Great Wall Motor(长城汽车);School of Information and Electronic Engineering, Zhejiang University of Science and Technology(信息电子工程学院,浙江理工大学)
Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalization
Sun Yat-sen University(中山大学);East China Normal University(华东师范大学);Guangdong University of Technology(广东工业大学);The Hong Kong Polytechnic University(香港理工大学)
Tuning-Free Long Video Generation via Global-Local Collaborative Diffusion
Space AI, Li Auto(理想汽车太空智能部门);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学);University of New South Wales(新南威尔士大学);Singapore University of Technology and Design(新加坡科技设计大学)
StableMotion: One-Step Motion Estimation with Diffusion Prior
Yingcai Honors College, University of Electronic Science and Technology of China(电子科技大学英才实验学院);School of Information and Communication Engineering, University of Electronic Science and Technology of China(电子科技大学信息与通信工程学院);Paradigm Inc.(第四范式);School of Aeronautics and Astronautics, Sichuan University(四川大学航空航天学院);School of Artificial Intelligence, Chongqing University of Posts and Telecommunications(重庆邮电大学人工智能学院)
Enhancing the Reliability of Segment Anything Model for Auto-Prompting Medical Image Segmentation with Uncertainty Rectification
Artificial Intelligence Innovation and Incubation Institute, Fudan University(复旦大学人工智能创新与孵化研究院);Shanghai Academy of Artificial Intelligence for Science(上海科学人工智能研究院)
Spectral Rectification for Parameter-Efficient Adaptation of Foundation Models in Colonoscopy Depth Estimation
Department of Biomedical Engineering, National University of Singapore, Singapore(新加坡国立大学生物医学工程系);Department of Radiotherapy, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, China(西安交通大学第一附属医院放疗科)
Automated Tuning for Diffusion Inverse Problem Solvers without Generative Prior Retraining
Department of Electrical \& Computer Engineering, University of Minnesota, MN, USA;Center for Magnetic Resonance Research, University of Minnesota, MN, USA
A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Model
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Zhejiang University(浙江大学)