DIVE: Taming DINO for Subject-Driven Video Editing
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(深圳先进技术研究院,中国科学院);vivo AI Lab(vivo AI实验室);NVIDIA;Adobe Research(Adobe研究院);Shenzhen University(深圳大学);Southeast University(东南大学);Shenzhen University of Advanced Technology(深圳大学先进技术学院)
Yi Huang, Wei Xiong, He Zhang, Chaoqi Chen, Jianzhuang Liu, Mingfu Yan, Shifeng Chen
Building on the success of diffusion models in image generation and editing, video editing has recently gained substantial attention. However, maintaining temporal consistency and motion alignment still remains challenging. To address these issues, this paper proposes DINO-guided Video Editing (DIVE), a framework designed to facilitate subject-driven editing in source videos conditioned on either target text prompts or reference images with specific identities. The core of DIVE lies in leveraging the powerful semantic features extracted from a pretrained DINOv2 model as implicit correspondences to guide the editing process. Specifically, to ensure temporal motion consistency, DIVE employs DINO features to align with the motion trajectory of the source video. For precise subject editing, DIVE incorporates the DINO features of reference images into a pretrained text-to-image model to learn Low-Rank Adaptations (LoRAs), effectively registering the target subject's identity. Extensive experiments on diverse real-world videos demonstrate that our framework can achieve high-quality editing results with robust motion consistency, highlighting the potential of DINO to contribute to video editing. Project page: https://dino-video-editing.github.io
DUET-DINO: Simultaneous Cross-View World Modeling for Latent Planning in Robot Manipulation
University of Technology Nuremberg (UTN)(纽伦堡工业大学);Technical University of Munich (TUM)(慕尼黑工业大学);Karlsruhe Institute of Technology (KIT)(卡尔斯鲁厄理工学院);NVIDIA(英伟达);Robotics Institute Germany(德国机器人研究所)
ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis
Peking University(北京大学);Peng Cheng Laboratory(鹏城实验室);The Chinese University of Hong Kong(香港中文大学);Tencent AI Lab(腾讯AI Lab);ARC Lab, Tencent PCG(腾讯PCG ARC实验室);Hong Kong University of Science and Technology(香港科技大学);Monash University(莫纳什大学)
Taming the Tri-Space Tension: ARC-Guided Hallucination Modeling and Control for Text-to-Image Generation
University of Bristol(布里斯托大学);South China University of Technology(华南理工大学);Brown University(布朗大学);Xi’an Jiaotong University(西安交通大学);Arizona State University(亚利桑那州立大学)
UniMC: Taming Diffusion Transformer for Unified Keypoint-Guided Multi-Class Image Generation
The Hong Kong University of Science and Technology(香港科学与技术大学);The University of Hong Kong(香港大学);The Chinese University of Hong Kong(香港中文大学);Peking University(北京大学);Tencent(腾讯);National University of Singapore(新加坡国立大学)
LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Exploring scalable medical image encoders beyond text supervision
Health Futures, Microsoft Research(微软研究院健康未来部门);Microsoft Health and Life Sciences(微软健康与生命科学部门);Department of Radiology, University of Cambridge(剑桥大学放射学系);Cambridge University Hospitals NHS Foundation Trust(剑桥大学医院NHS基金会信托);Microsoft Azure AI(微软Azure人工智能部门)
CaTok: Taming Mean Flows for One-Dimensional Causal Image Tokenization
Institute of Trustworthy Embodied AI, Fudan University(可信具身人工智能研究所,复旦大学);Shanghai Innovation Institute(上海创新研究院);Shanghai Key Laboratory of Multimodal Embodied AI(上海多模态具身人工智能重点实验室)
Dino U-Net: Exploiting High-Fidelity Dense Features from Foundation Models for Medical Image Segmentation
School of Biomedical Engineering (Suzhou), Division of Life Science and Medicine, University of Science and Technology of China, Hefei, China(生物医学工程学院(苏州),生命科学与医学系,中国科学技术大学,合肥,中国);Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China(苏州生物医学工程与技术研究所,中国科学院,苏州,中国);Shanghai Innovation Institute, Shanghai, China(上海创新研究院,上海,中国);Medical School of Tianjin University, Tianjin, China(天津大学医学院,天津,中国);Jinan Guoke Medical and Technology Development Co., Ltd., Pharmaceutical Valley New Drug Creation Platform, Jinan, China(济南国科医药科技发展有限公司,药谷新药创制平台,济南,中国)
TACO: Taming Diffusion for in-the-wild Video Amodal Completion
State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室);State Key Laboratory of General Artificial Intelligence, BIGAI(BIGAI 通用人工智能国家重点实验室);Tsinghua University(清华大学)