We propose Uni-Animator, a novel Diffusion Transformer (DiT)-based framework for unified image and video sketch colorization. Existing sketch colorization methods struggle to unify image and video tasks, suffering from imprecise color transfer with single or multiple references, inadequate preservation of high-frequency physical details, and compromised temporal coherence with motion artifacts in large-motion scenes. To tackle imprecise color transfer, we introduce visual reference enhancement via instance patch embedding, enabling precise alignment and fusion of reference color information. To resolve insufficient physical detail preservation, we design physical detail reinforcement using physical features that effectively capture and retain high-frequency textures. To mitigate motion-induced temporal inconsistency, we propose sketch-based dynamic RoPE encoding that adaptively models motion-aware spatial-temporal dependencies. Extensive experimental results demonstrate that Uni-Animator achieves competitive performance on both image and video sketch colorization, matching that of task-specific methods while unlocking unified cross-domain capabilities with high detail fidelity and robust temporal consistency.
ID-Animator: Zero-Shot Identity-Preserving Human Video Generation
University of Science and Technology of China（中国科学技术大学）；Hefei Institute of Physical Science, Chinese Academy of Sciences（中国科学院合肥物质科学研究院）；LightSpeed Studios, Tencent（腾讯天美工作室）
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Nanyang Technological University（南洋理工大学）；Applied Research Center (ARC), Tencent PCG（腾讯平台与内容事业群应用研究中心）
Video colorization is a challenging and highly ill-posed problem. Although recent years have witnessed remarkable progress in single image colorization, there is relatively less research effort on video colorization and existing methods always suffer from severe flickering artifacts (temporal inconsistency) or unsatisfying colorization performance. We address this problem from a new perspective, by jointly considering colorization and temporal consistency in a unified framework. Specifically, we propose a novel temporally consistent video colorization framework (TCVC). TCVC effectively propaga…
Microscopy images are powerful tools and widely used in the majority of research areas, such as biology, chemistry, physics and materials fields by various microscopies (scanning electron microscope (SEM), atomic force microscope (AFM) and the optical microscope, et al.). However, most of the microscopy images are colorless due to the unique imaging mechanism. Though investigating on some popular solutions proposed recently about colorizing images, we notice the process of those methods are usually tedious, complicated, and time-consuming. In this paper, inspired by the achievement of machine …
Hong Kong University of Science and Technology（香港科技大学）；USC Institute for Creative Technologies（南加州大学创意技术研究所）；City University of Hong Kong（香港城市大学）；Microsoft Research Asia（微软亚洲研究院）；Microsoft AI Perception and Mixed Reality（微软人工智能感知与混合现实部门）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）
This paper presents the first end-to-end network for exemplar-based video colorization. The main challenge is to achieve temporal consistency while remaining faithful to the reference style. To address this issue, we introduce a recurrent framework that unifies the semantic correspondence and color propagation steps. Both steps allow a provided reference image to guide the colorization of every frame, thus reducing accumulated propagation errors. Video frames are colorized in sequence based on the colorization history, and its coherency is further enforced by the temporal consistency loss. All…
Beijing Electronic Science and Technology Institute（北京电子科技学院）；SenseTime Research（商汤科技研究院）；Qing Yuan Research Institute, Shanghai Jiao Tong University（上海交通大学清源研究院）
In industry, there exist plenty of scenarios where old gray photos need to be automatically colored, such as video sites and archives. In this paper, we present the HistoryNet focusing on historical person's diverse high fidelity clothing colorization based on fine grained semantic understanding and prior. Colorization of historical persons is realistic and practical, however, existing methods do not perform well in the regards. In this paper, a HistoryNet including three parts, namely, classification, fine grained semantic parsing and colorization, is proposed. Classification sub-module suppl…
Tampere University（坦佩雷大学）；Aalto University of Technology（阿尔托理工大学）
This paper presents a generic face animator that is able to control the pose and expressions of a given face image. The animation is driven by human interpretable control signals consisting of head pose angles and the Action Unit (AU) values. The control information can be obtained from multiple sources including external driving videos and manual controls. Due to the interpretable nature of the driving signal, one can easily mix the information between multiple sources (e.g. pose from one image and expression from another) and apply selective post-production editing. The proposed face animato…
FSCM: Frequency-Enhanced Spatial-Spectral Coupled Mamba for Infrared Hyperspectral Image Colorization
School of Electronic and Optical Engineering, Nanjing University of Science and Technology（南京理工大学电子与光学工程学院）；School of Mechanical Engineering, University of Science and Technology Beijing（北京科技大学机械工程学院）；School of Instrument and Electronics, North University of China（北方大学仪器与电子学院）
PhotonSplat: 3D Scene Reconstruction and Colorization from SPAD Sensors
Department of Electrical Engineering, Indian Institute of Technology, Madras, India（电子工程系，印度理工学院Madras分校）；Department of Computer Science, University of California, San Diego, CA 92093（计算机科学系，加州大学圣地亚哥分校）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Amazon Web Services（亚马逊云科技）
This work studies feature representations for dense label propagation in video, with a focus on recently proposed methods that learn video correspondence using self-supervised signals such as colorization or temporal cycle consistency. In the literature, these methods have been evaluated with an array of inconsistent settings, making it difficult to discern trends or compare performance fairly. Starting with a unified formulation of the label propagation algorithm that encompasses most existing variations, we systematically study the impact of important implementation factors in feature extrac…
