Guiding a Diffusion Transformer with the Internal Dynamics of Itself
University of Electronic Science and Technology of China（电子科技大学）；National University of Singapore（新加坡国立大学）；Sun Yat-sen University（中山大学）；North China Institute of Computer Systems Engineering（华北计算机系统工程研究所）
Xingyu Zhou, Qifan Li, Xiaobin Hu, Hai Chen, Shuhang Gu
The diffusion model presents a powerful ability to capture the entire (conditional) data distribution. However, due to the lack of sufficient training and data to learn to cover low-probability areas, the model will be penalized for failing to generate high-quality images corresponding to these areas. To achieve better generation quality, guidance strategies such as classifier free guidance (CFG) can guide the samples to the high-probability areas during the sampling stage. However, the standard CFG often leads to over-simplified or distorted samples. On the other hand, the alternative line of guiding diffusion model with its bad version is limited by carefully designed degradation strategies, extra training and additional sampling steps. In this paper, we proposed a simple yet effective strategy Internal Guidance (IG), which introduces an auxiliary supervision on the intermediate layer during training process and extrapolates the intermediate and deep layer's outputs to obtain generative results during sampling process. This simple strategy yields significant improvements in both training efficiency and generation quality on various baselines. On ImageNet 256x256, SiT-XL/2+IG achieves FID=5.31 and FID=1.75 at 80 and 800 epochs. More impressively, LightningDiT-XL/1+IG achieves FID=1.34 which achieves a large margin between all of these methods. Combined with CFG, LightningDiT-XL/1+IG achieves the current state-of-the-art FID of 1.19.
Change Guiding Network: Incorporating Change Prior to Guide Change Detection in Remote Sensing Imagery
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）；Graduate School of Frontier Sciences, The University of Tokyo（东京大学前沿科学研究生院）
