Recently, diffusion models have made remarkable progress in text-to-image (T2I) generation, synthesizing images with high fidelity and diverse contents. Despite this advancement, latent space smoothness within diffusion models remains largely unexplored. Smooth latent spaces ensure that a perturbation on an input latent corresponds to a steady change in the output image. This property proves beneficial in downstream tasks, including image interpolation, inversion, and editing. In this work, we expose the non-smoothness of diffusion latent spaces by observing noticeable visual fluctuations resulting from minor latent variations. To tackle this issue, we propose Smooth Diffusion, a new category of diffusion models that can be simultaneously high-performing and smooth. Specifically, we introduce Step-wise Variation Regularization to enforce the proportion between the variations of an arbitrary input latent and that of the output image is a constant at any diffusion training step. In addition, we devise an interpolation standard deviation (ISTD) metric to effectively assess the latent space smoothness of a diffusion model. Extensive quantitative and qualitative experiments demonstrate that Smooth Diffusion stands out as a more desirable solution not only in T2I generation but also across various downstream tasks. Smooth Diffusion is implemented as a plug-and-play Smooth-LoRA to work with various community models. Code is available at https://github.com/SHI-Labs/Smooth-Diffusion.
See the Speaker: Crafting High-Resolution Talking Faces from Speech with Prior Guidance and Region Refinement
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Speech and Acoustic Laboratory, Joy Future Academy, Jingdong Corporation（语音与声学实验室、Joy Future Academy、京东公司）；Aarhus University（阿arhus大学）
Crafting Your Evolving Dreams: Concept-Incremental Versatile Customization
Mohamed bin Zayed University of Artificial Intelligence（Mohamed bin Zayed大学人工智能学院）；University of Trento（特伦托大学）；Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University（香港理工大学土木与环境工程系）；South China University of Technology（华南理工大学）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Institute of Big Data, Fudan University（复旦大学大数据研究院）；Shanghai Jiao Tong University（上海交通大学）
Smooth image-to-image translations with latent space interpolations
University of Trento（特伦托大学）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）；Shanghai Jiao Tong University（上海交通大学）；Tencent AI Lab（腾讯人工智能实验室）
LatentStealth: Unnoticeable and Efficient Adversarial Attacks on Expressive Human Pose and Shape Estimation
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University（北京未来区块链与隐私计算高级创新中心，人工智能学院，北航）；College of Cyber Security, Jinan University（网络安全学院，暨南大学）；National University of Singapore（新加坡国立大学）；School of Computer Science and Information Engineering, Hefei University of Technology（计算机科学与信息工程学院，合肥工业大学）；School of Computer Science and Engineering, Beihang University（计算机科学与工程学院，北航）
An Image Clustering Auto-Encoder Based on Predefined Evenly-Distributed Class Centroids and MMD Distance
Shanghai University（上海大学）
本文提出了一种新颖、有效、简单的端到端图像聚类自动编码算法：ICAE。该算法以PEDCC（Predefined Evenly-Distributed Class Centroids）为聚类中心，保证潜在特征的类间距离最大，并加入数据分布约束、数据增强约束、自动编码器重构约束和Sobel平滑约束，以提高聚类性能。具体来说，我们执行一对一的数据增强来学习更有效的特征。数据和增强数据被同时输入到自动编码器中以获得潜在特征和增强潜在特征，其相似性受到增强损失的约束。然后，利用最大平均差异距离（MMD），我们结合潜在特征和增强的潜在特征，使其分布接近PEDCC分布（类间均匀分布，类内Dirac分布），以进一步学习面向聚类的特征。同时，以原始输入图像和重建图像的均方误差作为重建约束，并利用Sobel平滑损失建立泛化约束，提高泛化能力。最后，在三个常用数据集MNIST，Fashion-MNIST，COIL 20上进行了广泛的实验。实验结果表明，该算法取得了迄今为止最好的聚类结果。此外，我们可以使用预定义的PEDCC类中心，以及解码器来清楚地生成每个类的样本。代码可以在https：//github.com/zyWang-Power/Clustering下载！
