Empowering Diffusion Models on the Embedding Space for Text Generation
School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；State Key Laboratory of Cognitive Intelligence（认知智能国家重点实验室）；Microsoft Research Asia（微软亚洲研究院）
Diffusion models have achieved state-of-the-art synthesis quality on both visual and audio tasks, and recent works further adapt them to textual data by diffusing on the embedding space. In this paper, we conduct systematic studies of the optimization challenges encountered with both the embedding space and the denoising model, which have not been carefully explored. Firstly, the data distribution is learnable for embeddings, which may lead to the collapse of the embedding space and unstable training. To alleviate this problem, we propose a new objective called the anchor loss which is more efficient than previous methods. Secondly, we find the noise levels of conventional schedules are insufficient for training a desirable denoising model while introducing varying degrees of degeneration in consequence. To address this challenge, we propose a novel framework called noise rescaling. Based on the above analysis, we propose Difformer, an embedding diffusion model based on Transformer. Experiments on varieties of seminal text generation tasks show the effectiveness of the proposed methods and the superiority of Difformer over previous state-of-the-art embedding diffusion baselines.
DiffPuter: Empowering Diffusion Models for Missing Data Imputation
Computer Science Department, University of Illinois at Chicago（伊利诺伊大学芝加哥分校计算机科学系）；Eric and Wendy Schmidt Center, Broad Institute of MIT and Harvard（MIT和哈佛大学Broad研究所埃里克和文迪·施密特中心）
Self-conditioned Embedding Diffusion for Text Generation
INRIA（法国国家信息与自动化研究所）；École normale supérieure（巴黎高等师范学院）；CNRS（法国国家科学研究中心）；PSL Research University（巴黎文理研究大学）；DeepMind（深度思维）；Massachusetts Institute of Technology（麻省理工学院）
Empowering machine learning models with contextual knowledge for enhancing the detection of eating disorders in social media posts
Universidad de León（莱昂大学）；Leibniz University of Hannover（汉诺威莱布尼茨大学）；L3S Research Center（L3S研究中心）；TIB Leibniz Information Centre for Science and Technology（莱布尼茨科学与技术信息中心）；Bakken and Baeck GmbH（Bakken and Baeck有限公司）
Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrast
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Renmin University of China（中国人民大学）；Xiaohongshu Inc.（小红书公司）
CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
LUMIA Lab（LUMIA实验室）；School of Artificial Intelligence（人工智能学院）；Shanghai Jiao Tong University（上海交通大学）；Shanghai Innovation Institute（上海创新研究院）；School of Computer Science（计算机科学学院）；Shanghai AI Laboratory（上海人工智能实验室）
Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
Nanyang Technological University（南洋理工大学）；Yale University（耶鲁大学）；NYU Shanghai（纽约大学上海校区）；Alibaba-NTU Singapore Joint Research Institute（阿里巴巴-国立科技大学新加坡联合研究机构）；Center for Data Science New York University（纽约大学数据科学中心）
Analyzing Diffusion and Autoregressive Vision Language Models in Multimodal Embedding Space
Nanyang Technological University（南洋理工大学）；Yale University（耶鲁大学）；NYU Shanghai（纽约大学上海分校）；Alibaba-NTU Singapore Joint Research Institute（阿里-国立新加坡大学联合研究机构）；University of the Chinese Academy of Sciences（中国科学院大学）；Center for Data Science（数据科学中心）；New York University（纽约大学）
