Score-based generative models have had remarkable success over the last decade in generating a diverse set of visually plausible images. A variety of architectures including CNNs, U-Nets, and Transformers have been used as the score-approximation network in such diffusion modeling; however, to date, relatively little is known about how these architectural choices impact generative behavior. In this work, to provide insight into this area, we propose an analytically solvable parameterization of the score function using an expansion in a 2D orthogonal wavelet basis. In particular, we derive interpretable optimal score functions in terms of the moments of the data distribution. We use this parametrization to provide an architecture-agnostic, moment-based analysis that reveals which attributes of the data distribution tend to matter most for denoising. Our score machine is flexible enough to partially mimic the relevant inductive biases of multiple architectures, including U-Nets, and CNNs, taking a step towards understanding why different score architectures can exhibit distinct generative behavior. Since our score is solvable in terms of the moments of the data, we can begin to understand how the data distribution interacts with the score network to produce the behavior we observe in diffusion models.
Category-aware EEG image generation based on wavelet transform and contrast semantic loss
Department of Integrated Neuroanatomy and Neuroimaging, Kyoto University Graduate School of Medicine(整合神经解剖与神经影像学系,京都大学医学研究院);JancsiLab, JancsiTech(Jancsi实验室,Jancsi科技);Sino-Finland Joint AI Laboratory for Child Health of Zhejiang Province(浙江省儿童健康中芬联合人工智能实验室)
Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation
Shanghai Innovation Institute(上海创新研究院);Fudan University(复旦大学);University of Science and Technology of China(中国科学技术大学);Nanjing University(南京大学);Baidu(百度)
WDformer: A Wavelet-based Differential Transformer Model for Time Series Forecasting
School of Computer Science and Technology, Zhejiang Normal University(浙江师范大学计算机科学与技术学院);Zhejiang Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University(浙江智能教育技术与应用重点实验室)
针对时间序列多域信息利用不足及传统注意力易引入噪声的问题,提出 WDformer,融合小波多分辨率分析、反转维度注意力与差分注意力,并在多个真实数据集上取得 SOTA 表现。
Does Machine Learning Outperform Traditional Fibrosis Scores in Predicting Liver Cirrhosis Risk? A Longitudinal EHR-Based Study
Center for Health Systems Innovation, Oklahoma State University(俄克拉荷马州立大学健康系统创新中心);Department of Management Science and Information Systems, Oklahoma State University(俄克拉荷马州立大学管理科学与信息系统系);Department of Health Services Administration, University of Alabama at Birmingham(阿拉巴马大学伯明翰分校健康服务管理系);Division of Gastroenterology and Hepatology, Department of Medicine, University of Alabama at Birmingham(阿拉巴马大学伯明翰分校消化内科与肝病科);College of Medicine, The University of Oklahoma Health Campus(俄克拉荷马大学健康校园医学院);Department of Family and Community Medicine, University of Alabama at Birmingham(阿拉巴马大学伯明翰分校家庭与社区医学系)
Scale-Equivariant Generative Forecasting: Weight-Tied Dilated Convolutions, Wavelet Scattering Inputs, and Spectral-Consistency Training for Self-Similar Time Series
TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting
Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Squirrel AI(松鼠AI);Beijing Institute of Technology(北京理工大学);Chongqing University(重庆大学)