We propose a generative flow-induced neural architecture search algorithm. The proposed approach devices simple feed-forward neural networks to learn stochastic policies to generate sequences of architecture hyperparameters such that the generated states are in proportion with the reward from the terminal state. We demonstrate the efficacy of the proposed search algorithm on the wavelet neural operator (WNO), where we learn a policy to generate a sequence of hyperparameters like wavelet basis and activation operators for wavelet integral blocks. While the trajectory of the generated wavelet basis and activation sequence is cast as flow, the policy is learned by minimizing the flow violation between each state in the trajectory and maximizing the reward from the terminal state. In the terminal state, we train WNO simultaneously to guide the search. We propose to use the exponent of the negative of the WNO loss on the validation dataset as the reward function. While the grid search-based neural architecture generation algorithms foresee every combination, the proposed framework generates the most probable sequence based on the positive reward from the terminal state, thereby reducing exploration time. Compared to reinforcement learning schemes, where complete episodic training is required to get the reward, the proposed algorithm generates the hyperparameter trajectory sequentially. Through four fluid mechanics-oriented problems, we illustrate that the learned policies can sample the best-performing architecture of the neural operator, thereby improving the performance of the vanilla wavelet neural operator.
Wavelet Flow Matching for Multi-Scale Physics Emulation
Department of Earth and Environmental Engineering(地球与环境工程系);Columbia University(哥伦比亚大学);University of Edinburgh(爱丁堡大学);Courant Institute of Mathematical Sciences(数学科学学院);New York University(纽约大学);Scripps Institution of Oceanography(斯克里普斯海洋研究所);Halıcıoğlu Data Science Institute(哈利奇数据科学研究所);University of California San Diego(加州大学圣地亚哥分校);CUNY New York City College of Technology(纽约市立大学纽约技术学院);Lamont-Doherty Earth Observatory(拉蒙特-多伊蒂地球观测站)
Wavelet Flow: Fast Training of High Resolution Normalizing Flows
York University(约克大学);Borealis AI(北极光人工智能);Ryerson University(瑞尔森大学);Samsung AI Centre Toronto(三星人工智能中心多伦多分部);Vector Institute(向量研究所)
规范化流是一类概率生成模型,它允许快速密度计算和有效采样,并且在模拟图像等复杂分布时有效。当前方法的一个缺点是其显著的训练成本,有时需要数月的GPU训练时间才能实现最先进的结果。本文介绍了小波流,一个多尺度,规范化的流架构的基础上小波。小波流具有信号尺度的显式表示,其固有地包括较低分辨率信号的模型和较高分辨率信号的条件生成,即,超分辨率。小波流的一个主要优点是能够为高分辨率数据构建生成模型(例如,1024 x 1024图像),这是不切实际的与以前的模型。此外,在标准(低分辨率)基准测试中,Wavelet Flow在每维比特数方面与之前的归一化流具有竞争力,同时训练速度最多快15倍。
Haar Wavelet based Block Autoregressive Flows for Trajectories
Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Bosch Center for Artificial Intelligence(博世人工智能中心);CISPA Helmholtz Center for Information Security(亥姆霍兹信息安全中心)
Scale-Equivariant Generative Forecasting: Weight-Tied Dilated Convolutions, Wavelet Scattering Inputs, and Spectral-Consistency Training for Self-Similar Time Series
Department of Artificial Intelligence, School of Engineering, Westlake University(人工智能系,工程学院,西湖大学);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(数学系统科学学院,中国科学院);Fudan University(复旦大学);South China University of Technology(华南理工大学);Microsoft AI4Science(微软AI4Science)
Beyond Fixed Patches: Enhancing GPTs for Financial Prediction with Adaptive Segmentation and Learnable Wavelets
School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);School of Mathematical Sciences, Tongji University(同济大学数学科学学院);Seek Data Group, Emoney Inc.(Seek Data集团,Emoney公司)
An open unified deep graph learning framework for discovering drug leads
Nanjing University of Posts and Telecommunications(南京邮电大学);Nanyang Technological University(南洋理工大学);National Engineering Research Center of Communications and Networking(国家通信与网络工程研究中心);Smart Health Big Data Analysis and Location Services Engineering Research Center of Jiangsu Province(江苏省智慧健康大数据分析与位置服务工程研究中心);Lee Kong Chian School of Medicine(李光前医学院);School of Biological Sciences(生物科学学院);Center for Biomedical Informatics(生物医学信息学中心)
Discovering Discrete Latent Topics with Neural Variational Inference
University of Oxford(牛津大学);DeepMind
主题模型作为文档的概率生成模型已被广泛研究。传统的推理方法寻求封闭形式的推导来更新模型,然而随着这些模型的表达能力的增长,对它们的参数进行快速和准确的推理的难度也在增加。本文提出了替代神经主题建模的方法,通过提供参数化分布的主题,允许在神经变分推理的框架内通过反向传播进行训练。此外,在一个破棒结构的帮助下,我们提出了一个循环网络,它能够发现一个概念上无限数量的主题,类似于贝叶斯非参数主题模型。在MXM Song Lyrics、20个NewsGroups和Reuters News数据集上的实验结果证明了这些神经主题模型的有效性和效率。
Functional MRI Time Series Generation via Wavelet-Based Image Transform and Spectral Flow Matching for Brain Disorder Identification
School of Information Technology, Monash University Malaysia(墨尔本大学马来西亚分校信息科技学院);School of Engineering, Monash University Malaysia(墨尔本大学马来西亚分校工程学院);Statistics Program, King Abdullah University of Science and Technology(国王阿卜杜勒·阿齐兹大学科学与技术学院统计学项目)
Inverse design of 3d molecular structures with conditional generative neural networks
Technische Universität Berlin(柏林工业大学);Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究所);Korea University(高丽大学);Max-Planck-Institut für Informatik(马克斯·普朗克信息学研究所)
Shandong Provincial Key Laboratory of Network Based Intelligent Computing(山东省网络智能计算重点实验室);University of Jinan(济南大学);Jinan, 250022, China(济南市,250022,中国)