Space Group Informed Transformer for Crystalline Materials Generation
Beijing National Laboratory for Condensed Matter Physics(北京凝聚态物理国家实验室);School of Physical Sciences, University of Chinese Academy of Sciences, Beijing 100190, China(中国科学院大学物理科学学院,北京100190,中国);Key Laboratory of Material Simulation Methods and Software of Ministry of Education, College of Physics, Jilin University, Changchun 130012, China(教育部材料模拟方法与软件重点实验室,吉林大学物理学院,长春130012,中国);State Key Laboratory of Superhard Materials, College of Physics, Jilin University, Changchun 130012, P. R. China(超硬材料国家重点实验室,吉林大学物理学院,长春130012,中华人民共和国);Songshan Lake Materials Laboratory, Dongguan, Guangdong 523808, China(松山湖材料实验室,东莞,广东523808,中国)
We introduce CrystalFormer, a transformer-based autoregressive model specifically designed for space group-controlled generation of crystalline materials. By explicitly incorporating space group symmetry, CrystalFormer greatly reduces the effective complexity of crystal space, which is essential for data-and compute-efficient generative modeling of crystalline materials. Leveraging the prominent discrete and sequential nature of the Wyckoff positions, CrystalFormer learns to generate crystals by directly predicting the species and coordinates of symmetry-inequivalent atoms in the unit cell. We demonstrate the advantages of CrystalFormer in standard tasks such as symmetric structure initialization and element substitution over widely used conventional approaches. Furthermore, we showcase its plug-and-play application to property-guided materials design, highlighting its flexibility. Our analysis reveals that CrystalFormer ingests sensible solid-state chemistry knowledge and heuristics by compressing the material dataset, thus enabling systematic exploration of crystalline materials space. The simplicity, generality, and adaptability of CrystalFormer position it as a promising architecture to be the foundational model of the entire crystalline materials space, heralding a new era in materials discovery and design.
National Center for Supercomputing Applications, University of Illinois Urbana-Champaign(国家超级计算中心,伊利诺伊大学厄巴纳-香槟分校);Beckman Institute for Advanced Science and Technology, University of Illinois Urbana-Champaign(贝克曼先进科学与技术研究所,伊利诺伊大学厄巴纳-香槟分校);The Grainger College of Engineering, Department of Aerospace Engineering, University of Illinois Urbana-Champaign(格拉inger工程学院,航空航天工程系,伊利诺伊大学厄巴纳-香槟分校);The Grainger College of Engineering, Department of Mechanical Science and Engineering, University of Illinois Urbana-Champaign(格拉inger工程学院,机械科学与工程系,伊利诺伊大学厄巴纳-香槟分校);The Grainger College of Engineering, Department of Civil and Environmental Engineering, University of Illinois Urbana-Champaign(格拉inger工程学院,土木与环境工程系,伊利诺伊大学厄巴纳-香槟分校);Civil and Urban Engineering Department, New York University Abu Dhabi(纽约大学阿布扎比分校土木与城市工程系);Department of Industrial and Manufacturing Systems Engineering, Kansas State University(工业与制造系统工程系,堪萨斯州立大学)
PINN-Cast: Exploring the Role of Continuous-Depth NODE in Transformers and Physics Informed Loss as Soft Physical Constraints in Short-term Weather Forecasting
Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants
Mondragon University(蒙dragon大学);University of the Basque Country (UPV/EHU)(巴斯克大学);Basque Centre for Applied Mathematics(巴斯克应用数学中心);KTH, Royal Institute of Technology(皇家理工学院);Ikerbasque, Basque Foundation for Science(巴斯克科学基金会)
Bayesian Physics Informed Neural Networks for Reliable Transformer Prognostics
Department of Electronics & Computing, Mondragon University(蒙龙大学电子与计算系);Department of Physical Chemistry, University of the Basque Country (UPV/EHU)(巴斯克国家大学物理化学系);Department of Mathematics, University of the Basque Country (UPV/EHU)(巴斯克国家大学数学系);Department of Computer Science and Artificial Intelligence, University of the Basque Country (UPV/EHU)(巴斯克国家大学计算机科学与人工智能系);Basque Centre for Applied Mathematics (BCAM)(巴斯克应用数学中心);Ikerbasque, Basque Foundation for Science(巴斯克科学基金会)
Physics-Informed Neural Networks with Fourier Features and Attention-Driven Decoding
High Technology High School(高科技高中);Institute for Gravitation and the Cosmos(引力与宇宙研究所);The Pennsylvania State University(宾夕法尼亚州立大学);Department of Physics(物理系);Princeton University(普林斯顿大学)
Disentangling Aleatoric and Epistemic Uncertainty in Physics-Informed Neural Networks. Application to Insulation Material Degradation Prognostics
Department of Electronics and Computer Science(电子与计算机科学系);Mondragon University(蒙得龙大学);Research & Development(研发);CARTIF Technology Centre(CARTIF技术中心);Department of Physical Chemistry(物理化学系);University of the Basque Country (EHU)(巴斯克国家大学(EHU));Basque Centre for Applied Mathematics (BCAM)(巴斯克应用数学中心(BCAM));Department of Computer Science and Artificial Intelligence(计算机科学与人工智能系)
Interpretable Physics-Informed Load Forecasting for U.S. Grid Resilience: SHAP-Guided Ensemble Validation in Hybrid Deep Learning Under Extreme Weather
1Dept. of Computer Science, Midwestern State University, Dallas, TX, USA 2O\&M Analytics, AES Clean Energy, The AES Corporation, Louisville, CO, USA 3Dept. of Information;Communication Engineering, Pabna University of Science
On the training of physics-informed neural operators for solving parametric partial differential equations
College of Civil Engineering, Tongji University(同济大学土木工程学院);Department of Engineering Science, University of Oxford(牛津大学工程科学系);Institute for Foundations of Data Science, Yale University(耶鲁大学数据科学基础研究所)
Physics-Informed Glass-Structure Descriptors for Assessing the Intrinsic Reactivity of Mixed Amorphous-Crystalline Precursors in Alkali-Activated Materials
Hebei University of Technology(河北工业大学);Rice University(莱斯大学);Rice Advanced Materials Institute, Rice University(莱斯大学先进材料研究所);Ken Kennedy Institute, Rice University(莱斯大学肯·肯尼迪研究所)
Development of a Physics-Informed Neural Framework, MEOWN, for Rapid Prediction of Muon Stopping Sites in Crystalline Materials, for understanding Quantum Magnet employing Muon Spectroscopy
Rajiv Gandhi Institute of Petroleum Technology(拉吉夫·甘地石油技术学院)
Discovery and recovery of crystalline materials with property-conditioned transformers
University College London(伦敦大学学院);Earth Rover Program(地球漫游者计划);Queen Mary University of London(伦敦大学玛丽女王学院);NOMATEN Centre of Excellence, National Centre for Nuclear Research(国家核研究中心NOMATEN卓越中心)