Validation of prediction uncertainty (PU) is becoming an essential task for modern computational chemistry. Designed to quantify the reliability of predictions in meteorology, the calibration-sharpness (CS) framework is now widely used to optimize and validate uncertainty-aware machine learning (ML) methods. However, its application is not limited to ML and it can serve as a principled framework for any PU validation. The present article is intended as a step-by-step introduction to the concepts and techniques of PU validation in the CS framework, adapted to the specifics of computational chemistry. The presented methods range from elementary graphical checks to more sophisticated ones based on local calibration statistics. The concept of tightness, is introduced. The methods are illustrated on synthetic datasets and applied to uncertainty quantification data extracted from the computational chemistry literature.
Quantum-chemistry-aided identification, synthesis and experimental validation of model systems for conformationally controlled reaction studies: Separation of the conformers of 2,3-dibromobuta-1,3-diene in the gas phase
Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. II. Applications
GAUCHE: A Library for Gaussian Processes in Chemistry
University of Cambridge(剑桥大学);University of Oxford(牛津大学);Secondmind Labs(Secondmind实验室);Stanford University(斯坦福大学);Genentech(基因泰克);University of Toronto(多伦多大学);Vector Institute(向量研究所);EPFL(洛桑联邦理工学院);Cornell University(康奈尔大学);Washington State University(华盛顿州立大学);King’s College London(伦敦国王学院);University College London(伦敦大学学院);Evonik Industries AG(赢创工业集团);Syracuse University(雪城大学);IST Austria(奥地利科学技术研究所);MILA Quebec AI Institute(魁北克人工智能研究所MILA);HEC Montreal(蒙特利尔高等商学院)
ElemeNet: Multiscale Molecular Machine Learning with Uncertainty Quantification Across the Periodic Table
Department of Chemical Engineering, Massachusetts Institute of Technology(麻省理工学院化学工程系);Center for Computational Science and Engineering, Massachusetts Institute of Technology(麻省理工学院计算科学与工程中心);Institute for Theoretical Chemistry, University of Stuttgart(斯图加特大学理论化学研究所);Institute for Artificial Intelligence, University of Stuttgart(斯图加特大学人工智能研究所);Department of Chemistry, Massachusetts Institute of Technology(麻省理工学院化学系)