Much effort has been devoted to making large and more accurate models, but relatively little has been put into understanding which examples are benefiting from the added complexity. In this paper, we demonstrate and analyze the surprisingly tight link between a model's predictive uncertainty on individual examples and the likelihood that larger models will improve prediction on them. Through extensive numerical studies on the T5 encoder-decoder architecture, we show that large models have the largest improvement on examples where the small model is most uncertain. On more certain examples, even those where the small model is not particularly accurate, large models are often unable to improve at all, and can even perform worse than the smaller model. Based on these findings, we show that a switcher model which defers examples to a larger model when a small model is uncertain can achieve striking improvements in performance and resource usage. We also explore committee-based uncertainty metrics that can be more effective but less practical.
Predicting Many Crystal Properties via an Adaptive Transformer-based Framework
Fudan University（复旦大学）；Shanghai Research Center for Quantum Sciences（上海量子科学研究中心）；Institute for Nanoelectronic Devices and Quantum Computing, Zhangjiang Fudan International Innovation Center, Fudan University（复旦大学张江国际创新中心纳米电子器件与量子计算研究所）；Hefei National Laboratory（合肥国家实验室）
DeepRepViz: Identifying Confounders in Deep Learning Model Predictions
Humboldt-Universität zu Berlin（柏林洪堡大学）；Charité – Universitätsmedizin Berlin（柏林夏里特医学院）；Einstein Center for Neurosciences Berlin（柏林爱因斯坦神经科学中心）；Freie Universität Berlin（柏林自由大学）
Identifying Interaction Sites in "Recalcitrant" Proteins: Predicted Protein and Rna Binding Sites in Rev Proteins of Hiv-1 and Eiav Agree with Experimental Data
Predicting Early Stages Of Alzheimer's Disease And Identifying Key Biomarkers Using Deep Artificial Neural Network And Ensemble Of Machine Learning Methodologies
Explainable artificial intelligence model for identifying Market Value in Professional Soccer Players
Zhili College（致理书院）；Tsinghua University（清华大学）；Research Centre for Athletic Performance and Data Science & Division of Sports Science and Physical Education（运动表现与数据科学研究中心&体育科学与体育教育系）
Measuring Financial Time Series Similarity With a View to Identifying Profitable Stock Market Opportunities
School of Computer Science, University College Dublin（都柏林大学学院计算机科学学院）；Insight Centre for Data Analytics, University College Dublin（都柏林大学学院洞察数据分析中心）；School of Economics and Management, Beihang University（北京航空航天大学经济管理学院）
