Simulation metamodeling refers to the construction of lower-fidelity models to represent input-output relations using few simulation runs. Stochastic kriging, which is based on Gaussian process, is a versatile and common technique for such a task. However, this approach relies on specific model assumptions and could encounter scalability challenges. In this paper, we study an alternative metamodeling approach using prediction intervals to capture the uncertainty of simulation outputs. We cast the metamodeling task as an empirical constrained optimization framework to train prediction intervals that attain accurate prediction coverage and narrow width. We specifically use neural network to represent these intervals and discuss procedures to approximately solve this optimization problem. We also present an adaptation of conformal prediction tools as another approach to construct prediction intervals for metamodeling. Lastly, we present a validation machinery and show how our method can enjoy a distribution-free finite-sample guarantee on the prediction performance. We demonstrate and compare our proposed approaches with existing methods including stochastic kriging through numerical examples.
Learning to Simulate: Generative Metamodeling via Quantile Regression
Department of Industrial and Systems Engineering, University of Minnesota(明尼苏达大学工业与系统工程系);School of Data Science, Fudan University(复旦大学数据科学学院);School of Management, Fudan University(复旦大学管理学院);Department of Decision Analytics and Operations, City University of Hong Kong(香港城市大学决策分析与运营系);Department of Industrial Engineering and Decision Analytics, The Hong Kong University of Science and Technology(香港科技大学工业工程与决策分析系)
E-QRGMM: Efficient Generative Metamodeling for Covariate-Dependent Uncertainty Quantification
School of Data Science(数据科学学院);School of Management(管理学院);Department of Decision Analytics and Operations, College of Business, City University of Hong Kong(商业学院决策分析与运营系,香港城市大学);The Laboratory for AI-Powered Financial Technologies(人工智能赋能金融技术实验室)
Frequentist prediction intervals for random-effects meta-analysis via confidence-distribution propagation
The Institute of Statistical Mathematics(日本统计科学研究所);The Graduate University for Advanced Studies(高级研究院);Institute of Medical Biometry and Statistics, Medical Faculty and Medical Center – University of Freiburg(弗赖堡大学医学院医学中心医学生物统计学与统计研究所)
Efficient conformal prediction intervals for time series: Online PID-Expert aggregation
School of Economics and Management, Beihang University(北京航空航天大学经济管理学院);Beijing Shuji Intelligent Technology Co., Ltd.(北京数智科技有限公司);Center for Forecasting Science, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院预测科学中心)
Extreme Conformal Prediction: Reliable Intervals for High-Impact Events
Research Institute for Statistics and Information Science, University of Geneva(日内瓦大学统计与信息科学研究院);Department of Industrial Engineering and Operations Research, Columbia University(哥伦比亚大学工业工程与运筹学系)
Conformal Prediction Intervals with Tail-Specific Guarantees
Sapienza University of Rome(罗马第一大学);MEMOTEF Department, Sapienza University of Rome(罗马第一大学MEMOTEF系);MRC–Biostatistics Unit, University of Cambridge(剑桥大学医学研究委员会生物统计单元)
Impact of existence and nonexistence of pivot on the coverage of empirical best linear prediction intervals for small areas
Department of Mathematics & Statistics, Eastern Kentucky University(东肯塔基大学数学与统计系);Institute of Mathematics for Industry, Kyushu University(九州大学产业数学研究所);Joint Program in Survey Methodology, University of Maryland(马里兰大学联合调查方法项目)
CONTINA: Confidence Interval for Traffic Demand Prediction with Coverage Guarantee
Key Laboratory of Road and Traffic Engineering, Ministry of Education at Tongji University(同济大学交通工程教育重点实验室);Urban Mobility Institute, Tongji University(同济大学城市交通研究院)
The MAPS Algorithm: Fast model-agnostic and distribution-free prediction intervals for supervised learning
Department of Mathematics, Imperial College London(帝国理工学院伦敦数学系);Great Ormond Street Institute of Child Health(大奥蒙街儿童健康研究所);Instituto de Ingeniería Biomédica, Universidad de Buenos Aires(布宜诺斯艾利斯大学生物医学工程研究所)
Questioning the Coverage-Length Metric in Conformal Prediction: When Shorter Intervals Are Not Better
School of Statistics and Data Science, Shanghai University of Finance and Economics, Shanghai, China(上海财经大学统计与数据科学学院);Department of Statistics and Data Science, Fudan University, Shanghai, China(复旦大学统计与数据科学系);Institute of Data Science and Statistics, Shanghai University of Finance and Economics, Shanghai, China(上海财经大学数据科学与统计研究所)
Climate-Invariant Conformal Prediction Intervals for Multi-Horizon Solar and Wind Forecasting
Department of Computer Science and Engineering, Dr. B.R. Ambedkar National Institute of Technology Jalandhar, India(计算机科学与工程系,Dr. B.R. 阿姆贝卡尔国立技术学院贾兰德哈尔,印度)