arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2102.01130cs.LGecon.EM
收藏

在预测出行行为中比较数百个机器学习分类器与离散选择模型:一项实证基准研究

Comparing hundreds of machine learning classifiers and discrete choice models in predicting travel behavior: an empirical benchmark

University of Florida(佛罗里达大学);University of Leeds(利兹大学);Massachusetts Institute of Technology(麻省理工学院)

Shenhao Wang, Baichuan Mo, Yunhan Zheng, Stephane Hess, Jinhua Zhao

AI总结:

本研究构建了一个实证基准,通过锦标赛模型分析6,970个实验,发现集成方法和深度学习等ML模型在统计上优于DCM族,并强调情境因素和固有不确定性对模型预测性能的关键影响。

摘要:

大量研究比较了机器学习(ML)与离散选择模型(DCMs)在预测出行需求方面的表现。然而,这些研究通常缺乏普适性,因为它们在比较模型时是确定性的,未考虑情境差异。为了解决这一局限性,本研究通过设计锦标赛模型开发了一个实证基准,从而高效地总结大量实验,量化模型比较中的随机性,并使用正式的统计检验来区分模型效应与情境效应。该基准研究比较了两个大规模数据源:一个通过文献综述编制的数据库,总结了来自35项研究的136个实验;以及我们自己的实验数据,包含来自105个模型和12个模型族的共6,970个实验。该基准研究得出了两个关键发现。首先,许多ML模型,特别是集成方法和深度学习,在统计上优于DCM族(即多项、嵌套和混合logit模型)。然而,本研究也强调了情境因素(即数据源、输入和选择类别)的关键作用,与仅关注模型类型的差异相比,这些因素能更有效地解释模型的预测性能。模型性能随数据源显著变化,随着样本量增大和替代集维度降低而提升。在控制所有模型和情境因素后,仍存在显著的随机性,这表明在此类模型比较中存在固有的不确定性。总体而言,我们建议未来的研究人员将更多注意力从特定情境的模型比较,转向检查模型在不同情境间的可迁移性以及表征ML中的固有不确定性,从而创建更稳健和更具普适性的下一代出行需求模型。

英文摘要:

Numerous studies have compared machine learning (ML) and discrete choice models (DCMs) in predicting travel demand. However, these studies often lack generalizability as they compare models deterministically without considering contextual variations. To address this limitation, our study develops an empirical benchmark by designing a tournament model, thus efficiently summarizing a large number of experiments, quantifying the randomness in model comparisons, and using formal statistical tests to differentiate between the model and contextual effects. This benchmark study compares two large-scale data sources: a database compiled from literature review summarizing 136 experiments from 35 studies, and our own experiment data, encompassing a total of 6,970 experiments from 105 models and 12 model families. This benchmark study yields two key findings. Firstly, many ML models, particularly the ensemble methods and deep learning, statistically outperform the DCM family (i.e., multinomial, nested, and mixed logit models). However, this study also highlights the crucial role of the contextual factors (i.e., data sources, inputs and choice categories), which can explain models' predictive performance more effectively than the differences in model types alone. Model performance varies significantly with data sources, improving with larger sample sizes and lower dimensional alternative sets. After controlling all the model and contextual factors, significant randomness still remains, implying inherent uncertainty in such model comparisons. Overall, we suggest that future researchers shift more focus from context-specific model comparisons towards examining model transferability across contexts and characterizing the inherent uncertainty in ML, thus creating more robust and generalizable next-generation travel demand models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑