预测技术的评估:人类不确定性的影响
Assessment of Prediction Techniques: The Impact of Human Uncertainty
摘要:
许多数据采掘方法的目标是建模和预测人类行为。一个重要的关注点是基于模型的预测的质量,例如寻找具有最佳预测性能的竞赛获胜者。在现实生活中,人们在做出决定时往往会有相当大的不确定性。它的评估和由此产生的影响,统计上明显的预测模型的评价是在这方面的贡献的主要焦点。我们确定相关的不确定性来源,以及其准确测量的能力有限,提出了一个不确定性意识更明显的评价数据采掘方法的方法,并讨论其影响现有的质量评估策略。具体来说,我们的方法从公共点范式切换到更合适的分布范式。这在推荐系统及其建立的预测质量度量的上下文中举例说明。讨论证实了全面的实验与真实用户,大规模的模拟,并讨论事先评估活动(即。Netflix奖)在人类的不确定性方面。
英文摘要:
Many data mining approaches aim at modelling and predicting human behaviour. An important quantity of interest is the quality of model-based predictions, e.g. for finding a competition winner with best prediction performance. In real life, human beings meet their decisions with considerable uncertainty. Its assessment and resulting implications for statistically evident evaluation of predictive models are in the main focus of this contribution. We identify relevant sources of uncertainty as well as the limited ability of its accurate measurement, propose an uncertainty-aware methodology for more evident evaluations of data mining approaches, and discuss its implications for existing quality assessment strategies. Specifically, our approach switches from common point-paradigm to more appropriate distribution-paradigm. This is exemplified in the context of recommender systems and their established metrics of prediction quality. The discussion is substantiated by comprehensive experiments with real users, large-scale simulations, and discussion of prior evaluation campaigns (i.a. Netflix Prize) in the light of human uncertainty aspects.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。