arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2008.09727cs.LGstat.ML
收藏

基于季节调整的大规模搜索引擎特征选择方法

Seasonal-adjustment Based Feature Selection Method for Large-scale Search Engine Logs

University of Tsukuba(筑波大学);Riken AIP(理化学研究所革新智能综合研究中心)

Thien Q. Tran, Jun Sakuma

摘要:

搜索引擎日志在跟踪和预测传染病爆发方面具有很大的潜力。更准确地说,人们可以使用一些搜索词的搜索量来近乎实时地预测传染病的感染率。然而,由于搜索日志的以下双向不稳定特性,使用搜索引擎日志进行准确和稳定的疫情预测是一项具有挑战性的任务。首先,搜索词的搜索量可能在短期内不规则地变化,例如,由于诸如媒体或新闻的量的环境因素。其次,搜索量也可能由于搜索引擎的人口统计变化而长期变化。也就是说,如果使用这种搜索日志来训练模型,而忽略这种特征,那么当这些变化发生时,所得到的预测将包含严重的错误预测。在这项工作中,我们提出了一种新的特征选择方法,以克服这种不稳定的问题。特别是,我们采用了一种季节调整方法,将每个时间序列分解为三个组成部分:季节性,趋势和不规则成分,并分别为每个组成部分建立预测模型。我们还精心设计了一个特征选择方法来选择合适的搜索词来预测每个组件。我们对十种不同的传染病进行了综合实验。实验结果表明,所提出的方法优于所有比较方法的预测精度为10种疾病中的7种,在即时预报和预测设置。此外,所提出的方法是更成功地选择语义相关的目标疾病的搜索条件。

英文摘要:

Search engine logs have a great potential in tracking and predicting outbreaks of infectious disease. More precisely, one can use the search volume of some search terms to predict the infection rate of an infectious disease in nearly real-time. However, conducting accurate and stable prediction of outbreaks using search engine logs is a challenging task due to the following two-way instability characteristics of the search logs. First, the search volume of a search term may change irregularly in the short-term, for example, due to environmental factors such as the amount of media or news. Second, the search volume may also change in the long-term due to the demographic change of the search engine. That is to say, if a model is trained with such search logs with ignoring such characteristic, the resulting prediction would contain serious mispredictions when these changes occur. In this work, we proposed a novel feature selection method to overcome this instability problem. In particular, we employ a seasonal-adjustment method that decomposes each time series into three components: seasonal, trend and irregular component and build prediction models for each component individually. We also carefully design a feature selection method to select proper search terms to predict each component. We conducted comprehensive experiments on ten different kinds of infectious diseases. The experimental results show that the proposed method outperforms all comparative methods in prediction accuracy for seven of ten diseases, in both now-casting and forecasting setting. Also, the proposed method is more successful in selecting search terms that are semantically related to target diseases.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑