arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1401.7909cs.SIphysics.soc-ph
收藏

什么时候有偏见?评估Twitter流媒体API的代表性

When is it Biased? Assessing the Representativeness of Twitter's Streaming API

Fred Morstatter, Jürgen Pfeffer, Huan Liu

摘要:

Twitter吸引了科学界的兴趣,不仅因为它庞大的用户群和内容,还因为它在分享数据方面的开放性。Twitter通过“Streaming API”共享其推文的免费1%样本,该服务根据研究人员设置的一组参数返回推文样本。最近,有研究指出,通过Streaming API返回的数据存在偏见,这引起了人们对研究场景中使用的数据服务完整性的担忧。虽然这些结果很重要,但在以前的工作中提出的方法依赖于限制性和昂贵的Firehose来找到流式API数据中的偏差。在这项工作中,我们解决的问题,找到样本偏差,而不需要“黄金标准”的Firehose数据。也就是说,我们专注于在Streaming API数据中找到标签趋势与Twitter上真实活动趋势显著不同的时间段。我们提出了一个解决方案,重点是使用一个开放的数据源,以找到流API的偏见。最后,我们评估的数据源在稀疏数据的情况下,并为用户发出相同的查询从不同地区的效用。

英文摘要:

Twitter has captured the interest of the scientific community not only for its massive user base and content, but also for its openness in sharing its data. Twitter shares a free 1% sample of its tweets through the "Streaming API", a service that returns a sample of tweets according to a set of parameters set by the researcher. Recently, research has pointed to evidence of bias in the data returned through the Streaming API, raising concern in the integrity of this data service for use in research scenarios. While these results are important, the methodologies proposed in previous work rely on the restrictive and expensive Firehose to find the bias in the Streaming API data. In this work we tackle the problem of finding sample bias without the need for "gold standard" Firehose data. Namely, we focus on finding time periods in the Streaming API data where the trend of a hashtag is significantly different from its trend in the true activity on Twitter. We propose a solution that focuses on using an open data source to find bias in the Streaming API. Finally, we assess the utility of the data source in sparse data situations and for users issuing the same query from different regions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑