arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.10517cs.IR
收藏

搜索引擎结果偏差度量的评价方法

Evaluation Metrics for Measuring Bias in Search Engine Results

Gizem Gezici, Aldo Lipani, Yucel Saygin, Emine Yilmaz

摘要:

搜索引擎决定我们看到一个给定的搜索查询。由于许多人是通过搜索引擎接触信息的,因此可以公平地期望搜索引擎是中立的。然而,搜索引擎结果不一定覆盖搜索查询主题的所有观点,并且它们可能偏向于特定观点,因为搜索引擎结果是基于相关性返回的,相关性是使用许多特征和复杂算法计算的,其中搜索中立性不一定是焦点。因此,重要的是要评估搜索引擎的结果相对于偏见。在这项工作中,我们提出了新的网络搜索偏见的评估措施,考虑到排名和相关性。我们还提出了一个框架来评估网络搜索偏见,使用建议的措施和测试我们的框架上两个流行的搜索引擎的基础上,57个有争议的查询主题,如堕胎,医用大麻,同性恋婚姻。我们衡量立场偏见(支持或反对),以及意识形态偏见(保守或自由)。我们观察到,立场不一定与意识形态倾向相关,例如,对堕胎的积极立场表明自由主义倾向,但对古巴禁运的积极立场表明保守主义倾向。我们的实验表明,没有一个搜索引擎遭受立场偏见。然而,这两个搜索引擎都受到意识形态偏见的影响,都倾向于一种意识形态倾向于另一种,这从我们社会两极分化的角度来看更为重要。

英文摘要:

Search engines decide what we see for a given search query. Since many people are exposed to information through search engines, it is fair to expect that search engines are neutral. However, search engine results do not necessarily cover all the viewpoints of a search query topic, and they can be biased towards a specific view since search engine results are returned based on relevance, which is calculated using many features and sophisticated algorithms where search neutrality is not necessarily the focal point. Therefore, it is important to evaluate the search engine results with respect to bias. In this work we propose novel web search bias evaluation measures which take into account the rank and relevance. We also propose a framework to evaluate web search bias using the proposed measures and test our framework on two popular search engines based on 57 controversial query topics such as abortion, medical marijuana, and gay marriage. We measure the stance bias (in support or against), as well as the ideological bias (conservative or liberal). We observe that the stance does not necessarily correlate with the ideological leaning, e.g. a positive stance on abortion indicates a liberal leaning but a positive stance on Cuba embargo indicates a conservative leaning. Our experiments show that neither of the search engines suffers from stance bias. However, both search engines suffer from ideological bias, both favouring one ideological leaning to the other, which is more significant from the perspective of polarisation in our society.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑