The creation, dissemination, and consumption of disinformation and fabricated content on social media is a growing concern, especially with the ease of access to such sources, and the lack of awareness of the existence of such false information. In this paper, we present an overview of the techniques explored to date for the combating of disinformation with various forms. We introduce different forms of disinformation, discuss factors related to the spread of disinformation, elaborate on the inherent challenges in detecting disinformation, and show some approaches to mitigating disinformation via education, research, and collaboration. Looking ahead, we present some promising future research directions on disinformation.
The COVID-19 epidemic is considered as the global health crisis of the whole society and the greatest challenge mankind faced since World War Two. Unfortunately, the fake news about COVID-19 is spreading as fast as the virus itself. The incorrect health measurements, anxiety, and hate speeches will have bad consequences on people's physical health, as well as their mental health in the whole world. To help better combat the COVID-19 fake news, we propose a new fake news detection dataset MM-COVID(Multilingual and Multidimensional COVID-19 Fake News Data Repository). This dataset provides the m…
Combating Misinformation in the Arab World: Challenges & Opportunities
New York University Abu Dhabi（纽约大学阿布扎赫尔分校）；Qatar Computing Research Institute（卡塔尔计算研究所）；College of Science and Engineering, Hamad Bin Khalifa University（哈马德·本·卡西姆大学科学与工程学院）；EURECOM Biot France（EURECOM法国生物研究中心）
Conventional preventive measures during pandemic include social distancing and lockdown. Such measures in the time of social media brought about a new set of challenges - vulnerability to the toxic impact of online misinformation is high. A case in point is the prevailing COVID-19; as the virus propagates, so does the associated misinformation and fake news about it leading to infodemic. Since the outbreak, there has been a surge of studies investigating various aspects of the pandemic. Of interest to this chapter include studies centring on datasets from online social media platforms where th…
IBM Research - Brazil（IBM研究院（巴西））；IBM Research - Almaden（IBM研究院（阿尔马登））
How can we best address the dangerous impact that deep learning-generated fake audios, photographs, and videos (a.k.a. deepfakes) may have in personal and societal life? We foresee that the availability of cheap deepfake technology will create a second wave of disinformation where people will receive specific, personalized disinformation through different channels, making the current approaches to fight disinformation obsolete. We argue that fake media has to be seen as an upcoming cybersecurity problem, and we have to shift from combating its spread to a prevention and cure framework where us…
Combating fake news needs a variety of defense methods. Although rumor detection and various linguistic analysis techniques are common methods to detect false content in social media, there are other feasible mitigation approaches that could be explored in the machine learning community. In this paper, we present open issues and opportunities in fake news research that need further attention. We first review different stages of the news life cycle in social media and discuss core vulnerability issues for news feed algorithms in propagating fake news content with three examples. We then discuss…
With the rapid increase in access to internet and the subsequent growth in the population of online social media users, the quality of information posted, disseminated and consumed via these platforms is an issue of growing concern. A large fraction of the common public turn to social media platforms and in general the internet for news and even information regarding highly concerning issues such as COVID-19 symptoms. Given that the online information ecosystem is extremely noisy, fraught with misinformation and disinformation, and often contaminated by malicious agents spreading propaganda, i…
What are the pathways for spreading disinformation on social media platforms? This article addresses this question by collecting, categorising, and situating an extensive body of research on how application programming interfaces (APIs) provided by social media platforms facilitate the spread of disinformation. We first examine the landscape of official social media APIs, then perform quantitative research on the open-source code repositories GitHub and GitLab to understand the usage patterns of these APIs. By inspecting the code repositories, we classify developers' usage of the APIs as offic…
In this work we study how pervasive is the presence of disinformation in the Italian debate around immigration on Twitter and the role of automated accounts in the diffusion of such content. By characterising the Twitter users with an \textit{Untrustworthiness} score, that tells us how frequently they engage with disinformation content, we are able to see that such bad information consumption habits are not equally distributed across the users; adopting a network analysis approach, we can identify communities characterised by a very high presence of users that frequently share content from unr…
State-sponsored "bad actors" increasingly weaponize social media platforms to launch cyberattacks and disinformation campaigns during elections. Social media companies, due to their rapid growth and scale, struggle to prevent the weaponization of their platforms. This study conducts an automated spear phishing and disinformation campaign on Twitter ahead of the 2018 United States Midterm Elections. A fake news bot account - the @DCNewsReport - was created and programmed to automatically send customized tweets with a "breaking news" link to 138 Twitter users, before being restricted by Twitter.…
Social media provides political news and information for both active duty military personnel and veterans. We analyze the subgroups of Twitter and Facebook users who spend time consuming junk news from websites that target US military personnel and veterans with conspiracy theories, misinformation, and other forms of junk news about military affairs and national security issues. (1) Over Twitter we find that there are significant and persistent interactions between current and former military personnel and a broad network of extremist, Russia-focused, and international conspiracy subgroups. (2…
Uncertainty-Aware Estimation of Mis/Disinformation Prevalence on Social Media
Ethical Technologies and Connectivity for Humanity Research Centre, Universitat Oberta de Catalunya（加泰罗尼亚开放大学伦理技术与人类连通性研究中心）；Universitat Pompeu Fabra（庞培法布拉大学）；Department of Engineering, Universitat Pompeu Fabra（庞培法布拉大学工程学院）；Science Feedback（科学反馈）
Illinois Institute of Technology（伊利诺伊理工学院）；Weill Cornell Medicine（威尔康奈尔医学院）
Social media has been one of the main information consumption sources for the public, allowing people to seek and spread information more quickly and easily. However, the rise of various social media platforms also enables the proliferation of online misinformation. In particular, misinformation in the health domain has significant impacts on our society such as the COVID-19 infodemic. Therefore, health misinformation in social media has become an emerging research direction that attracts increasing attention from researchers of different disciplines. Compared to misinformation in other domain…
University of Sheffield（谢菲尔德大学）；Austrian Research Institute for Artificial Intelligence（奥地利人工智能研究所）；Panjab University（旁遮普大学）
The explosion of disinformation accompanying the COVID-19 pandemic has overloaded fact-checkers and media worldwide, and brought a new major challenge to government responses worldwide. Not only is disinformation creating confusion about medical science amongst citizens, but it is also amplifying distrust in policy makers and governments. To help tackle this, we developed computational methods to categorise COVID-19 disinformation. The COVID-19 disinformation categories could be used for a) focusing fact-checking efforts on the most damaging kinds of COVID-19 disinformation; b) guiding policy …
Recent accounts from researchers, journalists, as well as federal investigators, reached a unanimous conclusion: social media are systematically exploited to manipulate and alter public opinion. Some disinformation campaigns have been coordinated by means of bots, social media accounts controlled by computer scripts that try to disguise themselves as legitimate human users. In this study, we describe one such operation occurred in the run up to the 2017 French presidential election. We collected a massive Twitter dataset of nearly 17 million posts occurred between April 27 and May 7, 2017 (Ele…
Arizona State University（亚利桑那州立大学）；The Penn State University（宾夕法尼亚州立大学）
In recent years, disinformation including fake news, has became a global phenomenon due to its explosive growth, particularly on social media. The wide spread of disinformation and fake news can cause detrimental societal effects. Despite the recent progress in detecting disinformation and fake news, it is still non-trivial due to its complexity, diversity, multi-modality, and costs of fact-checking or annotation. The goal of this chapter is to pave the way for appreciating the challenges and advancements via: (1) introducing the types of information disorder on social media and examine their …
Divergent Emotional Patterns in Disinformation on Social Media? An Analysis of Tweets and TikToks about the DANA in Valencia
PRHLT Research Center, Universitat Politècnica de València（瓦伦西亚理工大学PRHLT研究中心）；ValgrAI Valencian Graduate School and Research Network of Artificial Intelligence（瓦伦西亚ValgrAI人工智能研究生学院及研究网络）；School of Communication, Universidad de Navarra（纳瓦拉大学传播学院）
本研究针对瓦伦西亚 DANA 洪水事件构建了 650 条多平台数据集，通过情感、语言和音频分析揭示了虚假信息在不同平台上的差异化特征，并证明结合文本与音频特征的多模态检测模型能更有效地识别虚假信息。
Qatar Computing Research Institute（卡塔尔计算研究所）；HBKU（哈马德·本·哈利法大学）；Vellore Institute of Technology（韦洛尔理工学院）
Fighting the ongoing COVID-19 infodemic has been declared as one of the most important focus areas by the World Health Organization since the onset of the COVID-19 pandemic. While the information that is consumed and disseminated consists of promoting fake cures, rumors, and conspiracy theories to spreading xenophobia and panic, at the same time there is information (e.g., containing advice, promoting cure) that can help different stakeholders such as policy-makers. Social media platforms enable the infodemic and there has been an effort to curate the content on such platforms, analyze and deb…
Identifying and characterizing disinformation in political discourse on social media is critical to ensure the integrity of elections and democratic processes around the world. Persistent manipulation of social media has resulted in increased concerns regarding the 2020 U.S. Presidential Election, due to its potential to influence individual opinions and social dynamics. In this work, we focus on the identification of distorted facts, in the form of unreliable and conspiratorial narratives in election-related tweets, to characterize discourse manipulation prior to the election. We apply a dete…
Nowadays, artificial intelligence algorithms are used for targeted and personalized content distribution in the large scale as part of the intense competition for attention in the digital media environment. Unfortunately, targeted information dissemination may result in intellectual isolation and discrimination. Further, as demonstrated in recent political events in the US and EU, malicious bots and social media users can create and propagate targeted `fake news' content in different forms for political gains. From the other direction, fake news detection algorithms attempt to combat such prob…
In the 21st Century information environment, adversarial actors use disinformation to manipulate public opinion. The distribution of false, misleading, or inaccurate information with the intent to deceive is an existential threat to the United States--distortion of information erodes trust in the socio-political institutions that are the fundamental fabric of democracy: legitimate news sources, scientists, experts, and even fellow citizens. As a result, it becomes difficult for society to come together within a shared reality; the common ground needed to function effectively as an economy and …
For more than a decade scholars have been investigating the disinformation flow on social media contextually to societal events, like, e.g., elections. In this paper, we analyze the Twitter traffic related to the US 2020 pre-election debate and ask whether it mirrors the electoral system. The U.S. electoral system provides that, regardless of the actual vote gap, the premier candidate who received more votes in one state `takes' that state. Criticisms of this system have pointed out that election campaigns can be more intense in particular key states to achieve victory, so-called {\it swing st…
Northeastern University（东北大学）；Xi’an Jiaotong University（西安交通大学）；Universidad Nacional Autonoma de Mexico（墨西哥国立自治大学）
The limited information (data voids) on political topics relevant to underrepresented communities has facilitated the spread of disinformation. Independent journalists who combat disinformation in underrepresented communities have reported feeling overwhelmed because they lack the tools necessary to make sense of the information they monitor and address the data voids. In this paper, we present a system to identify and address political data voids within underrepresented communities. Armed with an interview study, indicating that the independent news media has the potential to address them, we…
The COVID-19 pandemic has been the subject of a vast amount of misinformation, particularly in digital information environments, and major social media platforms recently publicized some of the countermeasures they are adopting. This presents an opportunity to examine the nature of the misinformation and disinformation being produced, and the theoretical and technological paradigm used to counter it. I argue that this approach is based on a conception of misinformation as epistemic pollution that can only justify a limited and potentially inadequate response , and that some of the measures und…
University of Southern California（南加利福尼亚大学）；Peking University（北京大学）
The proliferation of fake news on social media has opened up new directions of research for timely identification and containment of fake news, and mitigation of its widespread impact on public opinion. While much of the earlier research was focused on identification of fake news based on its contents or by exploiting users' engagements with the news on social media, there has been a rising interest in proactive intervention strategies to counter the spread of misinformation and its impact on society. In this survey, we describe the modern-day problem of fake news and, in particular, highlight…
The daily exposure of social media users to propaganda and disinformation campaigns has reinvigorated the need to investigate the local and global patterns of diffusion of different (mis)information content on social media. Echo chambers and influencers are often deemed responsible of both the polarization of users in online social networks and the success of propaganda and disinformation campaigns. This article adopts a data-driven approach to investigate the structuration of communities and propaganda networks on Twitter in order to assess the correctness of these imputations. In particular,…
A rapidly evolving situation such as the COVID-19 pandemic is a significant challenge for AI/ML models because of its unpredictability. %The most reliable indicator of the pandemic spreading has been the number of test positive cases. However, the tests are both incomplete (due to untested asymptomatic cases) and late (due the lag from the initial contact event, worsening symptoms, and test results). Social media can complement physical test data due to faster and higher coverage, but they present a different challenge: significant amounts of noise, misinformation and disinformation. We believ…
Over the past couple of years, anecdotal evidence has emerged linking coordinated campaigns by state-sponsored actors with efforts to manipulate public opinion on the Web, often around major political events, through dedicated accounts, or "trolls." Although they are often involved in spreading disinformation on social media, there is little understanding of how these trolls operate, what type of content they disseminate, and most importantly their influence on the information ecosystem. In this paper, we shed light on these questions by analyzing 27K tweets posted by 1K Twitter users identifi…
MIT Lincoln Laboratory（麻省理工学院林肯实验室）；Fox School of Business, Temple University（天普大学福克斯商学院）；Yau Mathematical Sciences Center, Tsinghua University（清华大学丘成桐数学科学中心）；Department of Statistics, Harvard University（哈佛大学统计系）
The weaponization of digital communications and social media to conduct disinformation campaigns at immense scale, speed, and reach presents new challenges to identify and counter hostile influence operations (IOs). This paper presents an end-to-end framework to automate detection of disinformation narratives, networks, and influential actors. The framework integrates natural language processing, machine learning, graph analytics, and a novel network causal inference approach to quantify the impact of individual actors in spreading IO narratives. We demonstrate its capability on real-world hos…
With the increase in the time spent at home, social media platforms' role has become an integral part of the public discussion in the COVID-19 period. Individuals use social media platforms to express their emotions, interact, and engage in public debate. Therefore, it is essential to analyze social media platforms for those wanting to understand public opinion during the pandemic. This thesis is the first study that examines the Turkish Twitter-sphere to understand the change in public opinion during the COVID-19 outbreak. For that purpose, starting from 12 February 2020 (one month before the…
Due to their widespread adoption, social media platforms present an ideal environment for studying and understanding social behavior, especially on information spread. Modeling social media activity has numerous practical implications such as supporting efforts to analyze strategic information operations, designing intervention techniques to mitigate disinformation, or delivering critical information during disaster relief operations. In this paper we propose a modeling technique that forecasts topic-specific daily volume of social media activities by using both exogenous signals, such as news…
Pathogenic Social Media (PSM) accounts such as terrorist supporter accounts and fake news writers have the capability of spreading disinformation to viral proportions. Early detection of PSM accounts is crucial as they are likely to be key users to make malicious information "viral". In this paper, we adopt the causal inference framework along with graph-based metrics in order to distinguish PSMs from normal users within a short time of their activities. We propose both supervised and semi-supervised approaches without taking the network information and content into account. Results on a real-…
Misinformation/disinformation about COVID-19 has been rampant on social media around the world. In this study, we investigate COVID-19 misinformation/ disinformation on social media in multiple languages - Farsi (Persian), Chinese, and English, about multiple countries - Iran, China, and the United States (US), and on multiple platforms such as Twitter, Facebook, Instagram, Weibo, and WhatsApp. Misinformation, especially about a global pandemic, is a global problem yet it is common for studies of COVID-19 misinformation on social media to focus on a single language, like English, a single coun…
As our reliance on social media platforms and web services increase day by day, exploiters view these platforms as an opportunity to manipulate our thoughts ad actions. These platforms have become an open playground for social bot accounts. Social bots not only learn human conversations, manners, and presence but also manipulate public opinion, act as scammers, manipulate stock markets, and so on. There has been evidence of bots manipulating people's opinions and thoughts which can be a great threat to democracy. Identification and prevention of such campaigns that release or create these bots…
Online social media platforms are turning into the prime source of news and narratives about worldwide events. However,a systematic summarization-based narrative extraction that can facilitate communicating the main underlying events is lacking. To address this issue, we propose a novel event-based narrative summary extraction framework. Our proposed framework is designed as a probabilistic topic model, with categorical time distribution, followed by extractive text summarization. Our topic model identifies topics' recurrence over time with a varying time resolution. This framework not only ca…
Coordinated online behaviors are an essential part of information and influence operations, as they allow a more effective disinformation's spread. Most studies on coordinated behaviors involved manual investigations, and the few existing computational approaches make bold assumptions or oversimplify the problem to make it tractable. Here, we propose a new network-based framework for uncovering and studying coordinated behaviors on social media. Our research extends existing systems and goes beyond limiting binary classifications of coordinated and uncoordinated behaviors. It allows to expose …
On 26 January 2021, India witnessed a national embarrassment from the demographic least expected from - farmers. People across the nation watched in horror as a pseudo-patriotic mob of farmers stormed capital Delhi and vandalized the national pride- Red Fort. Investigations that followed the event revealed the existence of a social media trail that led to the likes of such an event. Consequently, it became essential and necessary to archive this trail for social media analysis - not only to understand the bread-crumbs that are dispersed across the trail but also to visualize the role played by…
Over the past years, political events and public opinion on the Web have been allegedly manipulated by accounts dedicated to spreading disinformation and performing malicious activities on social media. These accounts hereafter referred to as "Pathogenic Social Media (PSM)" accounts, are often controlled by terrorist supporters, water armies or fake news writers and hence can pose threats to social media and general public. Understanding and analyzing PSMs could help social media firms devise sophisticated and automated techniques that could be deployed to stop them from reaching their audienc…
Since the proliferation of social media usage, hate speech has become a major crisis. Hateful content can spread quickly and create an environment of distress and hostility. Further, what can be considered hateful is contextual and varies with time. While online hate speech reduces the ability of already marginalised groups to participate in discussion freely, offline hate speech leads to hate crimes and violence against individuals and communities. The multifaceted nature of hate speech and its real-world impact have already piqued the interest of the data mining and machine learning communit…
Participation on social media platforms has many benefits but also poses substantial threats. Users often face an unintended loss of privacy, are bombarded with mis-/disinformation, or are trapped in filter bubbles due to over-personalized content. These threats are further exacerbated by the rise of hidden AI-driven algorithms working behind the scenes to shape users' thoughts, attitudes, and behavior. We investigate how multimedia researchers can help tackle these problems to level the playing field for social media users. We perform a comprehensive survey of algorithmic threats on social me…
This paper evaluates Parler, the controversial social media platform, from two seemingly orthogonal perspectives: UX design perspective and data science. UX design researchers explore how users react to the interface/content of their social media feeds; Data science researchers analyze the misinformation flow in these feeds to detect alternative narratives and state-sponsored disinformation campaigns. We took a critical look into the intersection of these approaches to understand how Parler's interface itself is conductive to the flow of misinformation and the perception of "free speech" among…
