A multi-layer approach to disinformation detection on Twitter
Politecnico di Milano（米兰理工大学）
Francesco Pierri, Carlo Piccardi, Stefano Ceri
备注：
A revised version of this pre-print has been published on EPJ Data Science with the title "A multi-layer approach to disinformation detection in US and Italian news spreading on Twitter"
发表信息：
Published version on EPJ Data Science ("A multi-layer approach to disinformation detection in US and Italian news spreading on Twitter") Dec 2020
We tackle the problem of classifying news articles pertaining to disinformation vs mainstream news by solely inspecting their diffusion mechanisms on Twitter. Our technique is inherently simple compared to existing text-based approaches, as it allows to by-pass the multiple levels of complexity which are found in news content (e.g. grammar, syntax, style). We employ a multi-layer representation of Twitter diffusion networks, and we compute for each layer a set of global network features which quantify different aspects of the sharing process. Experimental results with two large-scale datasets, corresponding to diffusion cascades of news shared respectively in the United States and Italy, show that a simple Logistic Regression model is able to classify disinformation vs mainstream networks with high accuracy (AUROC up to 94%), also when considering the political bias of different sources in the classification task. We also highlight differences in the sharing patterns of the two news domains which appear to be country-independent. We believe that our network-based approach provides useful insights which pave the way to the future development of a system to detect misleading and harmful information spreading on social media.
National Intelligence University（国家情报大学）；Oettinger School of Science and Technology Intelligence（奥廷格科技情报学院）
Efforts by foreign actors to influence public opinion have gained considerable attention because of their potential to impact democratic elections. Thus, the ability to identify and counter sources of disinformation is increasingly becoming a top priority for government entities in order to protect the integrity of democratic processes. This study presents a method of identifying Russian disinformation bots on Twitter using centering resonance analysis and Clauset-Newman-Moore community detection. The data reflect a significant degree of discursive dissimilarity between known Russian disinform…
We investigate the presence (and the influence) of disinformation spreading on online social networks in Italy, in the5-month period preceding the 2019 European Parliament elections. To this aim we collected a large-scale dataset oftweets associated to thousands of news articles published on Italian disinformation websites. In the observation period,a few outlets accounted for most of the deceptive information circulating on Twitter, which focused on controversialand polarizing topics of debate such as immigration, national safety and (Italian) nationalism. We found evidence ofconnections betw…
State-sponsored "bad actors" increasingly weaponize social media platforms to launch cyberattacks and disinformation campaigns during elections. Social media companies, due to their rapid growth and scale, struggle to prevent the weaponization of their platforms. This study conducts an automated spear phishing and disinformation campaign on Twitter ahead of the 2018 United States Midterm Elections. A fake news bot account - the @DCNewsReport - was created and programmed to automatically send customized tweets with a "breaking news" link to 138 Twitter users, before being restricted by Twitter.…
We released over 1 million tweets shared during 2019 and containing links to thousands of news articles published on two classes of Italian outlets: (1) disinformation websites, i.e. outlets which have been repeatedly flagged by journalists and fact-checkers for producing low-credibility content such as false news, hoaxes, click-bait, misleading and hyper-partisan stories; (2) fact-checking websites which notably debunk and verify online news and claims. The dataset, which includes also title and body for approximately 37k news articles, is publicly available at https://doi.org/10.7910/DVN/ PG…
As the novel coronavirus spread globally, a growing public panic was expressed over the internet. We examine the public discussion concerning COVID-19 on Twitter. We use a dataset of 67 million tweets from 12 million users collected between January 29, 2020 and March 4, 2020. We categorize users based on their home countries, social identities, and political orientation. We find that news media, government officials, and individual news reporters posted a majority of influential tweets, while the most influential ones are still written by regular users. Tweets mentioning "fake news" URLs and d…
For more than a decade scholars have been investigating the disinformation flow on social media contextually to societal events, like, e.g., elections. In this paper, we analyze the Twitter traffic related to the US 2020 pre-election debate and ask whether it mirrors the electoral system. The U.S. electoral system provides that, regardless of the actual vote gap, the premier candidate who received more votes in one state `takes' that state. Criticisms of this system have pointed out that election campaigns can be more intense in particular key states to achieve victory, so-called {\it swing st…
Both politics and pandemics have recently provided ample motivation for the development of machine learning-enabled disinformation (a.k.a. fake news) detection algorithms. Existing literature has focused primarily on the fully-automated case, but the resulting techniques cannot reliably detect disinformation on the varied topics, sources, and time scales required for military applications. By leveraging an already-available analyst as a human-in-the-loop, however, the canonical machine learning techniques of sentiment analysis, aspect-based sentiment analysis, and stance detection become plaus…
Over the past couple of years, anecdotal evidence has emerged linking coordinated campaigns by state-sponsored actors with efforts to manipulate public opinion on the Web, often around major political events, through dedicated accounts, or "trolls." Although they are often involved in spreading disinformation on social media, there is little understanding of how these trolls operate, what type of content they disseminate, and most importantly their influence on the information ecosystem. In this paper, we shed light on these questions by analyzing 27K tweets posted by 1K Twitter users identifi…
Online manipulation of information has become more prevalent in recent years as state-sponsored disinformation campaigns seek to influence and polarize political topics through massive coordinated efforts. In the process, these efforts leave behind artifacts, which researchers have leveraged to analyze the tactics employed by disinformation campaigns after they are taken down. Coordination network analysis has proven helpful for learning about how disinformation campaigns operate; however, the usefulness of these forensic tools as a detection mechanism is still an open question. In this paper,…
In this study, we characterize the cross-platform mobilization of YouTube and BitChute videos on Twitter during the 2020 U.S. Election fraud discussions. Specifically, we extend the VoterFraud2020 dataset to describe the prevalence of content supplied by both platforms, the mobilizers of that content, the suppliers of that content, and the content itself. We find that while BitChute videos promoting election fraud claims were linked to and engaged with in the Twitter discussion, they played a relatively small role compared to YouTube videos promoting fraud claims. This core finding points to t…
The daily exposure of social media users to propaganda and disinformation campaigns has reinvigorated the need to investigate the local and global patterns of diffusion of different (mis)information content on social media. Echo chambers and influencers are often deemed responsible of both the polarization of users in online social networks and the success of propaganda and disinformation campaigns. This article adopts a data-driven approach to investigate the structuration of communities and propaganda networks on Twitter in order to assess the correctness of these imputations. In particular,…
In this work we study how pervasive is the presence of disinformation in the Italian debate around immigration on Twitter and the role of automated accounts in the diffusion of such content. By characterising the Twitter users with an \textit{Untrustworthiness} score, that tells us how frequently they engage with disinformation content, we are able to see that such bad information consumption habits are not equally distributed across the users; adopting a network analysis approach, we can identify communities characterised by a very high presence of users that frequently share content from unr…
MIT Lincoln Laboratory（麻省理工学院林肯实验室）；Fox School of Business, Temple University（天普大学福克斯商学院）；Yau Mathematical Sciences Center, Tsinghua University（清华大学丘成桐数学科学中心）；Department of Statistics, Harvard University（哈佛大学统计系）
The weaponization of digital communications and social media to conduct disinformation campaigns at immense scale, speed, and reach presents new challenges to identify and counter hostile influence operations (IOs). This paper presents an end-to-end framework to automate detection of disinformation narratives, networks, and influential actors. The framework integrates natural language processing, machine learning, graph analytics, and a novel network causal inference approach to quantify the impact of individual actors in spreading IO narratives. We demonstrate its capability on real-world hos…
Social media provides political news and information for both active duty military personnel and veterans. We analyze the subgroups of Twitter and Facebook users who spend time consuming junk news from websites that target US military personnel and veterans with conspiracy theories, misinformation, and other forms of junk news about military affairs and national security issues. (1) Over Twitter we find that there are significant and persistent interactions between current and former military personnel and a broad network of extremist, Russia-focused, and international conspiracy subgroups. (2…
The advent of social media changed the way we consume content favoring a disintermediated access and production. This scenario has been matter of critical discussion about its impact on society. Magnified in the case of Arab Spring or heavily criticized in the Brexit and 2016 U.S. elections. In this work we explore information consumption on Twitter during the last European electoral campaign by analyzing the interaction patterns of official news sources, fake news sources, politicians, people from the showbiz and many others. We extensively explore interactions among different classes of acco…
McGill University（麦吉尔大学）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）
As social media becomes increasingly prominent in our day to day lives, it is increasingly important to detect informative content and prevent the spread of disinformation and unverified rumours. While many sophisticated and successful models have been proposed in the literature, they are often compared with older NLP baselines such as SVMs, CNNs, and LSTMs. In this paper, we examine the performance of a broad set of modern transformer-based language models and show that with basic fine-tuning, these models are competitive with and can even significantly outperform recently proposed state-of-t…
As recent events have demonstrated, disinformation spread through social networks can have dire political, economic and social consequences. Detecting disinformation must inevitably rely on the structure of the network, on users particularities and on event occurrence patterns. We present a graph data structure, which we denote as a meta-graph, that combines underlying users' relational event information, as well as semantic and topical modeling. We detail the construction of an example meta-graph using Twitter data covering the 2016 US election campaign and then compare the detection of disin…
Fake news are affecting a large proportion of the population even becoming a danger to the society. Mostly, this disinformation flow take place through Internet. Being aware of that problem, in this work we propose a synthetic indicator that measures the user reputation in Twitter in order to analyze the credibility of the content in this social network. In order to show the indicator utility, we have analyzed data from some political topics in Spain from 2019 to 2020 and we have checked that bots plays a decisive role into the spread of news and, as might be expected, the link among popularit…
In this paper, a BERT based neural network model is applied to the JIGSAW data set in order to create a model identifying hateful and toxic comments (strictly seperated from offensive language) in online social platforms (English language), in this case Twitter. Three other neural network architectures and a GPT-2 model are also applied on the provided data set in order to compare these different models. The trained BERT model is then applied on two different data sets to evaluate its generalisation power, namely on another Twitter data set and the data set HASOC 2019 which includes Twitter an…
Sofia University（索非亚大学）；TIET；QCRI, HBKU（卡塔尔计算研究所，哈马德·本·哈利法大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
We study the problem of profiling news media on the Web with respect to their factuality of reporting and bias. This is an important but under-studied problem related to disinformation and "fake news" detection, but it addresses the issue at a coarser granularity compared to looking at an individual article or an individual claim. This is useful as it allows to profile entire media outlets in advance. Unlike previous work, which has focused primarily on text (e.g.,~on the text of the articles published by the target website, or on the textual description in their social media profiles or in Wi…
This paper presents TrollHunter, an automated reasoning mechanism we used to hunt for trolls on Twitter during the COVID-19 pandemic in 2020. Trolls, poised to disrupt the online discourse and spread disinformation, quickly seized the absence of a credible response to COVID-19 and created a COVID-19 infodemic by promulgating dubious content on Twitter. To counter the COVID-19 infodemic, the TrollHunter leverages a unique linguistic analysis of a multi-dimensional set of Twitter content features to detect whether or not a tweet was meant to troll. TrollHunter achieved 98.5% accuracy, 75.4% prec…
As people rely on social media as their primary sources of news, the spread of misinformation has become a significant concern. In this large-scale study of news in social media we analyze eleven million posts and investigate propagation behavior of users that directly interact with news accounts identified as spreading trusted versus malicious content. Unlike previous work, which looks at specific rumors, topics, or events, we consider all content propagated by various news sources. Moreover, we analyze and contrast population versus sub-population behaviour (by demographics) when spreading m…
Accurate estimation of user location is important for many online services. Previous neural network based methods largely ignore the hierarchical structure among locations. In this paper, we propose a hierarchical location prediction neural network for Twitter user geolocation. Our model first predicts the home country for a user, then uses the country result to guide the city-level prediction. In addition, we employ a character-aware word embedding layer to overcome the noisy information in tweets. With the feature fusion layer, our model can accommodate various feature combinations and achie…
Internet messengers and social networks have become an integral part of modern digital life. We have in mind not only the interaction between individual users but also a variety of applications that exist in these applications. Typically, applications for social networks use the universal login system and rely on data from social networks. Also, such applications are likely to get more traction when they are inside of the big social network like Facebook. At the same time, less attention is paid to communication capabilities of social networks. In this paper, we target Twitter as a messaging s…
National and Kapodistrian University of Athens（雅典国立卡波迪斯特里安大学）；NCSR Demokritos（德谟克利特国家科学研究中心）
The pervasiveness of the Internet and social media have enabled the rapid and anonymous spread of Hate Speech content on microblogging platforms such as Twitter. Current EU and US legislation against hateful language, in conjunction with the large amount of data produced in these platforms has led to automatic tools being a necessary component of the Hate Speech detection task and pipeline. In this study, we examine the performance of several, diverse text representation techniques paired with multiple classification algorithms, on the automatic Hate Speech detection and abusive language discr…
News media has long been an ecosystem of creation, reproduction, and critique, where news outlets report on current events and add commentary to ongoing stories. Understanding the dynamics of news information creation and dispersion is important to accurately ascribe credit to influential work and understand how societal narratives develop. These dynamics can be modelled through a combination of information-theoretic natural language processing and networks; and can be parameterised using large quantities of textual data. However, it is challenging to see "the wood for the trees", i.e., to det…
Social media provides a rich source of networked data. This data is represented by a set of nodes and a set of relations (edges). It is often possible to obtain or infer multiple types of relations from the same set of nodes, such as observed friend connections, inferred links via semantic comparison, or relations based off of geographic proximity. These edge sets can be represented by one multi-layer network. In this paper we review a method to perform community detection of multilayer networks, and illustrate its use as a visualization tool for analyzing different community partitions. The a…
Communicating through social platforms has become one of the principal means of personal communications and interactions. Unfortunately, healthy communication is often interfered by offensive language that can have damaging effects on the users. A key to fight offensive language on social media is the existence of an automatic offensive language detection system. This paper presents the results and the main findings of SemEval-2020, Task 12 OffensEval Sub-task A Zampieri et al. (2020), on Identifying and categorising Offensive Language in Social Media. The task was based on the Arabic OffensEv…
The openness feature of Twitter allows programs to generate and control Twitter accounts automatically via the Twitter API. These accounts, which are known as bots, can automatically perform actions such as tweeting, re-tweeting, following, unfollowing, or direct messaging other accounts, just like real people. They can also conduct malicious tasks such as spreading of fake news, spams, malicious software and other cyber-crimes. In this paper, we introduce a novel bot detection approach using deep learning, with the Multi-layer Perceptron Neural Networks and nine features of a bot account. A w…
We use data on frequencies of bi-directional posts to define edges (or relationships) in two Facebook datasets and a Twitter dataset and use these to create ego-centric social networks. We explore the internal structure of these networks to determine whether they have the same kind of layered structure as has been found in offline face-to-face networks (which have a distinctively scaled structure with successively inclusive layers at 5, 15, 50 and 150 alters). The two Facebook datasets are best described by a four-layer structure and the Twitter dataset by a five-layer structure. The absolute …
In the last few years, Online Social Networks (OSNs) attracted the interest of a large number of researchers, thanks to their central role in the society. Through the analysis of OSNs, many social phenomena have been studied, such as the viral diffusion of information amongst people. What is still unclear is the relation between micro-level structural properties of OSNs (i.e. the properties of the personal networks of the users, also known as ego networks) and the emergence of such phenomena. A better knowledge of this relation could be essential for the creation of services for the Future Int…
Relationships form the core of connected learning. In this study, we apply and extend social network analysis methods to uncover the layered network structure of relationships among Fanfiction.net authors and reviewers. Fanfiction.net, one of the world's largest fanfiction communities, is a space where millions of young people engage with written media, connect over shared interests, and receive support and mentoring from a distributed audience. Does an affinity space such as Fanfiction.net have the same structure as social networks Facebook and Twitter? We applied k-means clustering on millio…
Ego networks have proved to be a valuable tool for understanding the relationships that individuals establish with their peers, both in offline and online social networks. Particularly interesting are the cognitive constraints associated with the interactions between the ego and the members of their ego network, which limit individuals to maintain meaningful interactions with no more than 150 people, on average, and to arrange such relationships along concentric circles of decreasing engagement. In this work, we focus on the ego networks of journalists on Twitter, considering 17 different coun…
The Covid-19 pandemic caused disruptive effects for individuals, firms, and societies. In this paper, we offer insights on the major issues and challenges firms are facing in the Covid-19 pandemic, as well as their concerns for Corporate Social Responsibility (CSR) themes. To do so, we investigate large Italian firms' discussion on Twitter in the first nine months of the pandemic. We downloaded all Twitter posts from 1st of March, 2020, to 17th of November, 2020 by the accounts of the largest Italian firms, i.e. those with 250 or more employees. We then built the bipartite network of accounts …
Social networks often encode community structure using multiple distinct types of links between nodes. In this paper we introduce a novel method to extract information from such multi-layer networks, where each type of link forms its own layer. Using the concept of Pareto optimality, community detection in this multi-layer setting is formulated as a multiple criterion optimization problem. We propose an algorithm for finding an approximate Pareto frontier containing a family of solutions. The power of this approach is demonstrated on a Twitter dataset, where the nodes are hashtags and the laye…
State Key Lab of Software Development Environment, Beihang University（北京航空航天大学软件开发环境国家重点实验室）；School of Statistics, Capital University of Economics and Business（首都经济贸易大学统计学院）
Given the development and abundance of social media, studying the stance of social media users is a challenging and pressing issue. Social media users express their stance by posting tweets and retweeting. Therefore, the homogeneous relationship between users and the heterogeneous relationship between users and tweets are relevant for the stance detection task. Recently, graph neural networks (GNNs) have developed rapidly and have been applied to social media research. In this paper, we crawl a large-scale dataset of the 2020 US presidential election and automatically label all users by manual…
According to the Eurobarometer report about EU media use of May 2018, the number of European citizens who consult on-line social networks for accessing information is considerably increasing. In this work we analyze approximately $10^6$ tweets exchanged during the last Italian elections. By using an entropy-based null model discounting the activity of the users, we first identify potential political alliances within the group of verified accounts: if two verified users are retweeted more than expected by the non-verified ones, they are likely to be related. Then, we derive the users' affiliati…
University of Turin（都灵大学）；University of Exeter（埃克塞特大学）
Since their announcement in November 2020, COVID-19 vaccines were largely debated by the press and social media. With most studies focusing on COVID-19 disinformation in social media, little attention has been paid to how mainstream news outlets framed COVID-19 narratives compared to alternative sources. To fill this gap, we use cognitive network science and natural language processing to reconstruct time-evolving semantic and emotional frames of 5745 Italian news, that were massively re-shared on Facebook and Twitter, about COVID-19 vaccines. We found consistently high levels of trust/anticip…
Anticipating audience reaction towards a certain piece of text is integral to several facets of society ranging from politics, research, and commercial industries. Sentiment analysis (SA) is a useful natural language processing (NLP) technique that utilizes both lexical/statistical and deep learning methods to determine whether different sized texts exhibit a positive, negative, or neutral emotion. However, there is currently a lack of tools that can be used to analyse groups of independent texts and extract the primary emotion from the whole set. Therefore, the current paper proposes a novel …
Social media has been rapidly developing in the public sphere due to its ease of spreading new information, which leads to the circulation of rumors. However, detecting rumors from such a massive amount of information is becoming an increasingly arduous challenge. Previous work generally obtained valuable features from propagation information. It should be noted that most methods only target the propagation structure while ignoring the rumor transmission pattern. This limited focus severely restricts the collection of spread data. To solve this problem, the authors of the present study are mot…
In today's world, individuals interact with each other in more complicated patterns than ever. Some individuals engage through online social networks (e.g., Facebook, Twitter), while some communicate only through conventional ways (e.g., face-to-face). Therefore, understanding the dynamics of information propagation among humans calls for a multi-layer network model where an online social network is conjoined with a physical network. In this work, we initiate a study of information diffusion in a clustered multi-layer network model, where all constituent layers are random networks with high cl…
The cognitive constraints that humans exhibit in their social interactions have been extensively studied by anthropologists, who have highlighted their regularities across different types of social networks. We postulate that similar regularities can be found in other cognitive processes, such as those involving language production. In order to provide preliminary evidence for this claim, we analyse a dataset containing tweets of a heterogeneous group of Twitter users (regular users and professional writers). Leveraging a methodology similar to the one used to uncover the well-established soci…
The study of complex networks that account for different types of interactions has become a subject of interest in the last few years, specially because its representational power in the description of users interactions in diverse online social platforms (Facebook, Twitter, Instagram, etc.). The mathematical description of these interacting networks has been coined under the name of multilayer networks, where each layer accounts for a type of interaction. It has been shown that diffusive processes on top of these networks present a phenomenology that cannot be explained by the naive superposi…
Nowadays, millions of people interact on a daily basis on online social media like Facebook and Twitter, where they share and discuss information about a wide variety of topics. In this paper, we focus on a specific online social network, Twitter, and we analyze multiple datasets each one consisting of individuals' online activity before, during and after an exceptional event in terms of volume of the communications registered. We consider important events that occurred in different arenas that range from policy to culture or science. For each dataset, the users' online activities are modeled …
Recent availability of geo-localized data capturing individual human activity together with the statistical data on international migration opened up unprecedented opportunities for a study on global mobility. In this paper we consider it from the perspective of a multi-layer complex network, built using a combination of three datasets: Twitter, Flickr and official migration data. Those datasets provide different but equally important insights on the global mobility: while the first two highlight short-term visits of people from one country to another, the last one - migration - shows the long…
One of the reasons behind the success of Social Network Analysis is its simple and general graph model made of nodes (representing individuals) and ties. However, when we focus on our daily on-line experience we must confront a more complex scenario: people inhabitate several on-line spaces interacting to several communities active on various technological infrastructures like Twitter, Facebook, YouTube or FourSquare and with distinct social objectives. This constitutes a complex network of interconnected networks where users' identities are spread and where information propagates navigating t…
Being able to recommend links between users in online social networks is important for users to connect with like-minded individuals as well as for the platforms themselves and third parties leveraging social media information to grow their business. Predictions are typically based on unsupervised or supervised learning, often leveraging simple yet effective graph topological information, such as the number of common neighbors. However, we argue that richer information about personal social structure of individuals might lead to better predictions. In this paper, we propose to leverage well-es…
