The subreddit r/The_Donald was repeatedly denounced as a toxic and misbehaving online community, reasons for which it faced a sequence of increasingly constraining moderation interventions by Reddit administrators. It was quarantined in June 2019, restricted in February 2020, and finally banned in June 2020, but despite precursory work on the matter, the effects of this sequence of interventions are still unclear. In this work, we follow a multidimensional causal inference approach to study data containing more than 15M posts made in a time frame of 2 years, to examine the effects of such interventions inside and outside of the subreddit. We find that the interventions greatly reduced the activity of problematic users. However, the interventions also caused an increase in toxicity and led users to share more polarized and less factual news. In addition, the restriction had stronger effects than the quarantine, and core users of r/The_Donald suffered stronger effects than the rest of users. Overall, our results provide evidence that the interventions had mixed effects and paint a nuanced picture of the consequences of community-level moderation strategies. We conclude by reflecting on the challenges of policing online platforms and on the implications for the design and deployment of moderation interventions.
Evaluating the effects of moderation interventions is a task of paramount importance, as it allows assessing the success of content moderation processes. So far, intervention effects have been almost solely evaluated at the aggregated platform or community levels. Here, we carry out a multidimensional evaluation of the user-level effects of the sequence of moderation interventions that targeted r/The_Donald: a community of Donald Trump adherents on Reddit. We demonstrate that the interventions: 1) strongly reduced user activity; 2) slightly increased the diversity of the subreddits in which us…
We study the emergence of support for Donald Trump in Reddit's political discussion. With almost 800k subscribers, "r/The_Donald" is one of the largest communities on Reddit, and one of the main hubs for Trump supporters. It was created in 2015, shortly after Donald Trump began his presidential campaign. By using only data from 2012, we predict the likelihood of being a supporter of Donald Trump in 2016, the year of the last US presidential elections. To characterize the behavior of Trump supporters, we draw from three different sociological hypotheses: homophily, social influence, and social …
Should social media platforms override a community's self-policing when it repeatedly break rules? What actions can they consider? In light of this debate, platforms have begun experimenting with softer alternatives to outright bans. We examine one such intervention called quarantining, that impedes direct access to and promotion of controversial communities. Specifically, we present two case studies of what happened when Reddit quarantined the influential communities r/TheRedPill (TRP) and r/The_Donald (TD). Using over 85M Reddit posts, we apply causal inference methods to examine the quarant…
Reddit administrators have generally struggled to prevent or contain such discourse for several reasons including: (1) the inability for a handful of human administrators to track and react to millions of posts and comments per day and (2) fear of backlash as a consequence of administrative decisions to ban or quarantine hateful communities. Consequently, as shown in our background research, administrative actions (community bans and quarantines) are often taken in reaction to media pressure following offensive discourse within a community spilling into the real world with serious consequences…
Widespread conspiracy theories may significantly impact our society. This paper focuses on the QAnon conspiracy theory, a consequential conspiracy theory that started on and disseminated successfully through social media. Our work characterizes how Reddit users who have participated in QAnon-focused subreddits engage in activities on the platform, especially outside their own communities. Using a large-scale Reddit moderation action against QAnon-related activities in 2018 as the source, we identified 13,000 users active in the early QAnon communities. We collected the 2.1 million submissions …
Online social media foster the creation of active communities around shared narratives. Such communities may turn into incubators for conspiracy theories -- some spreading violent messages that could sharpen the debate and potentially harm society. To face these phenomena, most social media platforms implemented moderation policies, ranging from posting warning labels up to deplatforming, i.e., permanently banning users. Assessing the effectiveness of content moderation is crucial for balancing societal safety while preserving the right to free speech. In this paper, we compare the shift in be…
In this dataset paper, we present a three-stage process to collect Reddit comments that are removed comments by moderators of several subreddits, for violating subreddit rules and guidelines. Other than the fact that these comments were flagged by moderators for violating community norms, we do not have any other information regarding the nature of the violations. Through this procedure, we collect over 2M comments removed by moderators of 100 different Reddit communities, and publicly release the data. Working with this dataset of removed comments, we identify 8 macro norms---norms that are w…
The Big Ban Theory: A Pre- and Post-Intervention Dataset of Online Content Moderation Actions
IIT-CNR, Italy（意大利国家研究委员会意大利理工学院）；Dept. of Computer Science, University of Pisa, Italy（比萨大学计算机系）；Dept. of Information Engineering, University of Pisa, Italy（比萨大学信息工程系）
Increasingly people form opinions based on information they consume on online social media. As a result, it is crucial to understand what type of content attracts people's attention on social media and drive discussions. In this paper we focus on online discussions. Can we predict which comments and what content gets the highest attention in an online discussion? How does this content differ from community to community? To accomplish this, we undertake a unique study of Reddit involving a large sample comments from 11 popular subreddits with different properties. We introduce a large number of…
Echo chambers in online social networks, whereby users' beliefs are reinforced by interactions with like-minded peers and insulation from others' points of view, have been decried as a cause of political polarization. Here, we investigate their role in the debate around the 2016 US elections on Reddit, a fundamental platform for the success of Donald Trump. We identify Trump vs Clinton supporters and reconstruct their political interaction network. We observe a preference for cross-cutting political interactions between the two communities rather than within-group interactions, thus contradict…
We present a qualitative analysis of interviews with participants in the NoSleep community within Reddit where millions of fans and writers of horror fiction congregate. We explore how the community handled a massive, sudden, and sustained increase in new members. Although existing theory and stories like Usenet's infamous "Eternal September" suggest that large influxes of newcomers can hurt online communities, our interviews suggest that NoSleep survived without major incident. We propose that three features of NoSleep allowed it to manage the rapid influx of newcomers gracefully: (1) an acti…
As more people flock to social media to connect with others and form virtual communities, it is important to research how members of these groups interact to understand human behavior on the Web. In response to an increase in hate speech, harassment and other antisocial behaviors, many social media companies have implemented different content and user moderation policies. On Reddit, for example, communities, i.e, subreddits, are occasionally banned for violating these policies. We study the effect of these regulatory actions as well as when a community experiences a significant external event …
Federating Governance: How Community Rules Scale with Mastodon Instances
Luddy School of Informatics, Computing, and Engineering, Indiana University（印第安纳大学卢迪信息、计算与工程学院）；University of Virginia（弗吉尼亚大学）；Center Synergy of Systems, TUD Dresden University of Technology（德累斯顿工业大学系统协同中心）
McCormick School of Engineering and Applied Science, Northwestern University（西北大学麦考密克工程与应用科学学院）；Northwestern Institute on Complex Systems（西北大学复杂系统研究所）；KDI School of Public Policy and Management（韩国行政大学院公共政策与管理学院）；Santa Fe Institute（圣塔菲研究所）；Booth School of Business, The University of Chicago（芝加哥大学布斯商学院）；Seoul National University（首尔大学）；Complexity Science Hub, Vienna（维也纳复杂科学中心）
The proliferation of radical online communities and their violent offshoots has sparked great societal concern. However, the current practice of banning such communities from mainstream platforms has unintended consequences: (I) the further radicalization of their members in fringe platforms where they migrate; and (ii) the spillover of harmful content from fringe back onto mainstream platforms. Here, in a large observational study on two banned subreddits, r/The\_Donald and r/fatpeoplehate, we examine how factors associated with the RECRO radicalization framework relate to users' migration de…
