How to Balance Freedom and Censorship: Best Practices in Social Media Content Moderation

The necessity of social media content moderation is undeniable. The number of reasons to moderate is overwhelming: starting from preventing platforms from violating a law in a specific country or breaking a policy of a specific marketplace and finishing with protecting its users from harmful content. Harmful content encompasses a wide range of problematic material, which can be grouped into several categories for clarity:

• Explicit and Illegal Content: This includes NSFW material, child abuse imagery, terrorism-related content, and explicit violence. These are straightforwardly harmful and its moderation is typically uncontroversial, given the clear legal and ethical violations it presents.
• Potentially Legal but Objectionable Content: This includes adult content such as pornography, nudes, and sex educational materials, as well as some forms of hate speech and graphic content that might not be illegal. The moderation of this content is often debated because the line between acceptable and objectionable can be nebulous. For example, some platforms may allow artwork depicting a naked body, but remove nude content.
• Behavioral Issues: Includes bullying, trolling, harassment, and other forms of abuse, which are subjective and can be interpreted differently. Moderation in this area depends on whether a platform values freedom of expression more or aims to create a safe environment free from harassment.
• Politically or Culturally Sensitive Content: Including misinformation, extremist views without direct calls to violence, and politically sensitive content. These areas are particularly vulnerable to the freedom versus censorship debate, as the line between harmful speech and free expression can be particularly contentious.

This broader categorization helps in understanding the scope of social media content moderation needs and highlights the areas where the balance between protecting freedom of speech and preventing harm is most delicate and debated.

However, the borderline between moderation and censorship could be blurred and attempts to remove harmful content could be perceived as an act of censorship and violation of freedom of speech.

This article focuses on the balance between the two and show-cases clear examples of poor decisions made by popular platforms regarding freedom-censorship harmony. It also explores how various platforms have addressed these challenges, each employing unique strategies that reflect their specific user base and content dynamics. Secondary goal of this article is to provide recommendations and discuss the best practice moderation strategy a social media platform may follow.

Introduction: A Multifaceted Challenge of Content Moderation

To briefly summarize types of content moderation on Social Media Platforms, they could be: manual, where content is manually checked by human moderators, and automated, where the content is checked by AI. On top of that, moderation could be applied before or after the post is published, blocked by users themselves, or sent by users to moderators. Check the Table below to see the most common approaches for moderation. 

Table 1 – Most popular types of Social Media Moderation: (1) pre-moderation, (2) post-moderation, (3) reactive moderation, (4) automated moderation, (5) user-only moderation and (6) hybrid moderation [4]

Usually, pre-moderation is only done by AI, as manual checks would cause a delay. So, for small developing platforms a manual post-moderation could be the best choice. However, usually, big platforms prefer to set an AI pre-moderation filter to get benefits from automated moderation and then they apply reactive moderation so that moderators only check user-flagged or AI-flagged content.

This allows platforms to swiftly manage large volumes of content while still maintaining a level of human oversight crucial for nuanced decisions.

However, that is not the only strategy big platforms have tried! See the section below to check how Reddit implemented a User-only moderation type or how Tumblr let AI block content itself without confirmation from human moderators.

In the AI moderation case, you reduce the load from the moderation team by applying policy rules automatically to any post before it is even published. However, AI may struggle to understand the context and post’s intentions, which could lead to false positives. In other words, appropriate content could be mistakenly blocked. The root of such a problem is data complexity, which can be fixed with proper datasets and comprehensive AI architecture. 

The technical intricacies of training these AI systems involve creating advanced neural networks that can analyze not just text but also images and videos for harmful content. The ethical implications are significant, as these systems must balance accuracy with respect for user privacy and freedom of expression, ensuring they do not over-censor. 

Cases

Here’s an overview of the moderation process on Facebook. Posts suspected of breaking the company’s regulations, which range from spam to hate speech and content promoting violence, are identified and flagged either by users or automated machine learning systems. Straightforward violations are handled automatically, possibly resulting in post removal or account suspension. All other flagged content is sent to a queue where human moderators assess it. [9]

Twitter, for instance, has faced its own unique challenges with moderation. In 2020, Twitter introduced a new feature where tweets containing potentially harmful or misleading information related to COVID-19 were labeled to provide context. This approach aims to balance freedom of expression with the need for accurate information and public safety.

YouTube offers another interesting case. Its moderation system heavily relies on community guidelines and automated systems to detect and remove content that violates these policies. However, the platform has had to adjust its algorithms frequently to address criticisms about both undermoderation—such as failing to catch extremist content—and overmoderation, which often affects educational content or content creators discussing sensitive topics. This dynamic illustrates the complex balance platforms must maintain to foster a safe yet open digital environment.

Below we provide real-life edge cases examples of an imbalance of freedom and moderation. Those examples highlight the importance of allowing users to express themselves but limiting users not to breaking the law and spreading adversarial materials. Reddit serves as a demonstration of undermoderation and harm it caused, while Tumblr shows an example of AI automated overmoderation.

Reddit: Failure of freedom. Undermoderation

Throughout the development of WEB 2.0, an era of user-generated content, multiple social networks positioned themselves as a “bastion of free speech”. Reddit is one of the platforms that started as a platform where people could express themselves freely and anonymously. 

“The free speech policy was something I formalized because it seemed like the wiser course at the time…”

© CEO Yishan Wong

 However, in its early days, Reddit was infamous for loosening the leash too much. Their moderation policy includes 8 rules only, and their ambiguity and uncertainty make the boundary between violative and appropriate content very unclear. On top of that, Reddit provided a rather unpopular concept of communities (aka subreddits), with user-generated content and User-Only Moderation. 

That implies users will define their own moderation rules, built-in on top of 8 basic rules, for their own communities. Such user-defined moderation led to community-specific moderation rules, like the subreddit r/NASCAR banning users from posting memes and r/aww prohibiting any ‘sad’ content. Consequently, there was an environment for harmful and illegal communities. 

Infamous and currently banned community r/Pizzagate was used to spread conspiracy theories about the pedophilia ring connected to Hillary Clinton in 2016, just before the elections. This subreddit was used to spread hate-speech, conspiracies and eventually, this subreddit culminated in a man shooting a rifle inside a popular Washington pizzeria, believing he was saving children trapped in a sex slave ring. [2]

 Another incident happened on Reddit in the r/Jailbait community, where users were spreading sexually suggestive pictures of girls who looked suspiciously young. At that time, Reddit had no restrictions against nudes on the platform. Hence, users just put a comment that girls are older than 18 y.o., and Reddit closed their eyes to believing their users. Only after it was proven that one of the girls was 14 years old did the community was banned. However, the author wasn’t, and he then created multiple other communities like `r/ ChokeABitch` and others. 

That was definitely a policy recklessness, which was fixed with a new CEO coming and updating policy rules.    

Tumblr: Failure of censorship. Overmoderation 

In its early days, Tumblr was almost as pro-open speech as Reddit. Since Tumblr was founded in 2007, it has largely turned a blind eye to adult content. The company has tried to shield it from public view through Safe Mode and more stringent search filters. [3]

However, due to the incidents of child pornography found on the platform, Tumblr was removed from Apple’s app store in 2018. [7] Considering that in recent years, Apple sells almost half of all phones per year worldwide, a ban from the Apple store is practically a death sentence for an app. Primarily to survive on the market, the platform updated its policy including a point against adult content.

Tumblr is infamous for failing to implement a robust algorithm for content moderation. For example, to block any video/photo or gif posts with child sexual abuse (CSAM) Tumblr got a database of “known child sexual abuse material” and implemented a search for a match algorithm. All content with known illegal material was deleted. Obviously, that is not enough, as new material was posted. 

That is why they developed a rigorous AI pre-moderation model to filter posts before they are published. Human genitals, female-presenting nipples and any depictions of sex acts – all of them are getting banned before they are published. However, the algorithm also banned nonsexual nudity, like sex educational material, famous artwork, and TV show screenshots[10]. 

But that is not it! Tumbler’s AI banned posts of shoes, heart-shaped necklaces, tires, and many more. That even created a trend with the hashtag “#TooSexyforTumblr”, where users would share ridiculous decisions that the automated filter has removed. [8] 

Moreover, to eliminate any threats of inappropriate content, they provided a keyword-based AI filter, which disrupted ordinary users. For example, along with keywords like “porn”, “drugs”, and “sex”, they banned “Tony the Tiger”, “Eugene Levy”, numbers 69 and 420, “depression”, “PTSD”, and “bipolar”. 

Tumblr has eliminated any problematic content, ensuring they are aligned with the Apple Store’s policy but at what cost? Along this way, they removed erotic art, discussions of autistic people stimming, and more. [6] Their poor decisions on automated moderation approaches led to users leaving Tumblr in favor of Twitter and Instagram. That happened to Queer Woman Community [5], a community of “Internet Naked Girls” [1] and many others.

Such an attempt to protect users from harmful content with poor AI automated moderation only disrupted users. While Tumblr avoided being banned they also lost lots of users and communities, who eventually went to other, more open platforms.

Balancing Act: Discussing Social Media Content Moderation Strategies

Content moderation is a vital point for Social Media Platforms. Moderation ensures the safety and quality of the content users may see on the platform. A certain degree of moderation makes the internet a safer place to be, prevents extremists from organizing and planning terrorist attacks, blocks the spreading of illegal materials, or just cares for users to feel free using the platform at work (no NSFW content).

 However, while this moderation is aimed at improving user’s experience on the platform, it could feel more like censorship when mistakenly removing appropriate content. While relatively small platforms could afford manual pre- or post-moderation, that is rather ineffective for platforms that scale. Usually, big platforms like Facebook [9] apply AI-automated pre-moderation and then add reactive human-moderation where users and moderators work together to make content safer. Unfortunately, AI algorithms are imperfect, and therefore, human moderators must confirm any decisions that AI suggests to make.

The example with Reddit shows the necessity of a well-documented policy, prepared for illegal content spreading on the platform, a prepared moderation team, and the failure of a user-only moderated environment. It shows unreliability of users to moderate content and illustrates the necessity of AI automated or manual moderation.

The example with Tumblr shows the importance of a comprehensive AI algorithm, that would either have a lower False-Positive rate or have a human moderation approval in the moderation process. Designing a system with a low false negative rate reduces the likelihood of exposing users to damaging content, which is a desirable feature for online platforms. To design a system with a low false negative rate typically requires increasing the threshold which determines if content is deemed appropriate or not. In doing so, the false positive rate will likely increase as the system focuses on removing harmful content. However, a system with a high false positive rate can also damage platform reputation as users become frustrated by the removal of their content.

Several well-known alternative strategies exist, to hybrid AI automated moderation and manual moderation. Two popular approaches are Purgatory and Shadow Ban. In the case of purgatory strategy, if an algorithm marks content as inappropriate, it is sent to the moderation team and hides the post from the newsfeed. In this way, a potentially dangerous post wouldn’t be visible to users before the decision is made by manual moderation. An automated filter wouldn’t remove the posts themselves. Shadow Ban’s only difference is that it doesn’t notify the user that the post is hidden due to possible policy violations. Shadow banning is a well-known practice to mute a user to the rest of the community without their knowledge, which is helpful against bots, trolls and spammers.

Conclusion: Collaboration of Users, Moderators and AI

To conclude, implementing an automated AI pre-moderation, flagging doubtful posts for moderators, along with reactive manual moderation, is a good practice to reduce the load from human moderators and let them focus on suspicious posts only. AI shall not decide to block specific content, as the chance of high false positives would lead to strict and unreasonable censorship. In contrast, in opposite settings, false negatives would result in more harmful content being ignored by AI. Therefore, the best strategy is to combine the efforts of users who report suspicious content, AI algorithms that flag content before it is published and moderators who manually decide on each post. On top of that, Purgatory or Shadow Ban could be applied to posts reported by either users or AI algorithms to reduce the risk of other users being exposed to inappropriate content.

Let Trembit AI Solutions Improve your Social Media Content Moderation

Are you ready to elevate your social media platform’s moderation capabilities and ensure a safe, engaging environment for your users? At Trembit, we specialize in cutting-edge NSFW automated filter solutions and comprehensive content moderation systems tailored to the unique dynamics of social platforms.

Our AI-driven tools are meticulously designed to minimize false positives and accurately identify objectionable content, ensuring that your platform remains a welcoming place for constructive interaction while upholding the highest standards of user safety. With Trembit’s technology, you can automatically detect and manage explicit and potentially legal but objectionable content efficiently, allowing your team to focus on strategic initiatives rather than manual content checks.

Don’t let the challenges of content moderation hold your platform back. Partner with Trembit to harness the power of automation and expertise. Embrace our robust, scalable solutions that grow with your platform, ensuring compliance and user satisfaction. Contact us today to discuss how we can help you create a safer and more dynamic online community. Transform your content moderation strategy with Trembit—where technology meets peace of mind.

Let’s make the digital world a safer place together. Join us in setting new standards for social media safety and user engagement!

References

[1] V. Ashley, “Porn on Tumblr — a eulogy / love letter – Vex Ashley – Medium,” Medium, Dec. 06, 2018. https://vexashley.medium.com/porn-on-tumblr-a-eulogy-love-letter-6d45e70fefff (accessed Jul. 16, 2024).

[2] M. Haag and M. Salam, “Gunman in ‘Pizzagate’ Shooting Is Sentenced to 4 Years in Prison,” The New York Times, Jun. 23, 2017. Available: https://www.nytimes.com/2017/06/22/us/pizzagate-attack-sentence.html

[3] S. Liao, “Tumblr will ban all adult content on December 17th,” The Verge, Dec. 03, 2018. https://www.theverge.com/2018/12/3/18123752/tumblr-adult-content-porn-ban-date-explicit-changes-why-safe-mode 

[4] P. Mishra, “Social Media Moderation: How It Works and Importance,” Socialwalls Blog, Nov. 02, 2023. https://socialwalls.com/blog/social-media-moderation/ 

[5] “Queer Women Used Tumblr to Explore Sexuality. Now It’s Over.,” www.out.com. https://www.out.com/news-opinion/2018/12/04/queer-women-used-tumblr-explore-sexuality-now-its-over (accessed Jul. 16, 2024). 

[6] A. Silberling, “Tumblr is at war with Apple over ‘mature’ content on its app again,” TechCrunch, Dec. 29, 2021. https://techcrunch.com/2021/12/29/tumblr-ios-tags-ban-apple/ 

[7] “Tumblr removed from Apple app store over abuse images,” www.bbc.com, Nov. 20, 2018. Accessed: Jul. 16, 2024. [Online]. Available: https://www.bbc.com/news/technology-46275138 

[8] What Tumblr’s Ban on ‘Adult Content’ Actually Did, “What Tumblr’s Ban on ‘Adult Content’ Actually Did,” Electronic Frontier Foundation, May 20, 2019. https://www.eff.org/tossedout/tumblr-ban-adult-content 

[9] J. Vincent, “Facebook is now using AI to sort content for quicker moderation,” The Verge, Nov. 13, 2020. https://www.theverge.com/2020/11/13/21562596/facebook-ai-moderation 

[10] G. K. Young, “How much is too much: the difficulties of social media content moderation,” Information & Communications Technology Law, vol. 31, no. 1, pp. 1–16, Mar. 2021, doi: https://doi.org/10.1080/13600834.2021.1905593.