Judgment of the Social Media Tribunal held at Berlin,

2 FINDINGS OF FACT

1. Based on the testimony of several witnesses we find that the SM platforms have had a negative impact on democratic discourse, citizen well-being and safety based on the proliferation of misinformation, disinformation, and counter-factual conspiracy theories. In particular, we cite in support of this conclusion the testimony of Baroness Kidron, a member of the UK House of Lords and a founder of the UK 5 Rights Foundation. She cited, in particular, the impact of misinformation and disinformation as influencing the outcome of the Brexit vote in the UK and the response to the Covid-19 pandemic in many countries around the world. She recommends more aggressive regulation of SM companies because without regulation they pose a risk to democracy. She also urges that the SM companies be held accountable for the societal harms they cause. In particular, she concluded that the state has a duty, at the very least, to protect children.

2. We also cite, in support of this finding of fact, the testimony of journalist Carole Cadwalladr. In her testimony she explained how Cambridge Analytica used Facebook to support the campaign for the UK to leave the EU. She gave the example of ads targeted to one small town stating that “Turkey will join the EU and Muslims will flood your town.” She also cited pictures of ISIS terrorists committing murders. This type of speech and visual images, spreading fear and misinformation, caused a surprisingly strong vote in favor of leaving the EU. Ms. Cadwalladr was targeted with death threats because of her work as a journalist in exposing this conduct.

3. Another witness, Rewan al Haddad, the campaign director of Eko, an organization seeking to hold corporations accountable for their conduct testified that SM companies repeatedly failed to block ads containing hate speech, often accompanied by digital images. Her organization submitted ten ads containing hate speech to both Meta and X. The ads called for immigrant to be gassed and put in concentration camps. Immigrants were called animals and pathogens. Half of those ads were nonetheless approved by Meta and all ten were approved by X. These ads were to be used in Germany and in India, in order to influence critical elections. These posts should have been blocked according to the standards of permissible speech adopted by these companies in their own terms of service.

4. Based on the testimony of many witnesses, we find that there are reasonable grounds to believe that the SM companies, knowingly and intentionally allowed, and/or failed to remove, unlawful or inappropriate content including, but not limited to hate speech, incitement to commit violence against people based on their race, ethnicity, gender, or national origin, sexually explicit material targeted to children, violent acts involving children, and extortion schemes based on invasions of personal privacy. We cite the testimony of several witnesses in support of this finding of fact. Alexandra Pascalidou, a journalist, testified that she was a victim of racism and sexism, and was repeatedly subjected to hate speech. She testified that hate speech is not free speech but rather is used as a weapon to silence certain voices.

5. With respect to hate speech we cite, in particular, the testimony given by Nay San Lwin, a Rohingya activist and co-founder of Free Rohingya Coalition, and Tun Khin, a Rohingya activist and President of the Burmese Rohingya Organisation of the UK. Mr. Lwin testified that Facebook was responsible for spreading misinformation that led to the persecution of the Rohingya, a Muslim minority in Myanmar. He decried the lack of regulations prohibiting harmful content on SM. Both witnesses testified that many posts identified all Rohingya as criminals. Ultimately 750,000 Rohingya were forced to flee to Bangladesh. As noted, both witnesses testified that hate speech is not free speech. Both witnesses strongly supported the need for moderators to monitor the content of speech to prevent hate speech and the violence such speech encourages. Further, the witnesses testified that despite actual notice, Facebook failed to take down the reported hate speech.

6. The Tribunal also received documentary evidence from the Human Rights Council’s report of an international fact-finding mission on Mynamar stating that “Facebook has been a useful instrument for those seeking to spread hate (against the Rohingya community), in a context where, for most users, Facebook is the Internet. Although improved in recent months, the response of Facebook has been slow and ineffective”. That evidence also demonstrated that Facebook acted contrary to its own policies by failing to remove hate speech. We quote from Facebook’s own policy against hateful content as follows:

We believe that people use their voice and connect more freely when they don't feel attacked on the basis of who they are. That is why we don't allow hateful conduct on Facebook, Instagram or Threads…

We define hateful conduct as direct attacks against people – rather than concepts or institutions – on the basis of what we call protected characteristics (PCs): race, ethnicity, national origin, disability, religious affiliation, caste, sexual orientation, sex, gender identity and serious disease. Additionally, we consider age a protected characteristic when referenced along with another protected characteristic. We also protect refugees, migrants, immigrants and asylum seekers from the most severe attacks (Tier 1 below) . . .

We remove dehumanising speech, allegations of serious immorality or criminality, and slurs. We also remove harmful stereotypes, which we define as dehumanising comparisons that have historically been used to attack, intimidate or exclude specific groups, and that are often linked with offline violence. Finally, we remove serious insults, expressions of contempt or disgust, swearing and calls for exclusion or segregation when targeting people based on protected characteristics.

7. For this finding of fact, we also cite the testimony of Imran Ahmed, the CEO of the Center for Countering Digital Hate, located in the UK and the US. He pointed out that the algorithms used to spread hate speech and disinformation are hidden from view. He testified that transparency in the use of algorithms is essential. Transparency is important to ensure that content which violates a platform’s policies is not amplified by its own AI-based recommendations. He also testified that independent content moderation was essential to eliminating hate speech and dangerous conspiracy theories that often encourage and incite violent conduct. He testified that platforms should moderate their content through AI, but followed by human moderators, followed by a process for taking down offensive content after receiving and reviewing a report of such content on the platform. He concluded that the absence of effective fact-checking and content moderation causes an increase in hate speech on SM platforms. He recommended transparency of and accountability for the SM companies.

8. The Tribunal also received another Report prepared by the Center for Countering Digital Hate, titled “X content moderation failure.” The Report found that X continues to host 86% of the posts that have been reported to contain extreme hate speech despite its published policy to combat content “motivated by hatred, prejudice or intolerance”. A News report published in the Los Angeles Times on September 25, 2024, was also submitted to the Tribunal. The article is titled “Elon Musk’s X says its policing harmful content as scrutiny of the Platform grows.” The article noted that X’s approach “is to restrict the reach of potentially offensive posts rather than taking down the posts.” X’s 2024 transparency report, which was also received in evidence,showed that X chose to “label” content rather than removing or suspending accounts. The transparency report showed that of the 82 million complaints received by X, X only suspended one million accounts and removed or labelled 2.6 million posts.

We quote here the following clause from X’s Policy titled “Hateful conduct”

Hateful references:

We prohibit targeting individuals or groups with content that references forms of violence or violent events where a protected category was the primary target or victims, where the intent is to harass. This includes, but is not limited to media or text that refers to or depicts:

● genocides, (e.g., the Holocaust);

● lynchings.

What happens if you violate this policy?

The following is a list of potential enforcement options for content that violates this policy:

Making content less visible on X by:

Removing the Post from search results, in-product recommendations, trends, notifications, and home timelines.

Restricting the Post discoverability to the author’s profile. Downranking the Post in replies.

Restricting likes, replies, reposts, quotes, bookmarks, share, pin to profile, or engagement counts.

Excluding the Post from having ads adjacent to it. Excluding Posts and/or accounts in email or in-product recommendations.

Requiring Post removal.

For example, we may ask someone to remove the violating content and serve a period of time in read-only mode before they can Post again.

Suspending accounts that violate our Hateful Profile policy.

9. Having examined the documentary evidence and heard the witnesses, we conclude that reduced visibility/labelling of hate speech content rather than blocking of such posts is clearly unjustifiable and is violative of human rights. Dr. Julia Ebner, a professor at the University of Oxford who specializes in studying radicalism and extremism has published articles and books on the spread of hate speech and other harmful content on the Internet. She has concluded that the SM companies provide a safe haven to extremists who reach a very wide audience and inspire violence based on the hate speech they espouse. She gave an example of QAnon which has used platforms including Discord and Telegram to encourage antisemitism, misogyny, white supremacy, and jihadism. Like Mr. Ahmed, Dr. Ebner recommended the need for transparency of algorithms, and accountability on the part of SM companies for failing to prevent the spread of violent and hateful speech and to take down illegal content when reported.

10. We reject the defense that SM platforms were justified in allowing such materials to be available because of the concept of freedom of speech. Speech that incites or causes violence, tramples on human rights – such as the right to privacy and the right to protect one’s reputation -- violates the Rule of Law because it causes harm and is outside the bounds of ethical, legal, and humanitarian norms. It undermines the principles of respect, dignity, and peaceful coexistence that are essential to human rights.

11. We find that there are reasonable grounds to believe that the SM platforms are not proactive or reactive in ensuring that content posted on their platforms by users complies with their own policies, and the principles governing human rights. This failure is a direct result of SM platforms’ desire for increased profits, rather than protecting the rights of their users. Advertising is the way in which the SMs obtain their revenue and their profit. Digital advertising is worth one billion dollars. The evidence showed that the more the user accesses the platform, and the longer the user remains on the platform, the more the platform will benefit by users noticing the ads, looking at the contents of the ads, and potentially purchasing an advertised product. All these activities by users generate revenue to the platform.

12. For this finding of fact we rely, in particular, on the testimony of Professor Matthew Hindman of George Washington University, who testified that digital platforms make almost all of their money through advertising. He noted that Meta and X make 98% of their revenue through advertising, and Reddit makes 91% of its revenue through advertising. His research supports the conclusion that more money is made when users stay on the platform as long as possible. He reports that the platforms encourage addictive behavior by users, including children. Professor Hindman testified that the takedown rates for reported illegal content on these platforms was less than 5% per year. He further testified that the majority of AI based recommendations on Facebook were to amplify “overwhelmingly toxic” content by superusers (those who most frequently post) and that the majority of such posts were from fake accounts.

13. For this finding of fact, we also rely on the testimony of Imran Ahmed, identified in paragraph 7 above. He testified in detail regarding the use of algorithms by the SM companies. He explained that algorithms are used by the SM companies to learn as much as possible about each user. In doing so, a formula is developed to increase profits by keeping users on the platform as much as possible. He testified that the formula is as follows: time spent on the platform times the number of user times the frequency of ads times the price of ads determines the revenue.

14. The EU’s Digital Services Act (“DSA”), adopted in 2022, requires SM companies to undertake content moderation of material posted on their platforms. The term “content moderation” is cited twenty-two times in the DSA. Similarly, in India, the IT Act of 2000 requires SM companies to conduct due diligence, including reporting certain cybersecurity instances to a Computer Emergency Response Team within 6 hours and requiring that they take down content upon receiving notice of a court order or direction from a government agency or voluntarily based on a user filing a complaint to its grievance redressal officer. The Indian Protection of Children from Sexual Offenses Act (“POCSO”) of 2012 mandates that SM companies report criminal or inappropriate content such as Child Sexual Abuse Material (“CSAM”) to law enforcement agencies.

15. While neither the DSA, nor Indian law, govern all the world’s jurisdictions, we find that there are reasonable grounds to believe that the failure to review content, either prior to publication, or surely after receiving notice that certain content is dangerous and/or inappropriate, is grossly negligent and reckless. We also find that SM platforms have the technology to review content by using AI and/or by using highly trained human reviewers and that the cost of doing so would not be prohibitive. We also find that SM platforms undoubtedly know that some of the content displayed on their platforms is dangerous and/or inappropriate, often leading to adverse effects on the safety, mental health and well-being of their users. Nonetheless, the preponderance of the credible evidence shows that despite that knowledge, the SM platforms often permit this material to be published and/or decline to promptly take it down when informed by user(s) of the dangers posed by the continued access to this information.

16. We find that there are reasonable grounds to believe that the SM platforms not only violate the provisions of the laws, guiding principles, and conventions cited above, but also knowingly and intentionally violate their own Terms of Use and Privacy Policies, which explicitly prohibit the publication of hate speech, incitement to violence, threats of violence, extortion and other impermissible content. We cite, in particular, the testimony of Sophie Zhang, formerly employed by Facebook, who testified that the company did a very poor job of policing adherence to their own Terms of Use and Privacy policies, particularly with respect to the proliferation of fake accounts spreading misinformation. Zhang also testified that the size of the content moderation team at Facebook was disproportionately lean as compared to the number of complaints received. She believed that Facebook’s decisions were arbitrary, often motivated by business considerations, with no action or delayed action on reporting illegal content. This is circumstantial evidence, at the very least, that the SM platforms knowingly and intentionally are permitting their platforms to be used by bad actors, who have committed cybercrimes and/or have interfered in elections by disseminating false or misleading information, and/or have been complicit in encouraging genocide or the involuntary transfer of population.

17. For this finding of fact, we also cite the testimony of Nina Jankowicz, the former Executive Director of the Department of Homeland Security’s Disinformation Governance Board. She testified, for example, that elections in Romania were cancelled due to election interference by SM platforms. She noted the ubiquitous presence of fake ads and fake news on SM. She also testified that women who report fake news often become the target of psychological abuse and threats to their safety by bad actors. She testified that a lot of women just self-select out of public life because of the threats that they face online. She reported many instances to SM companies where she was targeted through memes but that the platform failed to remove the illegal content despite that it violated its own policies. She recommended that legislation be enacted (and enforced) calling for transparency of and accountability for SM’s actions. She further recommended that SM platforms should adopt robust fact checking practices to detect fake news and prevent its amplification.

18. For this finding of fact, we also cite the testimony of Professor Arno Lodder, a professor of internet governance and regulation at a university in Amsterdam. He is an expert in data protection and privacy and has frequently published on these topics. He testified that SM platforms have user interface designs that manipulate users to share more personal data than is necessary or that they even know they are sharing. He recommended deployment of privacy protection measures and that SM companies adopt adequate complaint procedures requiring that inappropriate content be swiftly removed. He believes that this requires human intervention to respond responsibly and quickly to legitimate complaints.

19. With respect to hate speech, we provide only a few examples in support of this finding. Witnesses such as Nay San Lwin, and Tun Khin have testified that SM was used in Myanmar to encourage sexual violence, assaults on the minority Muslim Rohingya population, causing hundreds of thousands to flee the country. Posts such as “all Rohingya must be killed” “Rohingya are vermin that must be eliminated” “Rohingya must be driven out of the country” were allowed to appear on a SM platform.

20. Posts were published on SM platforms suggesting that children were ugly and were hated by everyone. Other posts led children to engage in self-harm or encouraged children to engage in sexually explicit acts and then suggested that having done so they were threatened by sextortion perpetrators that if they did not pay money videos or pictures of those acts would be widely disseminated and were told that their life was over any way and their only choice was to commit suicide. Even after a child committed suicide, family members continued to be threatened with public exposure of the sexually explicit images.

21. We reject the defense contention that the SM platforms are not responsible for the behavior of third parties who may be committing cybercrimes or engaging in criminal activities such as extortion, cyberbullying or cyberstalking. It is true that the SM platforms are not committing those crimes directly. But by permitting these third parties to publish material that is criminal or will lead to criminal conduct, and by failing to remove such material despite actual notice and expeditiously suspend and terminate accounts used to perpetrate cybercrimes, the SM platforms are facilitating those crimes.

22. These types of SM content clearly violate human rights and freedoms, such as the right to privacy, and the exploitation of data, and cause severe consequences to the victims of such conduct including self-harm, psychological trauma, and death. By failing to act to prevent online criminal activity, the SM platforms violate the human rights of their users as specified in the law, conventions and guidance cited earlier.

23. In support of this finding of fact, we cite the testimony of Noelle Martin, a victim of violations of her right to privacy. She described what she termed as “image based sexual abuse.” This abuse started when she was only seventeen years old by the use of deep fakes of her image. These are created by bad actors who impose her face onto pornographic images that are then widely distributed on SM. She reported this conduct to the websites on which they were posted, but neither did anything to stop the spread of these images. She testified that this conduct disproportionately affects women and girls who are ruthlessly targeted for sextortion. Such images are supposedly banned by the platforms’ terms of service, but the bans are not enforced. As a result, she suffered both mental and physical harm. As a legal matter she testified that the companies violated Article 17 of the ICCPR (see conclusions of law below); Article 12 of the European Declaration of Human Rights, and Article 27 of the Universal Declaration of Human Rights.

24. Another victim, Hana Mossman-Moore, was a victim of cyberstalking which she testified was a result of exposure on the Internet. A friend of hers was stalked by a person who eventually broke into her flat and killed her. Ms. Mossman-Moore testified that she began receiving threatening messages on her SM sites including WhatsApp, Instagram and Facebook. Some of the posts referred to her as “Hooker Hana” and encouraged users to rape her. Intimate pictures of her were widely distributed. Eventually she was forced to flee the country. She testified that the SM companies did little or nothing to prevent or stop this misuse of their platforms.

25. We find that SM platforms have made little or no effort to protect children from harmful content. Moreover, we find that the SM platforms know and are fully aware that children are endangered due to the absence of parental control and parental access to their children’s SM accounts. We also find the SM platforms have been complicit in knowingly causing children, through its algorithmic recommendations, to become addicted to using SM.

26. In support of this finding of fact, we cite the testimony of Leanda Barrington-Leach the Executive Director of an NGO titled “5 Rights Foundation” She testified that children are an important market segment for SM companies. She testified that children are defined as under eighteen, although children under thirteen are often targeted. SM companies, such as TikTok, intentionally design their sites to maximize the time that children spend on their platforms. Examples include push notifications, continuous scrolling, encouraging connecting with friends and in-app purchases. She testified that many children use SM up to five hours per day and often wake up at night to check their feed. She testified that the SM companies do not take appropriate measures to protect children. She noted that the absence of age restrictions and the continued posting of harmful content, coupled with addictive AI based usage of SM, has led to negative mental health effects, like the loss of analytical skills, memory formation, contextual thinking, conversational depth, empathy and increased anxiety among children. She provided the example of Instagram failing to shut down accounts flagged multiple times by the UK secret police and NGOs for sharing AI generated CSAM. She recommends increased efforts on the part of SM companies to undertake content moderation and parental control on children’s use of SM. She further recommended that children’s right to privacy must be respected and their personal data should not be collected for commercial purposes.

27. One witness, who used the pseudonym Jane Doe, testified that intimate images she had placed in a supposedly safe and private place were stolen from Snapchat, a SM platform, despite using a privacy setting with a “my eyes only” feature. The images were widely disseminated and published on other SM after the data theft causing her great harm such that she was forced to flee her country. Snapchat failed to take down the illegal content that invaded her privacy despite repeated written requests to do so. According to the witness, the SM companies have failed to implement robust systems to detect and take down CSAM, despite the fact that technical means can be deployed to do just that.

28. Three parents of children who committed suicide testified that SM platforms were responsible for these tragic deaths. Nicola Harteveld testified that her daughter committed suicide as a result of online bullying on Snapchat. When her daughter was only twelve years old she was subjected to bullying, leading her to believe that she was worthless. She was then subjected to harmful content that encouraged her to commit suicide. She took her own life at the age of fourteen. Her testimony demonstrated that SM platforms have failed to protect vulnerable children, despite clear knowledge of the risk of psychological harm and of suicide resulting from cyberbullying. Nicola Harteveld recommended stronger accountability, content moderation, algorithm transparency of SM and the importance of cyber education.

29. Will Claxton testified that his son, Finn, committed suicide at the age of 16. Fin had been a high-achieving model student until he began using drugs that he purchased through the Internet. He also shared intimate sexual images on a SM website. Eventually he was subjected to “sextortion” through that SM website. The father testified that his son was “groomed” through online bad actors who were responsible for selling him drugs and subjecting him to “sextortion.” Although children on the website used by Finn were required to be thirteen years old, his son was able to enter a false age when he began using Discord, a SM website. Mr. Claxton, an engineer, believes that at least one SM website is a “playground for predators.” He recommended that parents should have the ability to manage their children’s accounts. He also recommended that SM platforms verify the identity of their users and adopt robust age verification mechanisms. Finally, he recommended the use of AI/ machine learning technologies to monitor what is happening on SM platforms, the need to remove unlawful content, and that alerts should be sent to parents of any inappropriate content viewed or posted by their children.

30. A third parent, Brandon Guffey, is a state legislator in South Carolina (USA). His son Gavin committed suicide at the age of seventeen. Gavin was active on SM, where he gained access to drugs. He was also “groomed” to share intimate sexual images. He became a victim of online sexual exploitation. A woman began to blackmail Gavin using a SM platform. Gavin managed to transfer $15 via CashAPP but then decided the only way out of the nightmare was to kill himself. After his death, Mr. Guffey received direct messages from the extortionist as did his younger son (now sixteen), and his fourteen-year-old cousin. Mr. Guffey was devastated but stayed in politics hoping to make a difference. Due to his efforts, his state passed a law criminalizing digital sextortion. As a result of this tragedy, Meta implemented end to end encryption which means that no evidence would be available of the extortion communications. It also banned non-US actors from messaging domestic children under the age of sixteen. But there is still no federal legislation protecting children or requiring accountability by SM companies. Mr. Guffey recommends parental control over the devices used by their children. He also recommends that SM platforms be required to “red flag” bad actor activity and notify parents of such activity. He also strongly recommends the need for national legislation mandating these actions.

31. Another witness, Christopher Hadnagy, an expert in the field of social engineering, testified that children are being targeted on SM through sextortion, cyberstalking and other harmful content. Harmful challenges such as Blue Whale entice children to indulge in self-harm activities. These challenges have even encouraged suicide in extreme cases.