Disinformation ecosystems: deepfakes, bot networks, coordinated inauthentic behavior

Intuition Beginner

A disinformation ecosystem is more than a single false claim. It is the network of actors, technologies, and platforms through which fabricated or misleading information is produced, amplified, and received by audiences. The phrase, drawn from Yochai Benkler and colleagues' Network Propaganda (2018), captures that disinformation spreads because an entire infrastructure sustains it.

Claire Wardle and Hossein Derakhshan's 2017 Council of Europe framework sorts harmful information into three families. Misinformation is false but spread without intent to harm — a journalist retweets an unverified rumor in good faith. Disinformation is false or misleading and deliberately created to cause harm. Malinformation is genuine information weaponized to inflict harm, such as a leaked private message. Distinguishing them matters because each demands a different response.

Modern disinformation differs from earlier propaganda (§30.02.03, Edward Bernays; Walter Lippmann's Public Opinion) in speed, cost, and deniability. A single operator can now command thousands of automated accounts, fabricate convincing video and audio, and reach millions before any fact-checker (§36.02.02) can respond. The 2016 US election interference by Russia's Internet Research Agency, the COVID-19 infodemic (§35.02.03, §35.06.03), and weaponized deepfakes are not isolated incidents but features of this ecosystem.

Visual Beginner

The Wardle-Derakhshan typology clarifies what is often muddled in everyday usage.

Type	Intent to harm?	Based on truth?	Example
Misinformation	No	False	Sharing a wrong event date
Disinformation	Yes	False or misleading	Fabricated quote attributed to a politician
Malinformation	Yes	True (but weaponized)	Revenge porn, leaked private messages

Worked example Beginner

Consider how Russia's Internet Research Agency (IRA) operated during the 2016 US presidential election, as documented in the Mueller Report. The IRA, based in St. Petersburg, employed hundreds of operatives who created fake American personas across Facebook, Twitter, Instagram, and YouTube.

The operation ran hundreds of accounts organized by theme: Black Lives Matter groups, pro-Second-Amendment pages, immigrant-rights communities, and Texas secessionists. Each account posted authentic-looking content to build a real following. Only after trust was established did the accounts inject divisive or fabricated political material, often timed to amplify existing tensions around rallies and protests (§30.07.*).

A second layer of automation amplified the human-run accounts. Bots — automated accounts running scripts — retweeted and liked the IRA content, manufacturing the appearance of organic popularity. Platform recommendation systems (§36.04.*), designed to surface engaging content, then pushed the most outrage-inducing posts to ever larger audiences. The disinformation did not need to persuade everyone; it needed to deepen division and erode trust.

This single operation combined disinformation (fabricated events), malinformation (weaponized real tensions), computational propaganda (bots and troll farms), and platform amplification. The cost was negligible against a traditional advertising campaign; the reach was global. The lesson: a modern disinformation ecosystem is an assembly line, and each stage can be staffed by a different actor.

Check your understanding Beginner

Formal definition Intermediate+

Definition (information disorder, Wardle-Derakhshan 2017). The Council of Europe framework partitions harmful information content into three categories by the two binary attributes accuracy and intent to harm:

• Misinformation: information that is false but not created or shared with the intent to cause harm.
• Disinformation: information that is false or deliberately misleading and is created or shared with the intent to cause harm.
• Malinformation: information that is grounded in reality but is used to inflict harm on a person, organization, or country.

The framework's value is taxonomic precision. It replaces the loose public usage of "fake news" — a term that conflates satire, error, propaganda, and fabricated content — with categories whose members share a remediation strategy: misinformation is addressed by correction, disinformation by attribution and disruption of the network, malinformation by protection of the target.

Definition (computational propaganda, Woolley-Howard 2018). Computational propaganda is the use of algorithms, automation, and human curation to distribute manipulated messages over social media networks. Its two principal instruments are:

• Bots: automated accounts executing scripts to post, amplify, or harass at machine speed and scale.
• Troll farms: human-operated account collectives that behave like bots in coordination while producing content a bot cannot.

Definition (coordinated inauthentic behavior, CIB). CIB is the coordinated action of multiple accounts that conceals the true identity, location, or purpose of the operators to manipulate public discourse. CIB is a network-level property: no single account need post false content, yet the aggregate behavior constitutes manipulation. Detecting CIB is nontrivial because the unit of analysis is the coordination pattern, not any individual post, and sophisticated operators mimic genuine grassroots activity closely enough to evade naive thresholds.

Definition (deepfake). A deepfake is synthetic media generated by a machine-learning model — most commonly a generative adversarial network (GAN) or diffusion model — in which a person's likeness, voice, or action is fabricated through face swapping, voice cloning, or full scene synthesis. The GAN architecture trains a generator $G$ against a discriminator $D$ in a two-player game: $G$ learns to map latent noise to realistic samples, $D$ learns to classify samples as real or generated. Voice cloning operates analogously on audio waveforms or spectrograms, requiring only seconds of target audio to produce convincing arbitrary speech (§33.07.*).

Key result: asymmetric diffusion in disinformation networks Intermediate+

Benkler, Faris, and Roberts's Network Propaganda (2018) analyzed several million media stories from 2015–2018 and found a striking asymmetry in how false and misleading content diffused through the American media ecosystem.

Result (asymmetric diffusion). Disinformation spread through a distinct, structurally segregated subsystem of the right-wing media ecosystem — anchored by outlets like Breitbart and Fox News — in which false stories could travel from fringe sites to mass audiences without correction by gatekeepers. The left-wing and center media ecosystems, by contrast, displayed far more internal correction: outlets cited and rebutted each other, and false stories were filtered before reaching mass audiences.

Three structural mechanisms explain the asymmetry. First, insular citation networks: the right-wing subsystem cited itself, creating a closed loop in which external correction could not penetrate. Second, audience insularity: audiences consumed nearly exclusively in-system sources, so counter-arguments never reached them. Third, institutional degradation: the weakening of legacy gatekeepers (major newspapers, broadcast networks) removed the shared factual baseline that previously limited diffusion.

The practical consequence is that the same fabricated claim, released into different media ecosystems, produces very different reach and persistence. Platform design (§36.04.*) compounds this: engagement-based ranking rewards the outrage that disinformation generates, regardless of the system's political lean. The result reframes the problem. Disinformation is not merely a matter of individual credulity; it is a property of network structure, and remediation must act on the structure — the citation loops, the audience segmentation, and the ranking incentives — not only on the individual reader.

Exercises Intermediate+

Advanced results Master

Participatory disinformation

Kate Starbird and colleagues (2019) recast a central assumption: disinformation is not only produced by covert actors but collaboratively co-produced by ordinary users. Studying alternative media during crisis events (mass shootings, terror attacks), they found that conspiratorial narratives emerged from thousands of users contributing fragments — speculation, screenshots, out-of-context photos — that aggregated into coherent conspiracy theories. No single user fabricated the whole; the crowd assembled it. This participatory disinformation reframes remediation: targeting the originator is impossible because the originator is a distributed collective, and the motivation is often sincere belief rather than paid manipulation.

The deepfake detection arms race

Synthetic-media generation and detection are locked in an adversarial loop structurally identical to the GAN's own generator-discriminator dynamic. Every detection advance — frequency-domain artifacts, blinking patterns, biological signal recovery — is countered within months by generators trained against the detector. Current research suggests no static detector will remain effective; durable defense likely requires provenance-based approaches (cryptographic content authenticity, C2PA) rather than detection alone. The implication for evidentiary practice is severe: in the limit, no video clip is self-authenticating, and the chain of custody, not the pixels, must carry the trust.

Inoculation theory and prebunking

Drawing on William McGuire's inoculation theory, recent interventions (van der Linden, Lewandowsky) show that exposing audiences to weakened doses of manipulation techniques — before they encounter the genuine disinformation — confers measurable resistance. "Prebunking" short videos explaining common tactics (false dichotomy, emotional manipulation, impersonation) reduce susceptibility on subsequent exposure. This is structurally analogous to vaccination (§35.06.03) and reframes media literacy (§29.07.*) as preventive rather than reactive: controlled exposure to the manipulative pattern, in a safe context, builds the recognition needed in the wild.

Information cascades and the network position of brokers

Formal network models of information diffusion (independent cascade, linear threshold) show that whether a fabricated claim goes viral depends heavily on the network position of its early adopters. A claim seeded at a high-betweenness broker — an account connecting otherwise separate communities — diffuses far further than the same claim seeded in a dense clique. This explains why coordinated operations invest in compromising or impersonating precisely these broker accounts, and why platform interventions that quarantine broker accounts during suspected operations are disproportionately effective per unit of action.

Connections Master

To propaganda history (§30.02.03)

Computational disinformation is continuous with the 20th-century propaganda lineage — Edward Bernays's Propaganda (1928) and Walter Lippmann's Public Opinion (1922), both covered in §30.02.03 on advertising and propaganda. What changes is not the intent but the infrastructure: Bernays manipulated mass audiences through broadcast media; contemporary operators manipulate segmented micro-audiences through algorithmic targeting at near-zero marginal cost. The Bernaysian insight that persuasion could be engineered persists; the engineering is now automated and scaled.

To computing and deep learning (§33.07.*)

Deepfakes are a direct application of the deep-learning foundations in §33.07.*. Generative adversarial networks (face swapping), diffusion models (image synthesis), and neural voice cloning all rest on the representational learning covered there. The same architectural advances powering creative tools power deception tools; the only difference is intent and governance. The unit thus inherits computing's open question of how to verify the provenance of generated content.

To algorithmic curation (§36.04.*)

Disinformation does not spread on merit alone; it spreads because engagement-based ranking rewards the high-arousal emotions (outrage, fear, disgust) that disinformation deliberately provokes. The filter-bubble and echo-chamber dynamics of §36.04.* explain why corrections fail to reach the audiences that need them: the algorithmic environment is structurally sealed. Platform amplification is not a neutral channel but an active participant in the ecosystem.

To infectious disease and vaccine hesitancy (§35.02.03, §35.06.03)

The WHO-declared COVID-19 infodemic made literal the metaphor of information-as-pathogen. Misinformation about viral transmission (§35.02.03, viral pathogenesis) and vaccine safety (§35.06.03, vaccine hesitancy) demonstrably increased vaccine refusal and non-pharmaceutical-intervention noncompliance, with measurable mortality consequences. The epidemiological modeling of rumor diffusion borrows directly from infectious-disease SIR models, treating believers, skeptics, and fact-checkers as compartments.

To social movements (§30.07.*)

Election interference and foreign influence operations target the fault lines that genuine social movements (§30.07.*) organize around — race, immigration, gun rights. The IRA's 2016 operation did not invent divisions; it amplified existing ones, co-opting the language and imagery of authentic movements to inject divisive content. Studying disinformation thus requires studying the movements it parasitizes.

To investigative journalism (§36.02.02)

Fact-checking infrastructure — ClaimReview, the International Fact-Checking Network (IFCN), and the verification workflows of §36.02.02 — is the principal reactive layer against disinformation. But investigative journalism's strength (deep, slow verification) is structurally mismatched against disinformation's speed. Bridging this gap requires computational tools that surface claims in near-real-time for verification, an open research problem.

To social psychology (§29.07.*)

Disinformation exploits the cognitive and social-psychological mechanisms of §29.07.*: confirmation bias, motivated reasoning, and intergroup prejudice. The contact hypothesis — that controlled intergroup contact reduces prejudice — has an analogue in media-literacy practice: controlled exposure to divergent sources and to manipulative techniques (inoculation) builds the resistance that unmediated contact with disinformation erodes. The most durable defenses combine structural (platform) and psychological (literacy) interventions.

To platform accountability (§36.06.*)

Facebook's third-party fact-checking program, Twitter/X's Community Notes, and broader content moderation all operate at the contested boundary covered in §36.06.*. Each is an attempt to internalize the externality that engagement-based business models impose on public discourse. None is sufficient alone; the ecosystem's resilience depends on layering structural, procedural, and educational countermeasures — the thread this unit traces.

Historical and philosophical context Master

From Bernays to bots

Edward Bernays's 1928 Propaganda argued that "the conscious and intelligent manipulation of the organized habits and opinions of the masses is an important element in democratic society." Bernays was writing in the era of broadcast media — one-to-many, expensive, professionally gatekept. Walter Lippmann's Public Opinion (1922) had already diagnosed the problem Bernays embraced: ordinary citizens cannot directly apprehend the complex world they are asked to govern, so they rely on the "pictures in their heads" supplied by media. The 20th-century settlement was a professional press that, however imperfect, served as a shared gatekeeper.

Computational propaganda dissolves that settlement. Samuel Woolley and Philip Howard's Computational Propaganda (2017) documented how automation — bots, sockpuppets, and the human-run troll farms that preceded and outlast them — collapses the cost of producing the appearance of public opinion. Where Bernays needed a press office and advertising budget, a contemporary operator needs a laptop and API access. The Internet Research Agency's 2016 US election operation, detailed in the Mueller Report, demonstrated the model at scale: hundreds of operatives, thousands of accounts, millions of impressions, for less than the cost of a single national ad buy.

The Cambridge Analytica moment

The 2018 Cambridge Analytica revelations — in which data on tens of millions of Facebook users was harvested to build psychographic targeting models for political advertising — crystallized public concern about computational influence. The company's own marketing overstated its effectiveness, but the underlying capability (microtargeting on psychological profiles derived from behavioral data) was real and is now standard practice. The episode is less a singular scandal than a window onto the routine machinery of data-driven persuasion.

The COVID-19 infodemic

On February 2, 2020, the WHO declared an "infodemic" alongside the COVID-19 pandemic. The accompanying disinformation — false cures, fabricated origins, vaccine conspiracy theories — spread through the same ecosystem this unit describes, with consequences measurable in excess mortality. The infodemic confirmed what researchers had warned: that public-health communication during a crisis is a disinformation contest as much as an information-delivery problem, and that the science of §35.02.03 and §35.06.03 cannot reach the public without traversing a contested information environment.

The Wardle-Derakhshan reframing

Claire Wardle and Hossein Derakhshan's 2017 Council of Europe report Information Disorder did two things. First, it retired the analytically useless term "fake news" — which had come to mean anything from fabricated content to unwelcome reporting — and replaced it with the mis/dis/mal taxonomy. Second, it insisted that the problem is not only about falsehood but about harm and intent, forcing analysts to specify which of the three they mean. The typology is now the field's lingua franca.

The philosophical problem of the marketplace of ideas

John Stuart Mill's marketplace-of-ideas defense of free speech assumes that truth prevails in open competition. The disinformation ecosystem strains that assumption past breaking. When fabricated content can be produced at near-zero cost, amplified by automated networks, and delivered by engagement-maximizing algorithms into structurally sealed audiences, the "competition" is rigged — not by censorship but by architecture. Jason Stanley's How Propaganda Works (2015) frames the consequence: propaganda that erodes the epistemic resources citizens need to evaluate claims is not merely a bad argument but an attack on the preconditions of democratic deliberation itself. The remediation question — how to defend the informational commons without state censorship — has no settled answer, and is the open problem this unit trains students to engage.

Bibliography Master

1. Wardle, C. and Derakhshan, H. (2017). Information Disorder: Toward an Interdisciplinary Framework for Research and Policymaking. Council of Europe. The mis/dis/mal typology that organizes the field.

2. Benkler, Y., Faris, R. and Roberts, H. (2018). Network Propaganda: Manipulation, Disinformation, and Radicalization in American Politics. Oxford University Press. The asymmetric-diffusion analysis of the US media ecosystem.

3. Woolley, S.C. and Howard, P.N. (eds.) (2018). Computational Propaganda: Political Parties, Politicians, and Political Manipulation on Social Media. Oxford University Press. The founding collection on bots, automation, and digital manipulation.

4. Starbird, K., Arif, A. and Wilson, T. (2019). "Disinformation as collaborative work: Surfacing the participatory nature of strategic information operations." Proceedings of the ACM on Human-Computer Interaction, 3(CSCW). Participatory disinformation co-production.

5. Howard, P.N. (2020). Lie Machines: How to Save Democracy from Troll Armies, Deceitful Robots, Junk News Operations, and Political Operatives. Yale University Press. The production pipeline of computational propaganda.

6. Lippmann, W. (1922). Public Opinion. Harcourt, Brace. The original diagnosis of media-shaped perception.

7. Bernays, E. (1928). Propaganda. Horace Liveright. The engineering-of-consent framework that computational propaganda automates.

8. Stanley, J. (2015). How Propaganda Works. Princeton University Press. The philosophical account of propaganda as erosion of deliberative capacity.

9. Vaccari, F. and Chadwick, A. (2020). "Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news." Social Media + Society. Empirical effects of synthetic media on trust.

10. World Health Organization (2020). Managing the COVID-19 infodemic: Promoting healthy behaviours and mitigating the harm from misinformation and disinformation. WHO. The infodemic declaration and framework.