.png)
#020: The Business Behind Rage Bait: Renée DiResta on Who Profits From Your Outrage and How to Spot Propaganda
.jpg)
Renée DiResta
Renée DiResta is a social media researcher and the author of Invisible Rulers: The People Who Turn Lies into Reality. She studies adversarial abuse online, ranging from state actors running influence operations, to spammers and scammers, to issues related to child safety. In October 2024, she joined the Georgetown University McCourt School of Public Policy as an Associate Research Professor. Prior to that, she was the Technical Research Manager at the Stanford Internet Observatory. Renée has advised Congress, the White House, state legislatures, and business organizations on issues related to technology and policy, including information operations, generative AI, election security, researcher transparency, child safety, and more. In 2018-2019, at the behest of SSCI, she led outside teams investigating both the Russia-linked Internet Research Agency’s multi-year effort to manipulate American society and elections, and the GRU influence campaign deployed alongside its hack-and-leak operations in the 2016 election. Renée is a contributing editor at Lawfare and a contributor at The Atlantic. Her bylined writing has also appeared in Wired, Foreign Affairs, Columbia Journalism Review, New York Times, Washington Post, Yale Review, The Guardian, POLITICO, Slate, and Noema, and she also publishes in academic journals. She has been a Presidential Leadership Scholar; an Emerson Fellow, a Truman National Security Project fellow, Mozilla Fellow in Media, Misinformation, and Trust, a Harvard Berkman-Klein affiliate, and a Council on Foreign Relations term member.
LISTEN ANYWHERE
Your social media feed is full of rage bait, fake accounts, and propaganda, and someone is making money every time you engage with it. Renée DiResta, author of Invisible Rulers and a Georgetown researcher who has studied online manipulation for over a decade, breaks down how rage bait, propaganda, and AI-generated content end up in your social media feed, and why so much of it comes down to money. We get into how fake accounts pose as everyday Americans, how foreign bot networks try to hijack real debates, why fact-checking and Community Notes struggle to keep up with misinformation, and how to spot an influencer who profits from making you distrust everyone else. Renée also shares the questions she asks herself before reacting to anything that makes her mad, and how to take back some control over your algorithm.
Renee DiResta (00:05)
whenever you have an inauthentic
operation, whenever you have a propaganda campaign, an influence operation, there's like multiple components that a platform will look at when deciding if it is legitimate or not.
what a lot of people started doing was looking at some of these highly provocative, monetized MAGA influencers and saying.
shit, that guy's in Kenya. That guy's in Kenya. That guy's in
you know, Nigeria, this one's in Pakistan, this one's India, this one's Eastern Europe somewhere, right? I think in that moment it sort of like lifts the veil and shows you that some guy and like Lagos is the one pretending to be riled up about some school board in America. It's all bullshit, it's all manufactured.
Siara Singleton (00:44)
I think I've become harder to rage bait, which is very exciting.
Renee DiResta (00:48)
Yeah,
Siara Singleton (00:48)
So yeah.
Renee DiResta (00:48)
I well
that guru model is where they have all of the answers and you know, and they're
constantly eroding trust in other speakers, right? I am telling you this because the government doesn't want you to know. I am telling you this because like you can't trust the media. The media is lying to you. I am telling you the truth.
what you started to see with QAnon is it moves into this like cult-like conspiratorial, you know, model of if your relatives are telling you this isn't true, you have to stop talking to them because they're part of it, also.
Siara Singleton (01:18)
I have to admit I didn't know. I did not know that platform was being monetized in that way.
Hi Renee, welcome.
Renee DiResta (09:21)
Thanks for having me. It's great to be here.
Siara Singleton (09:23)
Pleasure to have you. you study pretty much everything that fascinates me.
propaganda, online manipulation, AI slop, all of the things that I have likely fallen for, by the way.
Renee DiResta (09:35)
All the best parts of the internet.
Siara Singleton (09:36)
yeah,
it's a wide array of everything wrong with the internet. but I was very curious. The first thing I want to ask is: like, what is it about online manipulation that continues to hold your attention?
Renee DiResta (09:48)
I think it's that it's there's a lot of similarities, right?
I I've been doing this for about almost twelve years now. So there's a lot of stuff that's the same where I'm like, yeah, I've seen that before, seen that before, seen that before. but there's all you know, there's so many different new and novel ways to use technology in terrible with terrible applications and so I I get to see all kinds of new and interesting stuff. I like seeing what state actors try to do. Spammers and scammers get very innovative. You know, there's th it's just a combination of
what I get to see and then the idea that, you know, it's having an impact that that helping people understand h
Siara Singleton (10:19)
Mm-hmm.
Renee DiResta (10:20)
how scams work or how they're being manipulated actually does you know, help them.
Siara Singleton (10:25)
Yeah. We what is can you define state actor for us?
Renee DiResta (10:29)
Yeah, sure. So so operations that are linked to a government in some way. So that might be the US government, that might be Iran, Israel, Russia of course is usually the most notorious, China quite a bit often is in the information manipulation game as well. So entities that are linked to governments and what's kinda cool about it is sometimes they don't do it directly. Right. So you used to see governments use their own intelligence services or their own militaries to
put out propaganda or to run influence operations, increasingly they'll actually hire out to mercenaries, right? They'll try to make other people who look like they are independent do the work of the state for it. So it's also got that kind of cloak and dagger element to it. You see elements of you know, the way Cold War spies used to start up front media properties. Now you just do it digitally.
Siara Singleton (11:16)
Mm. It's it's kind of like the political version of an industry plant. Is that fair? Okay.
Renee DiResta (11:20)
Yeah, exactly. Yes. Ast astroturf comes
up as like a term fairly often. It's just that it's linked to it's linked to a government in some way.
Siara Singleton (11:28)
Could I think we should define astroturfing 'cause I this is
Renee DiResta (11:30)
Uh-huh.
Siara Singleton (11:31)
a new term in my lexicon in the past couple of years, but then I'm also wondering, could astroturfing be used for good? Do you think?
Renee DiResta (11:38)
It's an interesting question.
So the astroturfing is when you're trying to create an impression of fake grassroots. If everybody I think maybe is familiar with seeing astroturf on a field at a a sports arena or something, you know that's not
Well, actually, I don't know in w in how many cases it is real grass. I don't go to many sporting events, but AstroTurf is the fake grass. so when you have AstroTurf in a political context, it means fake grassroots, people who are trying to look organic and aren't. they might be
Siara Singleton (12:07)
Mm-hmm.
Renee DiResta (12:07)
paid, they might be hired in some way, they might not even exist. You actually can have Astroturf now sometimes refers to completely fake people, fake accounts that are not representative of any real person in the world.
But they're trying to create a majority illusion. They're trying to make it look like a whole lot of people believe a certain thing or are taking a certain action. And that's where the term comes from. The question of if you can do it for good is a very, very sort of thorny ethical question. because it gets at one of two things. Either you believe that the end justifies the means, right? Ergo.
using fake people to create this impression if it is towards a noble end might still be a good thing. That's an argument that some people make. That's very much an eye of the beholder around what is the good thing that happens at the end there. The other argument is that it is inherently deceptive to make up people or to pay people to say a thing that they don't actually organically believe or want to say themselves.
or in the case of again, bots to simply fabricate a entity that n doesn't have any opinions actually. So that's that's the sort
Siara Singleton (13:14)
Yeah.
Renee DiResta (13:15)
of arguments on both sides of that debate.
Siara Singleton (13:19)
Is it always artificial figures that are involved in astroturfing? Or could you like I guess it's a question of what's the difference between a campaign and astroturfing? Because couldn't you find people who genuinely hold certain beliefs and somewhat artificially push them and promote them and amplify their voice or multiple voices?
Renee DiResta (13:39)
Mm-hmm. You can.
Yeah, artificial amplification is a thing. And you will see, sometimes I think one of the thorny ethical questions is when you are paying someone, are they still acting completely
Siara Singleton (13:51)
Yeah.
Renee DiResta (13:52)
independently? there are you know, you'll see these occasionally these small scandals will happen where people are paid. You know, this is this is actually more of a trope than a thing that happens in real life, and I just wanna kind of caveat that. But people who are like
paid to show up to a protest or people who are paid to go and appear to stand and you know
be photographed as being part of a crowd. A lot of people turned out for this thing. Actually arguing that people are paid is more often the thing that happens. Like pretending that you're trying to delegitimize a protest by saying the protesters are paid is often part of the you know a very common political narrative you'll see to delegitimize a real social movement.
So this question of do you ever want to pay people it to have that affiliated angle,
Siara Singleton (14:35)
Yeah.
Renee DiResta (14:35)
i it can have these sort of unintended consequences. And so I think most people decide that it is you know, most movements decide that it is just not worth going that route, that if they're gonna compensate their people in some way they're gonna do it for something else.
Siara Singleton (14:48)
Yeah. I think of like crisis actors, I think that's what they were calling it years ago. That was a big Yeah, he got in some trouble for that, didn't he?
Renee DiResta (14:54)
That was the Alex Jones term. Yes. Yeah, well,
because I mean he applied it in like egregiously horrible ways, to just to school shootings and things like that, right? So that
Siara Singleton (15:03)
Yeah. Yeah.
Renee DiResta (15:05)
the parents were crisis actors, the kids weren't real. as again, as I mentioned, to delegitimize,
Siara Singleton (15:08)
Yeah, that wasn't good.
Renee DiResta (15:10)
to delegitimize this very real atrocity that had happened. And so when you start accusing people of being paid, delegitimizing a real movement as being a
manufactured apparatus of a state or a party that is secretly funneling money to make them show up and look like it's just a real popular uprising. that allegation is a very serious allegation. It can create doubt and uncertainty in the minds of people who trust that person who's claiming that this was all manufactured. And so not creating that that appearance at any point at at any time that that is a that that is what is happening is I think important to to most
Siara Singleton (15:46)
Yeah.
Renee DiResta (15:46)
movements. Yeah.
Siara Singleton (15:48)
Yeah, and I think that's what's so hard about learning about this space is like you don't want
to go too far to the other side where like now you can't trust anything. because the purpose, of this show specifically is to help empower people so that they can, you know, detect when something is BS, but also still
Renee DiResta (16:02)
Yep.
Siara Singleton (16:03)
have the discernment to navigate in not this tin hat paranoid way. So I would like to focus specifically on
the bad actors with intentional purposes. you introduced me to the term adversarial abuse. Can you explain what that means in plain English and what that might look like? Yeah.
Renee DiResta (16:20)
Yeah.
So I worked at this place called Stanford Internet Observatory, and we started it in twenty nineteen, and it was right around the time when you were starting to see a whole lot of different ways in which the same technologies, the same platforms were being exploited by different bad actors with different motivations. So we've talked a little bit about state actors. You would also see spammers and scammers, which I mentioned, which oftentimes will put out
politically inflected content, right? They'll put out, right now we're seeing the data center conversation. We can talk about that and what's happening there. But you for example, the data center conversation, but they're not doing it because they have any skin in the game on either side. They don't actually care what happens with the data centers. What they care about is making money. And if they're paid per engagement or paid when they manage to manipulate somebody to take an action and maybe buy a product. you see spammers and scammers actually quite innovative when it comes to
finding ways to use technology to manipulate people. another area that we paid a lot of attention to was child safety. that term gets used to mean a lot of different things. Sometimes it's kids on the internet or screen time. When I use it, what we were looking at was child exploitation content. So we were very not literally looking at just to be clear, what we were concerned
Siara Singleton (17:38)
Yeah.
Renee DiResta (17:39)
about was child exploitation content.
what's sometimes called CSAM, particularly in the age of AI, you can AI generate images of of often children, non-consensual intimate imagery, and NCII refers to when it happens to people who are older, women, men sometimes. And so we would look at that aspect. How does AI when abused produce this type of content? and so what we did at at SIO was we we came up with the term adversarial abuse to refer to
Something where there is some kind of intentional abuse of the technology by someone who wants to make a person either take an action that might be harmful to them, encounter false or misleading or manipulative content, or again, in the case of something like sex tortion, cause a particular harm directly to a person. So those were the areas that we looked at, and we looked at how new technology transformed adversarial abuse.
So not just social media, but when you have the incorporation of the ability to create photorealistic images or video quite effortlessly, what does that do? How does that how does that change the world? So we looked at just the intersection of tech and how people wanted to manipulate others.
Siara Singleton (18:50)
And my understanding is it's not so much that bad actors are hacking or doing anything that the platform doesn't already allow, they're just more so exploiting the weaknesses of a platform. Is that okay?
Renee DiResta (18:59)
Exactly. Yes.
So you can use generative AI image technology to make very, very legitimate images of people or situations or scenarios. you can also use it in very, very terrible ways. So the problem is not necessarily the technology itself, it's the application and the there's a little metaphor that that
people have heard like you can use a hammer to build a house or as a weapon, right? It's a tool or a weapon. and it really is whose hands it goes into that determines which of those directions it goes.
Siara Singleton (19:28)
Mm-hmm.
I feel like defining words is always a really helpful way for people to learn things because they might
Renee DiResta (19:34)
Sure.
Siara Singleton (19:34)
be aware of something but they don't have a term for it. So I'm curious if there's like different forms of adversarial abuse that we can name and also which ones the common person might come about frequently, like even daily when they're online.
Renee DiResta (19:44)
Yeah. How do you
Yeah, you'd be surprised, I think, how often so spam and scams are very common. they're actually quite ubiquitous in a lot of places now. I open my Facebook and because of how the monetization incentives are on Facebook today, more than ever before in the last year and a half they changed their content mod monetization program. You'll just see fake accounts posting rage bait into groups, right?
Siara Singleton (20:13)
Yep.
Renee DiResta (20:14)
And it's not even political rage bait.
Sometimes I saw one last night that was don't you really hate when people don't clean up after their dogs, right? With like dozens of comments under it, you know, in a Walmart coupon group. It literally's just in a Walmart, like a Walmart coupon group, right? where ostensibly
Siara Singleton (20:24)
Like yeah Yeah.
Renee DiResta (20:31)
the purpose of the group is to like find deals. I spend a lot of time in these in these groups and spaces just looking at this account by the way clearly is not it is not real. It had an AI generated profile image.
when you kind of clicked back through, the profile was locked down, it had no visibility into what it was. I had been looking at the way in which you're starting to see these people use these very normal rage bait tactics in groups because they are monetized accounts and they hope that if they get enough hits and the Walmart group happened to be a public group so it could theoretically spread and be shared, that was going to be a a thing that would lead them to make some money.
In the meantime, they've gone and they've
Siara Singleton (21:09)
Mm-hmm.
Renee DiResta (21:10)
like riled up a group of people. Now, you know, everybody does get mad when people don't clean up after their dogs. This is a very small niche example. But the point is, because of the monetization structure of the platform, that kind of content is incentivized. Spammers, scammers who want to direct you to spend money will put up AI generated products that don't exist, right? I don't know if you see these in your feed. I see them in
Siara Singleton (21:33)
Yeah.
Renee DiResta (21:34)
Instagram has really cleaned it up a lot because I think a lot of people were getting scammed, but I got this you know, ad for Sheen, you know, Shine, I never know how pronounce that. where it was like a it
Siara Singleton (21:43)
Think Sheanne, I think.
Renee DiResta (21:44)
was this bizarre pillow of a goat. Hideous, absolutely hideous. I I posted it to social because it was so so like it was like so shockingly ugly that it had like hundreds of comments under it. I I'll send you I'll send this to you. You can
Siara Singleton (21:54)
I have to find a pick and and add it to the episode.
Renee DiResta (21:59)
But it was this this shockingly ugly goat. and it was clearly AI generated. Like it it was supposed to look like it had been meticulously embroidered in almost like a three D sort of way. But clearly anybody who goes and buys that is gonna be like you know, is gonna be scammed. It's not gonna look like that. It's
Siara Singleton (22:13)
Yeah.
Renee DiResta (22:14)
gonna be some flat print. So you would see a lot of these things when generative AI started where people were being manipulated into buying products that didn't exist or
renting Airbnb type places that didn't exist. So that's where you see just ways in which it comes up in in
Siara Singleton (22:27)
that sucks.
Renee DiResta (22:29)
in scams.
Siara Singleton (22:31)
I would be very
upset about an Airbnb scam. That's that's terrible. Yeah.
Renee DiResta (22:33)
I know. Well, you're spending a lot of money on that, right? Yeah,
spending a lot of money on that. If you're going to stay somewhere, you get s you know you take your kids or your, you know, your partner or whatever, you go somewhere and like the place doesn't exist. the state actor stuff is in the conversation. it is used, funny enough, by political entities to delegitimize movements or to rile up movements.
People are often not aware of how often they are being rage baited, again, for financial reasons, as we discussed, or because some political entity wants you to be angry about something. I can talk about
Siara Singleton (23:13)
Yeah.
Renee DiResta (23:14)
the data centers example because this actually spans both, and this has been in the news a lot in the last week and a half. because Elon Musk announced that Twitter had taken down a network of 200,000.
fake accounts, of which about 200 had been creating content. And this network was talking about the data center conversation, opposition to AI data centers. Now this is a real debate that is actually happening in the US. If you go on socials, you will see very real people making content about how they feel about data centers. they they will show up to their city council meetings and testify, right? They are real, they are showing up in person, they are very concerned about this. some are very supportive, some are not supportive. So you have
really opinions that run the gamut. But this Chinese network was boosting opposition to data centers. This serves the Chinese government's intentions in two ways. One, it creates discord in America. It's a common, you know, common, constant thing that you see adversaries do. The other thing is it it makes that position potentially look bigger, look like the dominant opinion. Everybody's opposed to data centers. Here's the
Here's evidence, here's 200,000 accounts
Siara Singleton (24:20)
Yeah.
Renee DiResta (24:21)
that are that are that are saying this. In reality, when you look at the the what the Chinese accounts actually accomplished though, and this is what Twitter didn't put out, what X didn't put out, they got almost no engagement. So they're in there. 200 accounts are posting, the other you know, 198,800 are amplifying them. but
The thing that is happening there is it is a
It's like a it's a manipulative effort to rile people up, but if they're not actually getting attention, then they're just sort of like tweeting into the void.
Siara Singleton (24:54)
Yeah, yeah.
Renee DiResta (24:56)
What's happening on the Bangladeshi side is that they are sorry, that I just like jumped the gun there. What's happening on Facebook is you're seeing paid scammers, often from Southeast Asia, in this particular case from Bangladesh.
That are running pages that look like they belong to a US state. So it's like what I love about living in California, right? except it's not being run by a person in California. It's being run
Siara Singleton (25:19)
Yeah.
Renee DiResta (25:20)
by a person overseas who is trying to capitalize on the meta monetization program. And so by putting out this content about data centers, they don't care. That person in Bangladesh who's trying to make this money doesn't actually care about the data center conversation at all.
They just care about posting content that will get a lot of engagement, that will rile people up, that will then let them earn money, right? Those pages actually get more engagement than the Chinese content that X announced its takedown of. And all of this is very small compared to the real opinions that real people have. So that's where you start to see that dynamic of the real opinion being kind of.
you know, throwing gasoline on the fire of real opinions, but either for profit or for for financial motivation. I hope I made that clear, though it's sort of a convoluted two part example.
Siara Singleton (26:11)
Yeah.
Well yeah, it's interesting because I think when I was originally thinking about adversarial abuse, I was assuming it would be like in that example, I would have assumed that you were headed to maybe potentially an AI company, one of the big AI giants, trying to change, the public opinion about data centers
Renee DiResta (26:28)
Well
Siara Singleton (26:28)
using adversarial abuse or something like that. But you're talking about people who they don't care. It's it's it's all the incentives money. It's not necessarily which
Renee DiResta (26:35)
The incentive
is money
Siara Singleton (26:36)
the incentive was
money and the AI example too, but
Renee DiResta (26:39)
Well the thing that was interesting about that was X announced this takedown of these two hundred thousand accounts the last week of August, first week of September. It was very recent. Open AI had actually announced that this network was operating using OpenAI's Chat GPT services, or their API maybe it was.
earlier. They had announced it back in July that this was the thing that was happening. And so it was the same network, the same content. Open AI just announced it months before X did and actually was very transparent about the fact that it was happening and that it didn't seem to have gotten any engagement. So OpenAI, to its credit, did disclose that last part as well, where X put out no information that would help the public understand how prevalent this had actually been. And I've been trying to remember, I think I said
Eighteen thousand eight hundred was so it's it's basically like two hundred accounts of the batch were actually active and producing content.
Siara Singleton (27:31)
Okay. And then I'm curious where you think ownership lies. Because I know like with the the nature of this type of abuse, it's gonna be ever changing and it might be a little hard to keep up with, but where do you think the onus lies? Because it feels like there's things Twitter could do to combat this. It also feels like there's things
OpenAI could have done to combat this in terms of blocking certain prompts. Like if you think that a prompt is going to be used for some sort of bot amplification, couldn't they on their side try to limit something like that from happening with their models?
Renee DiResta (28:04)
So
yes. So they do actually try. Anthropic put out an announcement just yesterday, or maybe it was two days ago now, but in the last week, announcing several other state actor takedowns, right? So well, let me let me let me rephrase that. Anthropic and the AI companies can't take content down. They are not responsible for the distribution of content.
What you see the state actors doing is using them for the creation of content, right? Or the creation of malware or the creation of code that they can put to use in some way. So audience persona creation, things like that. So what they are doing over the AI companies is they're constantly looking at are our services being used to create in a manipulative way? The social media companies
Siara Singleton (28:49)
Mm-hmm.
Renee DiResta (28:50)
are looking at the distribution. Are our services being used to
push this content out so that audiences see it? Are they using us as the sort of drop site, the host for this content that then they're going to try to reach real users on our platforms to manipulate them? So that's the difference in how these these two like sides of the same coin work, right? So in the manipulation
Siara Singleton (29:11)
Yeah.
Renee DiResta (29:12)
campaign you have to create the content, then you have to distribute it and make people
see it. So the AI companies have been quite proactive at disclosing
when those manipulative creation efforts happen. we've seen them point at
Siara Singleton (29:25)
Okay.
Renee DiResta (29:26)
a number of nation states. Russia shows up in the most recent anthropic one, where they're saying like, hey, these state linked entities are trying to make manipulative content. Then you have to go and look at does anybody see it? That's the second question. Because it's not enough, you know, you can I can make content on my, you know, my Claude account all day long, and it doesn't mean everyone any of them see it if I do a bad job of putting it out there.
So it's still that social
Siara Singleton (29:49)
Yeah.
Renee DiResta (29:49)
game. That social game still matters. And what you've seen on that side is a walk back from transparency. Even in 2023, the US Congress began launching investigations into tech companies, alleging that their content moderation and takedown policies, when they said they were taking down inauthentic accounts.
They really were secretly trying to censor political viewpoints. This became a very politicized conversation. And the platforms used to put out very transparent data sets. They used to work with researchers, including me, to do analysis of these state after takedowns, and all of that stopped. All of that came to a halt in in 2023, 2024 timeframe. So what X put out, just one tweet length.
statement that said we took down two hundred thousand accounts that really doesn't communicate very much to people. You can't go look at it. You can't go see what were those accounts doing. You don't you have to kind of just trust Elon, trust X, you know, trust me bro, this is what they were actually doing. No, you used to be able to see because independent researchers would go and write something up looking at those data sets and that doesn't happen anymore 'cause that all got reframed as like censorship and collusion between academics and platforms to like influence political viewpoints.
By describing what was in the takedowns, and that's because sometimes the takedowns would involve America, you know, political entities like Turning Point USA or others that were willing to do what we kind of talked about earlier,
Siara Singleton (31:14)
Yeah.
Renee DiResta (31:15)
which is does the end justify the means if you want to astroturf a political conversation domestically, what should the platform do then? Is it just
inauthenticity that matters or should they only be actioning against foreign adversaries targeting a domestic conversation?
Siara Singleton (31:32)
Yeah, a and then I guess that's where community notes comes into question of if that works or not. Because we're
Renee DiResta (31:37)
So
Siara Singleton (31:38)
using community notes on X and Meta, correct? Meta whatever
Renee DiResta (31:40)
Yes.
Siara Singleton (31:42)
app yeah.
Renee DiResta (31:43)
yeah, Threads and Instagram both have them. I don't know if it's on blue yet. but it's on it's on Threads
Siara Singleton (31:49)
Okay.
Renee DiResta (31:49)
and Instagram. So I I'm a pretty active participant in community notes. I have been since twenty twenty two.
when it was called Bird Watch when X first rolled it out. That's a little bit different. So whenever you have an inauthentic
Siara Singleton (32:02)
Okay.
Renee DiResta (32:04)
operation, whenever you have a p a propaganda campaign, an influence operation, there's again, there's like kind of multiple components that a platform will look at when deciding if it is legitimate or not. And the substance of the message, like the content itself and whether it is true or false, is often not the thing that is making that decision. That is
Separate. We use this little mnemonic
Siara Singleton (32:26)
Yeah.
Renee DiResta (32:27)
ABC, actors, behaviors, and content, right? Those are the three things that we're looking at. So when you're looking at actors, you're looking at is the account real? Is it who it claims to be? Is it impersonation? Is it straight up fake? Is it a Russian troll pretending to be an American, right? That's where you get at these notions of authenticity. Is the actor real?
real? Is it are they who they claim to be. That's not to say that an anonymous account is inherently manipulative or bad. It's not at all. Actually a lot of people, particularly dissidents, use an use anonymity to be able to speak authentically but just
Siara Singleton (33:03)
Yeah.
Renee DiResta (33:03)
in an anonymous capacity so that they don't get, you know, disappeared by a government or something. So when you're looking at actors though, you're looking at are they what they say they are? If they're telling you they're a Black Lives Matter activist, are they? Or are they a Russian troll? That was a very
popular common example in 2016. The behavior
Siara Singleton (33:21)
Yeah.
Renee DiResta (33:21)
piece is are they out there like trying to game an algorithm? So you know this, you're a content creator, when you put something out there, you're not guaranteed exposure. You can put something out and a small group of people will see it. If you want to reach a larger group of people, you need that engagement, right? So inauthentic
Siara Singleton (33:40)
Mm.
Renee DiResta (33:40)
behavior might be like using an SMM panel to get a whole lot of fake likes in hopes that you can push your content out.
It might be using, again, 200,000 fake accounts where only 200 are talking, and all the rest of them are out there doing just hitting the retweet button, right? Retweet, quote tweet, just trying to do something to call attention to the 200 posters. And they're also hitting that like button, trying to make it look like it's got real engagement when it doesn't. So that's where the sort of behavior
Siara Singleton (34:06)
Yeah.
Renee DiResta (34:06)
piece comes in. And then content really is the last thing. And whether they're saying something true or false,
That is actually usually much less important than how they're sort how they're framing it, how they're pretending like how they're engaging the the community. But we do look at it mostly to understand what was the intent of this operation, right? If the Chinese accounts are talking about
Siara Singleton (34:27)
Yeah.
Renee DiResta (34:28)
data centers, if the Russian accounts were talking about BLM, do they have some ulterior motive? Or are they, you know, do they want those movements to go in a particular direction that helps them?
Or are they just trying to play both sides against each other and rile up Americans and cause strife? So that's how
Siara Singleton (34:43)
Yeah.
Renee DiResta (34:44)
that actually works. When we talk about community notes, that's actually just straight up fact checking or adding context. And so that is even in the realm of completely authentic people saying it with their whole chest, putting it out there on Twitter, and then somebody else says, like, actually, you know what, that's not true. And that's what a community note is for.
So it's not gonna it's not
Siara Singleton (35:05)
Okay.
Renee DiResta (35:05)
advocating that it be taken down. It's not saying it's not authentic. It's just saying like this piece of content, regardless of who said it, regardless of what the behaviors of amplification are, this claim isn't true. And it's a very narrow solution for that one problem. We can talk about how well it works if you want, but
Siara Singleton (35:23)
Yeah. Yeah.
Well, I'm I'm curious, like, 'cause I know there's a big difference between misinformation and disinformation. I feel like misinformation
Renee DiResta (35:31)
Mm-hmm.
Siara Singleton (35:31)
happens all the time and it can be an accident, whereas
Renee DiResta (35:33)
Yeah.
Siara Singleton (35:34)
obviously disinformation's intentional. But I did hear you say that those two terms can be misdiagnoses of the problem and to focus on propaganda. So I'd love to hear more about your view on that and like how those terms miss the mark for the rest of us to like really understand what's going on.
Renee DiResta (35:50)
Sure. Yeah, so
disinformation is sort of a lot of the things that I've been describing when I say I I I tend to use the term influence operation a little bit more because it is a broader.
Disinformation campaign refers to something that is intentionally false and manipulative, or I should just say manipulative, it doesn't necessarily have to be false. Sometimes it can just be pushing out political opinion, but it is often false. And what it is trying to do is again, entities that aren't what they seem are out there attempting to influence a conversation with information that is highly inflected and often false.
Or manipulative in some way. Decontextualized facts can also be used in disinformation campaigns where it is technically true but used in a completely incorrect or different context to create a false impression. So that's that's the term disinformation. It comes out of sort of a Soviet term because it was a strategy in the Cold War to actually produce quite a lot of this. And they would use this was long before social media.
So they would use what we call front media properties, meaning you would create a news outlet that looked like it was independent when in reality it was bankrolled by or actually run by something like an intelligence service
Siara Singleton (37:07)
Okay.
Renee DiResta (37:07)
or a government. So it would look like an independent media voice telling you a thing. In reality, it is linked to the state surreptitiously behind the scenes. So disinformation falls in that 30 milieu.
Misinformation just refers to things that are, you know, false or misleading or just wrong. so and they're often accidentally false, misleading or wrong. So, you know, this is where you get at just claims maybe about scientific things, vaccines is very common, claims about you know, what is the actual truth of the climate conversation? Where where where are the actual facts?
Sometimes people are just wrong, right? They've heard something, they sort of repeat
Siara Singleton (37:48)
Yeah.
Renee DiResta (37:48)
it, and that's a very common, very common thing that that happens. The reason I have written since about 2020, so a little over six years now, about why I don't like these terms is that they really require you know you're you spend people spend a lot of time sitting there trying to figure out is this fact true? And
That's not actually what matters to the people who are sharing it, right? That fact is secondary to what it signals or the belief that it gives them about maybe the let's stick with something in public health. if the C D C has been telling you that vaccines don't cause autism, but they do, the real
claim there is not about the vaccines and autism. It's not the false claim about the vaccines and autism. It's the reflection of distrust in the government. That's what you're actually hearing when people are spreading that
Siara Singleton (38:41)
Yeah.
Renee DiResta (38:41)
particular false and misleading claim. And so when you treat it as a problem of a fact, no no no if if if if everybody heard that vaccines don't cause autism, that belief would go away. That's that
Siara Singleton (38:52)
It'd be fine, yeah.
Renee DiResta (38:53)
right, that's not how the psychology works. That's not how people come to form beliefs. That's not why people share this content.
A lot of the time when you see people who are sharing something that you know to be false and misleading, they're doing it because they genuinely believe that it's true and they believe they're helping their community by telling them that it's true, right? That they are resisting a tyrannical
Siara Singleton (39:13)
Yeah.
Renee DiResta (39:14)
government or they're resisting, you know, the man, the media, whatever it is, but whoever the villain is in the story. that is an act of trying to convey distrust in
the establishment, the apparatus, the experts, whatever they are. and that's what you're really seeing there. It's n you this this happens. I write a lot about election integrity. It's a big area that I work in. When people are saying the election is rigged, they're again they're not saying, now that you've shown me the voter roles and the paper that went through that one machine, now I know that that machine was good and that election was fair. No, that's not
They'll they'll come because the the goalposts will move. It'll be well the people who did the investigation were really the liars, right? The l the investigation was rigged. And so what you're hearing a lot of the time is actually a reflection of distrust in government. But it got treated a lot as a problem of facts. If only we fact checked this, everybody would know. And that's the real challenge, I think, when you talk about things in terms of misinformation.
Sometimes it is that simple. You know, we've all seen people on the internet say, Whoops, I got it wrong. It happens very rarely. People get very excited when they see it, right? when someone says,
Siara Singleton (40:18)
Rare. Yeah.
Renee DiResta (40:20)
I got that wrong. Actually, here you're yeah, it's very rare because people get very invested. I shared this thing. I'll look stupid if it turns out not to be true. So let me find some justification
Siara Singleton (40:29)
Yes.
Renee DiResta (40:30)
for why it really is true, right? And that's a thing that you see where it's a
It's a natural human
Siara Singleton (40:38)
It's painful to watch.
Renee DiResta (40:39)
psychological thing. Yeah, it is. It is. But it's but that's that's what's usually happening there. So when there was a lot of efforts by platforms to address misinformation, a lot of the time they took the form of fact checking. And what you started to see, and you know, I I always hate sounding partisan, but you just do sometimes in this job, what you started to see on the political
Siara Singleton (40:59)
Yeah.
Renee DiResta (41:00)
right was a real effort to delegitimize fact checking.
Because when there were claims that had been made that were so demonstrably false, for example, about the 2020 election, saying, well, the fact checkers are lying to you was a way to allow their audience, their base to continue to hold that belief. Because if the fact checkers are lying to you, then what politician X said to you is still true,
Siara Singleton (41:26)
Yeah.
Renee DiResta (41:26)
right? You weren't you weren't misled. That
trust, you know, you had trust in that person. That wasn't broken when they lied to you. Instead it's really those fact checkers over there. They're the ones who are lying to you. You should trust me. And that's the
Siara Singleton (41:40)
Yeah.
Renee DiResta (41:41)
that's the dynamic that you see playing out. So the reason community notes came about was actually because of this phenomenon where people would simply dismiss the fact check. You know, of course Snopes is lying when they say politician X did this thing. He never would have done that, you know? and the
idea is that community correction, meaning like me going and writing a note, though this is how community notes works for those who don't know. I can write a note and it goes into a queue and all these other people who are participants in the program, it's all anonymous by the way, you know you never see who wrote the note or who's rating the note, but all these raters come in.
Siara Singleton (42:16)
okay, good.
Renee DiResta (42:17)
It's it's it's anonymized, I should say, like it is linked to a persistent account. So every note that I write shows up as the same note writer.
and all the notes that people rate show up as the same rater. but that's more for understanding
Siara Singleton (42:32)
Mm-hmm.
Renee DiResta (42:32)
what I'm about to explain,
So I'll I'll just explain community notes then. the way that community notes works is it allows an ordinary person like me or you to write a note and then all of the other people who participate in the program and anyone can join go and rate the note. And if a sufficient number of people who are seen as being seen by the algorithm as being frequently politically divergent, meaning
They often rate notes in in completely different ways, right? They they t they they indicate, they signal
Siara Singleton (43:02)
Yeah.
Renee DiResta (43:03)
to the ranking algorithm, the what's called a bridging based ranking algorithm, that they are generally speaking on opposite sides of an issue frequently. But on this note they agree
That's what ultimately makes that note get seen as high quality, and then that's what gets it appended to a post. So that's how that process works. It's not that it knows that you're a Democrat or Republican, it's not out there reading your tweets. It's just that it looks at how you typically vote. And when people who typically vote in different directions all of a sudden are agreeing on a note, that's seen as like consensus and it's going to show up and the
Siara Singleton (43:36)
Okay.
Renee DiResta (43:37)
public is going to see it. And that's how it works.
Siara Singleton (43:41)
Yeah.
Renee DiResta (43:42)
It's a nice idea.
There are a lot of problems with it. There were the problems
Siara Singleton (43:46)
Mm-hmm.
Renee DiResta (43:47)
with professional fact checking was more like scale, right? It's very hard for a fact checker to respond to all of the false and misleading nonsense on the internet at any given point. which means that they decide what they want to rate. And the argument is that the fact checker is biased in its determination. Maybe more the argument that you see a lot is that, well, fact checkers are only rating
right wing politicians, they're not rating left wing politicians. And so that that got very politicized. the fact checkers are all lefties. They're funded by lefty organizations like George Soros, you know. And then that's the argument that was made
Siara Singleton (44:20)
Mm-hmm.
Renee DiResta (44:21)
to delegitimize the fact checkers. But this entity, this community notes project, allows for anybody to contribute and everybody to vote. And so it's seen as being more democratic. The problem is
Most notes that get submitted never even get one rating. It because when you think about what you're doing on social media,
Siara Singleton (44:41)
Okay.
Renee DiResta (44:41)
right? When I open my Threads account, do I wanna go write notes and rate notes for Mark Zuckerberg? Like, is that the best use of my time, my limited free time? Do I wanna be on a platform on Instagram and instead of scrolling and enjoying myself and
Siara Singleton (44:53)
No
Renee DiResta (44:54)
watching content, I'm like, no, I'm gonna go write some community notes for the billionaire who doesn't want to pay for that service.
Siara Singleton (44:59)
Yeah. I'm
gonna edit, yeah.
Renee DiResta (45:01)
So it's not incentivized.
Yeah, you I mean So all of a sudden it's
Siara Singleton (45:05)
Yeah, that's a really good point.
Renee DiResta (45:07)
like it's it's just a perverse incentive. Like, look, I do it because I'm a researcher. This is my job, right? I want to understand where these programs can work. I would love it to work. Nothing would make me happier than seeing Mark Zuckerberg decide that instead of paying the meta-content program paying the scammers in Bangladesh making the BS content that I am then asked to write a note about.
Pay me for writing the note, take that money away from the content creator that produced the bullshit and give it to the note writers that are actually sitting
Siara Singleton (45:36)
Yeah.
Renee DiResta (45:36)
there serving as your de facto fact checking army because you've decided not to pay fact checkers anymore. That's how I see it.
Siara Singleton (45:42)
Mm-hmm. Yeah, and it's like
if you're getting no clout for reviewing the note, no one's gonna do it. Cause I feel like if you got like some sort of clout, right? Like if you got some sort of like
Renee DiResta (45:53)
Yeah.
Siara Singleton (45:54)
thank you and and recognition, people would be people would be reading notes here left and right, but because it's anonymous Yeah.
Renee DiResta (45:59)
Yeah, there's no there's no leaderboard. There's no leaderboard. It's just it's absolutely thankless. Like Twitter
Twitter's is a little bit more
Siara Singleton (46:07)
Yeah.
Renee DiResta (46:08)
sophisticated. Also the other thing is that notes that show up on Twitter are often around like viral posts or viral moments where the whole platform is paying attention to a thing. Facebook is so fragmented, right? Whatever you know, and it's so personalized and and
Siara Singleton (46:21)
It is
Renee DiResta (46:22)
my threads algorithm and your threads algorithm are completely different. And so the stuff that I see
You know, you're you're kind of like, what is the incentive that I have to do this? How many people are really gonna see this post or really gonna see this correction? And so the actually the trajectory that meta has gone is AI is writing the notes, right? So Meta AI generates the note because they're having such a hard time getting people to write them. So meta AI generates the notes and then people like me can go and rate them.
And you'll see this, it'll usually happen though, for things where like people are really, really passionate about it. And it's hard sometimes for the bridging to work because when it's on a political post, everybody who disagrees with the post will go hit that rating button, right? If if they're so motivated, they'll be motivated by the desire to like own the other side or see their side's information get corrected. You're not gonna see that same energy from
the side that put out the false or misleading content, they're not gonna go and say, like, yes, it's absolutely true, this is false. You know, they're not gonna do that. So you have these notes that get written
Siara Singleton (47:27)
Yeah.
Renee DiResta (47:28)
even by an AI that never show up because the public can't agree. It can't come to consensus. And that is the reason why I spend a lot of time participating in the program is actually because I'm very curious, like what happens when when Americans are so hopelessly divided
That even things where you can literally go to a website and look at the number of ballots released on some day, it is like it is right there. And people will still not vote to say that that statistic is true because they've just decided, well, the election officials must have rigged it, and so the ballots themselves are fake. And so
Siara Singleton (48:04)
Yeah.
Renee DiResta (48:05)
you're never gonna hit that consensus, and that's the problem. That's really sort of where we find ourselves today.
Siara Singleton (48:10)
Mhm. It's so destabilizing
and it's frustrating and I feel like I cling to the idea that facts exist. I cling to like science and math
Renee DiResta (48:19)
Ha ha ha.
Siara Singleton (48:20)
'cause I'm like this is the only way that I'm staying like sane in this world. Like I can't imagine going through the world being like there's no information source that I can trust. So I I have empathy for people.
Also, I'm probably extremely receptive to it myself. something that came up because you earlier you were talking about how there's so much focus on if the fact is true or not, but it's often about influencing to the audience. This is what lots of people believe. Even if you don't believe it, lots of people believe it, And
Renee DiResta (48:49)
Right.
Siara Singleton (48:50)
that was very pivotal for me to understand, because and I'm gonna say this because I
I actually don't think that it is partisan. I think this is bipartisan and that's kind of the whole point. But I think there was a campaign going on about like should women vote? And it's like, this is like one of those things like, come
Renee DiResta (49:06)
Yes.
Siara Singleton (49:09)
on. conservative friends, liberal friends, nobody nobody thinks women shouldn't vote. Like this is a joke. And that one was so
egregious that I was like, this has to be, fake. And so learning about propaganda gave me some sense of like almost not peace, but understanding of okay, that's why this happens sometimes. And I think if more people could understand that, they could look at something, absurd and say, I know what this is. And
I want people to be able to do that. So I'm curious if you have, I guess,
Renee DiResta (49:39)
Totally.
Siara Singleton (49:40)
advice for people of like how can you, when you look at the internet and you see something that's it's giving you that initial feeling of rage or fear or absurdity, how can you look at something and say this is most likely a form of adversarial abuse or artificial in some manner?
Renee DiResta (49:55)
Yeah, so a couple of ways. First, it's not necessarily artificial, right? Like there are people who are just sort of like trolls and provocateurs and the reason they're doing it is they make money, right? That that's the thing that I think people now I am 45, right? I've been on the internet for a very long time and the
shift to content being
Monetized is something that I think particularly older generations, since it wasn't how it always was, are not thinking in those terms when they see stuff on the internet. Whereas my son, who is 12, will tell you that it's all bullshit because people are just trying to make a buck, right? That's that's how he sees like what's happening on YouTube. There is there is no moment when he sees content where he's not thinking about like, somebody's making money on this, right? This is a job that they're earning a living. It's not
Siara Singleton (50:45)
Good.
Renee DiResta (50:45)
that it's bad that they're making money, he's just thinking of it like.
This is how content works, right? He doesn't see it as
Siara Singleton (50:50)
Yeah.
Renee DiResta (50:51)
information. He sees it as content. Whereas people who are older, and I would say maybe like I don't know, maybe 3540 is the cutoff where you remember a time when social media wasn't monetized for the users quite so much, for the creators, like that really kind
Siara Singleton (51:04)
Mm-hmm.
Renee DiResta (51:05)
of came later. they really do see it as just like people are just putting out information, they're just posting. When you look at something like X, where monetization happened within the last
three years or so. I'm a little bit tired today. My my exact recall's off. But in the last three years or so, maybe Elon bought it in 2022. I think monetized it a year later. when it became monetization around engagement, you really sh you started to see very different types of posts go viral. Because all of a sudden it didn't matter if you were right or not. If you were first, right? If you were first when something happened, you became the authoritative tweet that everybody would see because you would get a lot of engagement.
And then the the algorithm would push it out. And making people understand that that is happening, like that content creation there is a function of the fact that they're getting paid. That it is so much a function of incentives, like why people choose to do this. I don't know if you remember, I don't know if you're ever on X, but they did this
Siara Singleton (51:58)
It was.
Renee DiResta (51:58)
interesting thing where they they brief like actually it might still be turned on. This one might be still turned on, Nikita Beer did it.
and he's since left, but he did some really interesting, I think quite worthwhile stuff where he put out he made he made transparent where the accounts were coming from. Do you remember this? So you could go and you could
Siara Singleton (52:15)
No.
Renee DiResta (52:16)
click into a profile and you could see what country that account was registered to. So what they found that day, what a lot of people started doing was looking at some of these highly provocative, monetized MAGA influencers and saying.
Siara Singleton (52:31)
Yeah,
Renee DiResta (52:31)
shit, that guy's in Kenya. That guy's in Kenya. That guy's in
Siara Singleton (52:32)
I do remember now. Yes.
Renee DiResta (52:35)
you know, Nigeria, that guy's in China, this one's in Pac not China, this one's in Pakistan, this one's India, this one's Eastern Europe somewhere, right? and it became I think in that moment it sort of like lifts the veil and shows you that some guy and like Lagos is the one pretending to be riled up about you know, some school board in America. It's all bullshit, it's all manufactured.
Siara Singleton (52:55)
Yeah.
Renee DiResta (52:56)
And that's where you get at
I think they then tried to argue that monetization should be tied to the region where you live, right? That that this would be a way to mitigate that. And Elon rolled that back for sure. That was sort of like they tried to do it and then Elon walked it back. So the questions again of like, what does the platform need? It needs engaging content. What does the creator need? They need to earn engagement, even if they're pretending to be someone they're not.
What does the person see? Well, they just see content that seems enraging and, you know, it's it's not even that it's inauthentic in the sense that it's some, you know, Russian troll trying to influence American political opinions. No, it's just some guy trying to make a buck, knowing that the way that engagement works, the more comments he gets from certain types of you know, from accounts, you saying like, Man, that really sucks. I can't believe they're doing that, that's a you know, another penny or whatever. So that that's the problem.
Siara Singleton (53:49)
Yeah.
I have to admit I didn't know. I did not know that platform was being monetized in that way. I think sometimes
Renee DiResta (53:57)
Yeah.
Siara Singleton (53:58)
when X news happens, my brain just goes 'cause I'm just like, that stresses me out. I stopped using it many years ago. I was a big Twitter girl back in the day, but yeah, so I didn't know that. That sh makes so much sense now to me why X is the way that it is now. And I guess TikTok, for instance.
is similar and that you get monetized based on how much reach and obviously more engagement means more reach. But is that unique to is that unique to TikTok and X or is it the same for Instagram and
Renee DiResta (54:29)
No, it's
it really varies and X has changed it so many times at this point that I don't I don't know what the the like the absolute most current iteration is. Sometimes it's like engagements, meaning they have to comment. Sometimes it's like some subset of the audience, like in this case it was the paid ones, right? The blue checks had to comment. a lot of the others will do based on like view count or something. You have to watch for
Siara Singleton (54:54)
Yeah.
Renee DiResta (54:54)
a period of time.
Siara Singleton (54:57)
Right, with the the YouTube.
Renee DiResta (54:58)
yeah, but there's interesting ways. So I talk a lot to like science communicators and people, 'cause one of the things I write a lot about, I just put out a report yesterday, is how do you get like researchers or scholars or people who have some deep expertise in something to be participants in these conversations? We can point to the vaccine one, right? It was all like wellness influencers, mom influencers, like
Where are the doctors? You know, now they're there. Af
Siara Singleton (55:22)
Yeah.
Renee DiResta (55:23)
after COVID, now they're there because they saw what happened during COVID. They saw what happened and how incompetent the CDC
Siara Singleton (55:29)
Yeah.
Renee DiResta (55:30)
was at communication. So now you see the frontline physicians, the frontline pediatricians out there saying, please get your kid a measles vaccine. Here I am, I'm on camera, I'm talking to you just as me. Like, ask ask me questions in the comments. Let's talk about this. That didn't happen.
prior to COVID, right? That was this was something that I wrote about for years as a mom with three kids who's absolutely passionately pro-vaccine, saying like, come on guys, like you you're ceding the ground to these people who have massive audiences.
Siara Singleton (55:57)
Yeah.
Renee DiResta (55:58)
They're shaping public opinion and like you're not even in it. Like where are you? So I write Yeah, so they're busy, that's the thing, I know.
Siara Singleton (56:02)
Where are you? They're busy. They're in the hospital.
Renee DiResta (56:06)
That's that's the answer. That's exactly the problem, right?
Siara Singleton (56:08)
Ha ha.
Renee DiResta (56:08)
Where, you know, as much as the where are you is really more for the institutions, like where is CDC comms
Siara Singleton (56:14)
Yeah.
Renee DiResta (56:14)
on this? Come on, like because it it became the frontline doctors during COVID who would do a shift and get off and talk about the morgue trucks, right? To push back against this COVID isn't real BS that started in early twenty twenty. And so you had these frontline physicians who were incredibly compelling because they were telling it from a first person point of view.
You had people like Dr. Mike, right, who really started putting himself
Siara Singleton (56:37)
Yeah.
Renee DiResta (56:38)
out there sitting down doing debates. His clips show up all over socials now because he's so popular and prominent. But it took him years to kind of like get to that point. And also when you do that stuff, a lot of the time your institutional community looks down on it, right? That's not real scholarship, that's not real communication, that's just social media. And so what I write a lot about, and I put up this big white paper on Carnegie, which is a think tank here in DC.
yesterday with my colleague Rachel Kleinfeld and Margot Treadwell. And we actually wrote about like here is actually how you do it, right? Here is how you engage on social. Not like you put out one post and you hope for the best or you throw your PDF up and you hope for the best. No, you've got to be out there. You have to be talking to people. you should
Siara Singleton (57:22)
Yeah, consistent.
Renee DiResta (57:23)
be doing podcasts. You should be like making yourself accessible. Yeah. So
That for me has been I feel like I just went off on this tangent for you, but that for me has been a thing where I'm like,
Siara Singleton (57:31)
No, it's gone.
Renee DiResta (57:32)
how do you get good information out there? Like you have to be competing as like crappy as the incentives are, as much as we should be arguing that the incentives should be different and the algorithms should be different, letting like encouraging people to go and participate, I think, is actually very important because otherwise you have just seeded the ground to
you know, whoever wants
Siara Singleton (57:52)
Yeah.
Renee DiResta (57:52)
to to monetize and to sell you their detox or whatever and that becomes part of the problem.
Siara Singleton (57:58)
Yeah. And I'm definitely plugging your conversation with Dr. Mike because d I am so what I'll say about myself is I'm a very disloyal
Audience member for content creators, it takes me years to feel trust for someone because I'm like terrified of grifters. Lots of people call themselves doctors, lots of people call themselves therapists, and it
Renee DiResta (58:16)
Yeah.
Siara Singleton (58:17)
you I've been burned what especially when I was younger, where I was like, that who is this this guy or this girl? I will say Dr. Mike is one that has stood true that I'm like, okay, but that's because I'm looking for those patterns
the signs of grifting. And so I would love your opinion
Renee DiResta (58:32)
Yeah.
Siara Singleton (58:34)
for like how can we look at an influencer and say, are you legit or not? Because it's really hard. A lot of now we are actually entering the age where people are going and they're getting certain certifications so that they can be influential in certain industries. And it's like quite scary because people make very important
Renee DiResta (58:50)
Yeah. It's
Siara Singleton (58:53)
life decisions based on these people.
Renee DiResta (58:55)
Totally. So it's an interesting question. there is this this sense of like, are you being pushed to pay for something? Right? Is there a a strong profit motive where they're like, my supplement will solve this problem for you? My you know, in my book I call them like the gurus, right? The the
Siara Singleton (59:12)
Yeah.
Renee DiResta (59:13)
people who position themselves as a guru, meaning,
This is how you have to live your life. I am gonna give you all of the answers. And if you just follow me and do what I say, your life is gonna improve exponentially. You're gonna be healthier, you're gonna be prettier, you're gonna be whatever it is. that guru model is where they have all of the answers and you know, and they're communicating them to you, and you're gonna buy this thing to make that happen. You're gonna sign up for my webinar pay for my course to make that happen. You're gonna, you know, and then the other thing that you see with that, because again, one of the lines is like, some people really are.
trying to earn a living and they do it through courses and things like that. But one of the things that you'll see is also kind of constantly eroding trust in other speakers, right? I am telling you this because the CDC is keeping it from you. I am telling you this because the government doesn't want you to know. I am telling you this because like you can't trust the media. The media is lying to you. I am telling you the truth. You know, and that's where you see like when you look at that rhetoric, you start to see it happen and there are certain accounts that are just constantly out there.
where they are really actively profiting off of making you distrust things that are not them. And I wrote this article a really long time ago, like 2018, talking about the rise of QAnon, right? And the
Siara Singleton (1:00:27)
Mm-hmm.
Renee DiResta (1:00:28)
the way in which that because people are often
lonely in real life or they don't know who to trust or you know you have a baby and all of a sudden you're up all hours of the night trying to figure out why your baby won't sleep. It's very hard. You're not gonna bother your real life friends with that. Maybe they don't even have kids. So you go to the internet and that and people are in those vulnerable mindsets and that's where it becomes really easy to feel like, I found community.
I found camaraderie. But what you started to see with QAnon is it moves into this like cult-like conspiratorial, you know, QAnon started as a conspiracy theory, but it moves into this very cult-like model of if your relatives are telling you this isn't true, you have to stop talking to them because they're part of it, also. Right. And you start to see that very kind of cult-like
Siara Singleton (1:01:13)
Yeah.
Renee DiResta (1:01:14)
model take shape around what was an online conspiracy theory that really starts to become a cult.
So this is where that you know you see very few things like QAnon now. That was this real kind of twenty eighteen to twenty twenty twenty twenty one kind of time. but that's in part because a lot of those techniques, a lot of that guru type behavior moves into so many other fields and so many other types of
influencers start trying to solve everybody's problems with this one magic pill that you can take. There's a great pod
Siara Singleton (1:01:45)
Yeah.
Renee DiResta (1:01:46)
called Decoding the Gurus. They've been at this for a while where they actually will go deep on specific creators that have these tendencies and
they'll break down for you like here's how content creator X or influential celebrity X is doing this thing and why this sort of like checks the boxes of their sort of like guru meter that they have these criteria. And it's actually
I've gotten outreach from people in my own like I have a web form on my site and I get all sorts of mail. and I've gotten inbound from people who were like who referenced that pod in particular as the thing that made them see for the first time, like
Siara Singleton (1:02:22)
Mm.
Renee DiResta (1:02:23)
just how predatory some of this behavior was. I've gotten maybe three or four messages from people referencing that one specifically, and I like it a lot for that reason. I think it
I think they're really doing a service by explaining this stuff in a very empathetic and accessible way.
Siara Singleton (1:02:38)
Yeah. I need to check that out. I think I don't know if this is fair, but one of my like red flags is when there's big expensive conferences or events centered around one individual. That feels weird. do you know what I mean? Like it's like I'm trying to think of a really good example here. I don't wanna I don't necessarily wanna call someone out, but
It's something like they could put the information out, you know, on online. But instead
Renee DiResta (1:03:03)
Yeah, totally. Yeah.
Siara Singleton (1:03:06)
we're we're gonna charge them 5K, we're gonna make them travel and they're gonna come to me and talk to me in person.
Renee DiResta (1:03:11)
Yeah, yeah, yeah. And it
it becomes very much a almost like a you know, Tony Robbins is the one who comes to mind. If you've ever seen the documentary, yeah, where like people are really there Yeah, no, I think there's there's
Siara Singleton (1:03:20)
Yeah, I was gonna say I'm glad you said it.
Renee DiResta (1:03:24)
a few of them, right? They're there's even the ones where the the podcast becomes this source of like
It's a source of hope for so many people, a source of not not
Siara Singleton (1:03:34)
Yeah.
Renee DiResta (1:03:34)
hope and a it's more a like a way to alleviate desperation might be a better way to say that. where there really are these very real issues that they're having. You see this in women's health, right? There's a lot of these people who have very strong opinions about how women should take care of themselves. Birth control became a whole thing. I imagine you saw that conversation too, right?
What does it do to your body? How can you have that conversation where what you want to hear, what you should what you should see as a sign of trust is somebody acknowledging that they don't have all the answers, right? That they don't know all of the things, that
Siara Singleton (1:04:03)
Yes. Yep.
Renee DiResta (1:04:06)
that there's actually some nuance there and that, you know, I had a terrible experience on birth control. I hated it. I stopped taking it. I'm actually fine sharing that opinion, but that is my opinion as one random person, not as a medical professional. Right? And so
Siara Singleton (1:04:18)
Yeah.
Renee DiResta (1:04:19)
how do you
When and when I say that, that that's not a sign that you are gonna have a bad experience or it will be bad for you or you should not take it. That's something you should a decision you should make in conjunction with your doctor, you know, your psychiatrist, your
Siara Singleton (1:04:33)
Yes.
Renee DiResta (1:04:35)
trusted medical, you know, medical person who is helping you navigate for you that set of circumstances. Instead a lot of people will hear like, this person I really like and trust said this was terrible for them. and and they will read into that I think more than they than they potentially
Siara Singleton (1:04:49)
Yeah.
Renee DiResta (1:04:49)
should.
And for someone who's gonna profit off of it, saying, Instead of taking this, you should take that. By the way, here's my referral code for you to go and get it from this online pharmacy, you know that's a different thing. Yeah.
Siara Singleton (1:04:58)
Yeah, my goodness. It's crazy. It's really crazy.
Yeah. your book is called Invisible Rulers, The People Who Turn Lies Into Reality. And so we've talked about a lot of the different types of people who that could be. But I think that you, correct me if I'm wrong, described modern propaganda as like participatory. So I'm
Renee DiResta (1:05:17)
Yeah.
Siara Singleton (1:05:18)
curious like what that means to you and what role like this audience, like what are we doing? What should we not be doing? How are we playing in it?
Renee DiResta (1:05:23)
Yep.
So it's it ties into the the like our chat about outrage, right? When you see that post that somebody has put up on on a social platform and you feel enraged by it, like what do you go do next? This is a real question. Do you go and say,
I'm going to hit the share button because I'm mad and I want other people to see this, right? That's sort of like righteous indignation. You hit the share button or you quote, you know, you quote, quote the post, quote tweet, quote thread, whatever, regram. are
Siara Singleton (1:05:54)
Yeah.
Renee DiResta (1:05:55)
you doing that because you are angry and you just want people to see it, even if you haven't actually gone and checked to see if it's real or reputable? This is a thing that really catches people. It's not a partisan behavior at all. It is something that is just pure psychology.
I mean I do it too. It's like a you know, the algorithm is very good at knowing what you really care about. And I remember during COVID talking to when everybody was locked down and staring at their phones all day long, right? Kind of doom scrolling, but not even just about the pandemic, all sorts of things were kinda coming up as people were just locked down and, you know, staring at their phones all day. I remember a person who I was friendly with who was conservative, coming to me
And saying, like, you know, you're like a mom activist, you do all this stuff for education and for health for kids and stuff, but I don't hear you talking about the trans medicine conversation at all. How do you not talk about X, Y, and Z? And he had all of these specific examples. And I was like, I don't think you understand. Like, I literally never see this. You are absolutely
Siara Singleton (1:06:57)
Yeah.
Renee DiResta (1:06:58)
mad with rage about a conversation that does not hit my feed ever.
Like it is just not now, that's not to say that there's not crazy shit in my feed. There is, of course. I get vaccine stuff all the time.
Siara Singleton (1:07:11)
Yeah.
Renee DiResta (1:07:11)
and I imagine you don't, right? You know? And so some things break through, some things break through and become like major national
Siara Singleton (1:07:15)
Yeah.
Renee DiResta (1:07:18)
conversations, but a lot of the time the algorithm is showing you what is gonna make you mad. And so for this particular person, he was seeing tons of stuff about sort of like youth trans medicine, youth gender medicine, and
And I was like, you know, you see more of it because you engage with it. And it makes it seem like the biggest issue, the biggest conversation, something that is happening to everyone everywhere. And that's just not reality. That's just not, you know, you're seeing a very narrow subset oriented to you to make you mad. When you engage with it, you get more of it. And this happens with this happens with politics. You know, there are there were some really funny articles that got written about the Israel-Palestine debate on TikTok.
Right, the the sort of conflicts there, not debates, probably a bad word, but conflicts,
Siara Singleton (1:08:02)
Mm-hmm.
Renee DiResta (1:08:02)
around like after October seventh. And there were these articles written about how like TikTok's algorithm is pushing out a disproportionate amount of pro-Palestinian content. When what's actually
Siara Singleton (1:08:13)
Mm-hmm.
Renee DiResta (1:08:13)
happening there is you have a user base that is often the quite progressive, that is engaging with that, believes it already, has it reinforced, the algorithm pushes out more, and it just becomes a flywheel effect. This is what you're seeing happen. If you're in
That group, that community, that demographic, you're gonna get more of it. The problem starts to happen when it creates also like blind spots, right? Maybe I should know more about this issue that is riling up all of these people over here. Maybe there is some truth to whatever, you know, pick your outrageous issue of the day. and social media creates both that,
It's it's just that it's not being presented to you in a way where you're learning something. It's being presented to you in a way where it is intended to outrage you. It's not like here are the ten facts you need to know about Palestine, here are the ten facts you need to know about Israel, here is the nuanced you know, news story of what happened there today. Instead it's rage and rage and rage and
Siara Singleton (1:09:09)
Bridge.
Renee DiResta (1:09:09)
yeah, and that's and that's what starts to happen. So
Siara Singleton (1:09:12)
Yeah. before I ask this next question, I'll say like I have moved to unbiased news sources. I like to try and train my algorithms. So when I'm seeing like the big fear bait like my algorithm knew me very well politically. So I had to stop interacting, scrolling really quickly past, never liking, never sharing, never looking at the comments because I wanted my algorithm to stop showing me that.
And now I'm like into the unbiased news sources, but I still have a little bit of healthy skepticism 'cause I
Renee DiResta (1:09:40)
Yeah.
Siara Singleton (1:09:41)
never wanna be but yeah, I'm just curious if like you've been into that at all or if you feel like it's at least better than what most people are doing.
Renee DiResta (1:09:47)
So
I well, I I have two answers for that. One, it's sort of my job to know where the rage conversations are, right? So, I in part because when I am looking at state actor campaigns like this, you know, or the Bangladeshi interference in this data center conversation, I just like to know where the perspectives are, where the positions are. Like I I actually follow a ton of partisan accounts.
really spanning the full political spectrum from sort of far left to far right. obviously I have my own political opinions also as like a a you know a human who exists in the world. so this does mean that I'm constantly seeing stuff that I find crazy and outrageous. But for me it just becomes like it just kind of goes by if that makes sense. I'm like, okay here's what
Siara Singleton (1:10:32)
Yeah.
Renee DiResta (1:10:32)
here's what they're doing today, you know, here's what we're all mad about today. Yeah, what what
Siara Singleton (1:10:34)
Yeah, what are they pushing yeah.
Renee DiResta (1:10:36)
do we what do we got today? and
Again, I've been doing this like twelve years now and it's like it's sort of what I'm like, you know, paid to do. So
so and it's again, it's more important because I just I want to understand not I'm not gonna respond to the narratives. I actually want to understand like where is the when are certain things picked up by an algorithm, when are certain things showing signs of inauthentic participants in a conversation. how do I just have a sense of like what my community notes writer, you know, I have an AI community note writer also.
what it needs to know about the world, you know, is it correct? I should know these
Siara Singleton (1:11:07)
Mm-hmm.
Renee DiResta (1:11:08)
stories before while I while I review what it's doing. so there's that. The other thing though is that on the platform Blue Sky, which I know people have very strong kind of opinions about, either they really like it or they really do not like it, and it is very much a kind of progressive left space, one of the things that it has though is the ability to create feeds and
Myself and some colleagues at Berkeley decided that we were going to build a feed that that was like the feed without rage bait. And more importantly, we were gonna give it sliders so that people who wanted to could actually tweak. Do you want to see news content in here? Maybe you just want to see friends. Do you want to see people that your friends follow? Like what are the social like if you were to create your ideal feed just dragging some stuff with some sliders, there's like four sliders.
how could you do it? And how could we show you why a certain post made it into your feed? So when you click on the little information icon on the post, you can actually see this is in here because you like this topic, this is in here because you engaged with that person.
Siara Singleton (1:12:09)
Yeah.
Renee DiResta (1:12:09)
This is also this is surfacing because it's recent. This is servicing because a lot of people are talking about it. it gives you the ability to and we so we built this. It's called My Sky, it's a feed up on Blue Sky.
and you can just go and you can change your toggles to have your experience however you want it. I use it a lot. it's my main feed, even though like I built it with them, so like this is very self serving. But but the thing in all seriousness, I never feel inclined to switch away from it because it is when I'm consuming my personal social media time. What I like about it is that I am I have sort of like
made it where I'm like, I see a lot of my friends now. This is great. I don't see them on Facebook anymore. You know, most of them aren't like big content creators. They're not putting out posts. They're gonna get a ton of virality. They're not gonna land in my feed unless I go look for them. But telling this
Siara Singleton (1:12:59)
Yeah.
Renee DiResta (1:12:59)
algorithm, I actually I actually care about those people, right? I want to know what my
Siara Singleton (1:13:03)
Mm-hmm.
Renee DiResta (1:13:04)
friends are doing. And it lets me have that. So I can open my app and I just have that experience. And I didn't have to go through like 10 million
don't show me more of this, don't show me more of this, don't you you're clicking that button, like, come on, seriously, no, I mean it, which is why you're describing, right? Which doesn't even
Siara Singleton (1:13:17)
Which like doesn't even work. Yeah.
Renee DiResta (1:13:20)
work. Whereas with this, I think this is really a model for how social media should be. That's why we built it as academics. We make no money on this. we pay money to run it, you know. but the
Siara Singleton (1:13:30)
huh.
Renee DiResta (1:13:30)
but the reason that we wanted to do it, the reason that we wanted to launch it was actually to see like can you depolarize communities by showing them even
content on hard topics, right? Israel-Palestine, you can show people content on hard topics, but that isn't intended to be rage bait. That's not intended to make them feel
Siara Singleton (1:13:52)
Yeah.
Renee DiResta (1:13:52)
angry. And is this a thing that we can actually build? And the reason we did it on Blue Sky is it's an open protocol. So we can. We don't have to ask permission. We can just do it as opposed to Instagram
Siara Singleton (1:14:01)
Yeah.
Renee DiResta (1:14:02)
where, you know, you're at the mercy of whatever Mark decides is good, you
Siara Singleton (1:14:05)
Mm.
Renee DiResta (1:14:05)
know.
Siara Singleton (1:14:06)
And I do think Blue Sky will continue to influence what they do because I think there's some sort of tweaks you can do with your algorithm on at least Instagram now, which like who knows
Renee DiResta (1:14:15)
Yeah.
Siara Singleton (1:14:16)
how how effective it is. I I think it hides a little bit of the truth of your algorithm, but
There's an influence happening and so I I guess we could be thankful for that. I am curious what you think, like if you were to give the audience tips into when they open their feeds today, tips to scroll more thoughtfully, and to maybe detect when something has ulterior motives than what it looks like on the surface.
Renee DiResta (1:14:41)
Yeah.
So I think the number one thing is did this make me really angry and did it really happen? You know, these two things
Siara Singleton (1:14:49)
Yeah.
Renee DiResta (1:14:50)
go together closely. is like the thing that I ask myself, because I don't want this to sound like mean in any way, but is like, does the world need to hear from me on this?
know? So
Siara Singleton (1:15:00)
Yeah. yeah. yeah. That's a good one.
Renee DiResta (1:15:04)
and oftentimes the answer is no. Like the world does not need my take on this, right? sometimes I want to give it anyway. Threads is the platform that I find most often. I'll I'll go and I'll leave a comment. Because because here's the thing, right? On threads, if I want people to see my post, I have to be commenting on other people's posts. This is how Instagram works also. So I have to think about like
What am I gonna comment on? Because the comments are what it pushes out to your friends and your followers, right? So it's an interesting
Siara Singleton (1:15:32)
Mm.
Renee DiResta (1:15:33)
question where I'm like, Threads has a really funny dunking commenter culture. And done right, that can just be like, you know, harmless ribbing. Done wrong, it can be very, very mean. And so I do try to think about you know, how should I engage in this moment? Is this a thing that I need to
respond to in some way? Sometimes I decide the answer
Siara Singleton (1:15:53)
Yeah.
Renee DiResta (1:15:54)
is yes, right? Sometimes I'm like, yeah, you know what? We're gonna do it. sometimes I'm like, no, there's really no upside to me, to me doing that. And I think that having just that consciousness, just that reflection where when you do it, you're doing it intentionally as opposed to because you're mad. or because,
Siara Singleton (1:16:08)
Yeah.
Renee DiResta (1:16:08)
you know, you're you're playing the the sort of clout game, right? And so that becomes the
you know, the the question. I also ask like who can I amplify, right? A how can I when I see something that I think is really important, how can I how can I boost it in a in a way that's like constructive? Like can I give the person props for saying it? Like when somebody's acknowledging that they were wrong about something, I think it's actually really important to give positive
Siara Singleton (1:16:34)
Good.
Renee DiResta (1:16:34)
feedback in those moments. I think just changing the the the culture of how people
react to getting fooled by AI. This is something you see all the time now. How many people have you seen in your feed like sharing something or commenting on something and then all the other comments are like, it's AI you know? Yeah, yeah, yeah, yeah.
Siara Singleton (1:16:49)
It's AI. That's the worst. I hated when it started happening
to me 'cause I was like, no and for me it was the animal videos because I wanted to believe it was true.
Renee DiResta (1:16:57)
Yeah, yeah, yeah, yeah, yeah.
A hundred percent. Yeah, there's all the cute animal videos. I remember I saw yeah, no, or you like I do Google, I do Google or because look, I used to I used to be in the position, like when I was at SIO, of doing the work with media to put out these PSAs. Like, here's how you tell if that
Profile picture is fake, right? You look at the eyes, you look at the teeth, does the ears blend into the collar? Are the earrings irregular? You know, all these different tells that we had for a while. the fingers count them. None of that works anymore. Just to be clear, it does not
Siara Singleton (1:17:28)
No. It doesn't work.
Renee DiResta (1:17:30)
work, right? And so I think it's really important where, you know, I say regularly, like, I don't know, you know, I say
Siara Singleton (1:17:36)
Yeah.
Renee DiResta (1:17:37)
this stuff, and sometimes it's my job to work on deciding if something is authentic or not, right? And
Siara Singleton (1:17:43)
Mm-hmm.
Renee DiResta (1:17:44)
Yeah, I remember one where there was very prominent civil rights person, person in the civil rights space, constantly posting civil rights content, very prominent figure who posted this video right after the administration changed and the ICE you know, the sort of ICE actions really began to increase, of a of a baby being taken from its mom somewhere in New York, somewhere kind of up in the Bronx. And I was like, okay, like I know that area. Let me go and let me look at this video.
And my cause my first thought was like, is it AI, right? Like, is this real?
Siara Singleton (1:18:14)
Yeah.
Renee DiResta (1:18:14)
Like it was so incendiary, like, is this real? And I started going through it because the thing was, like, it was it was shot on an old camera, right? Old camera phone. So it had some jankiness to it that could have been AI artifacts. It turned out, spoiler, it
Siara Singleton (1:18:26)
Mm-hmm.
Renee DiResta (1:18:29)
was actually old footage. It wasn't ICE at all, it was a custody
dispute, right? So it was a custody dispute video where the baby was being removed.
in response to a a custody situation. And the and the video was entirely authentic, but it had been put out and reframed as ICE tears mom and baby apart. All right. And so that's where it's not even enough to know, is it real? What you also have to be thinking about is, is it true? And these are not the same thing. That video is real. The claim it was being used to support was not true.
There are other videos, I'm sure, where you could find a a video of IES doing a family separation. That is a real thing that happens. In this particular case,
Siara Singleton (1:19:08)
Mm-hmm.
Renee DiResta (1:19:09)
that was not what was happening. So it was a it was a really challenging one. I was like in there looking at, you know, people were wearing like Yankees jerseys and I like I was like, okay, is that number and that player back then the same?
Siara Singleton (1:19:20)
Yeah.
Renee DiResta (1:19:23)
You know? so it was no. Yeah. Yep.
Siara Singleton (1:19:24)
my god. And we shouldn't have that burden on us, honestly. It's just absolutely ridiculous.
one last question before we close out? But I'm curious if there's something, either a campaign, maybe AI slop but I'm definitely curious if there's been like an adversarial abuse campaign, the last one that you remember you believed. At least for a while. And it's shifted your own opinion.
Renee DiResta (1:19:43)
That's interesting. man,
I'm so skeptical about everything. That's that that's the downside of this job, is like you just know, it's like
Siara Singleton (1:19:50)
That's good.
Renee DiResta (1:19:53)
it's you know, the I I remember I was joking around with a reporter on a call and I just sort of was like, the entire internet is fake and like that was the quote that he stuck in the article and I was like, Okay, you know, that's my fault for not being like
Siara Singleton (1:20:04)
here we go.
Renee DiResta (1:20:06)
so on it, you know, media be very precise, be very, you know, like
You never get into that casual
Siara Singleton (1:20:10)
Yeah.
Renee DiResta (1:20:12)
The casual chat is what they take. I was Amazon doesn't know what Amazon sells, which is true, but whatever. How do you how do you say that constructively, right?
Siara Singleton (1:20:18)
Yeah. Mm-hmm.
Renee DiResta (1:20:21)
but I think that it it's actually that the entire internet isn't fake. There are many people out there who are very real and doing many real things.
Siara Singleton (1:20:29)
Mm-hmm.
Renee DiResta (1:20:30)
when you do hit that sort of cynical, it's all it's all bullshit, you know. it's hard. But as far as like what have I gotten duped by? I I definitely get these the AI influencer content that shows up where I'm like, is this does this product exist in the world? Can I buy that dress? You know?
Siara Singleton (1:20:49)
Yeah.
Renee DiResta (1:20:50)
the so I I see that
Because I follow I think such a broad range of political content,
This is maybe gonna sound a little bit strange, but I see it all as stories now. I I s I see it all as like which propaganda narrative
Siara Singleton (1:21:04)
Mm-hmm.
Renee DiResta (1:21:05)
is it that I'm gonna be looking at today? so I I think I'm like
Siara Singleton (1:21:07)
I'm starting to as well.
Renee DiResta (1:21:09)
a little bit and it's not
to say that that means that I am innately distrustful of all politicians now. I think I just think a little bit differently about like what are the incentives that this person has to say this thing? What are the incentives that are underpinning this conversation or what that corporation did?
I have also though gotten gotten duped by some of the animal ones because those are very, very realistic. And yeah, I I feel like there must be something. There
Siara Singleton (1:21:31)
Yeah. And cute.
Renee DiResta (1:21:34)
must be something. I've definitely Googled, found a fact check and been like, that thing wasn't true. so yeah. Yeah.
Siara Singleton (1:21:39)
same. Yeah. It happens to me all the time.
Yeah, I
Renee DiResta (1:21:42)
Yeah.
Siara Singleton (1:21:42)
think I'm I'm becoming harder to ra like I'm more skeptical than ever, but I think I've become harder to rage bait, which is very exciting.
Renee DiResta (1:21:49)
Yeah,
Siara Singleton (1:21:50)
So yeah.
Renee DiResta (1:21:50)
I well I feel like my job has made me
less partisan, not because it means I like all the parties, but because it
Siara Singleton (1:21:57)
Yeah.
Renee DiResta (1:21:58)
means I don't like any of them. You know? That's kind of where I've landed, which I don't know if that's like so yeah, so that's that's my my personal evolution over the last twelve years
Siara Singleton (1:22:01)
No, I I'm I'm with ya there. Okay.
So interesting. okay, so where can we find out more about you and your work?
Renee DiResta (1:22:13)
Yeah, so I'm on threads and blue sky a lot. I do a lot of like
I need to do more video. I do much more text-based stuff. I am on Instagram, renee.dresta. I think I'm most active now probably on Substack. I spend a lot more time
I have a newsletter that is not on Substack for people who hate Substack. It's on my website. But what I have come to like about Substack actually is that what I say does reach those people who are inclined to disagree with me. I actually can have a conversation,
Siara Singleton (1:22:42)
Mm.
Renee DiResta (1:22:43)
with people when I'm putting out content who you know, we didn't even talk about this, but like when I become the subject of conspiracy theories, because that's what happens to you when you do this work, those people can reach
Siara Singleton (1:22:52)
Yeah, I heard about that.
Renee DiResta (1:22:54)
Yeah.
That's how Stanford Internet Observatory got shut down, right? This claim that we
were like secretly part of a cabal that tried to help steal the twenty twenty election, you know, by mass censoring conservatives
Siara Singleton (1:23:03)
So ridiculous.
Renee DiResta (1:23:05)
with tens of millions of censored tweets. And when you're like,
Okay, we'll have Elon release the tweets. Where are they? Show me the tweets. Where's where
Siara Singleton (1:23:10)
Yeah, where's his tweet about that? Yeah.
Renee DiResta (1:23:13)
where are my where are the 22 million tweets we censored? Like what were they? Show me them. Show me what they look like. Because I had
Siara Singleton (1:23:18)
Yeah.
Renee DiResta (1:23:18)
I was under a congressional subpoena. All of my emails went out the door. All of my research went out the door. Jim Jordan spent two years combing through my emails. Stephen Miller sued me. There was discovery in that case. Where
Siara Singleton (1:23:28)
Yeah.
Renee DiResta (1:23:29)
is the evidence? Show the public the evidence. If we did this thing, show it to them. I want them to see it. If I did it, show it to them.
Haul me in front of a public hearing. Come on, let's do it, you know. But that's not what they want to do. That's not how it works, right? They're they're gonna smear you, frame you, and put out
Siara Singleton (1:23:42)
Yeah, it doesn't matter.
Renee DiResta (1:23:44)
a story about you. And you don't get to respond to that. You don't get to be in that story. So the reason I like Substack is that I can actually reach and connect with people who have heard those stories and want to ask the questions. And that for me is very different
Siara Singleton (1:23:57)
Mm-hmm.
Renee DiResta (1:23:59)
than social media where people have really moved into opposing camps and the right, you know, the right is on Twitter, the left is on Blue Sky.
threads is kind of a mishmash. so that for me is like where you sort of unexpected
Siara Singleton (1:24:10)
Yeah.
Renee DiResta (1:24:11)
rant at the end there. But like I've I really do feel like Substack has become a place where you still do have this
Siara Singleton (1:24:15)
It's grown on me.
Renee DiResta (1:24:16)
you have this conversation between divergent groups of people that otherwise don't encounter themselves on social and I've come to appreciate that.
Siara Singleton (1:24:23)
awesome. I'm gonna link it in the description.
Renee DiResta (1:24:25)
Cool.
Siara Singleton (1:24:26)
and then one last question I ask everyone this, so it's however you interpret it, but what is one thing that you are logging out of right now and one thing you're logging into?
Renee DiResta (1:24:33)
Yeah.
I spend less time posting now, actually. I unless there's something that, you know, I really feel I need to to go and say. So I don't spend as much time on social as I did. I spend a lot more time in group chats with friends, where I feel like I still get news and information
But I'm also hearing about their lives. Like that that's what social kind of left behind, right? It became like to use Dana Boyd's term, like parasocial media, as opposed to where I actually encountered my friends. So I I really feel like it's important to like reinvest in those relationships with people I know and like and want to spend time with, even if they're not in the same city or town as me. And then my kids, right? I've got three kids and it is really
Siara Singleton (1:25:18)
Mm.
Renee DiResta (1:25:18)
humbling when your child says to you something like, you know, they ask you a question, you're distracted by your phone.
And then they're like, was that a real no or a distracted no? You know? you're your five year my
Siara Singleton (1:25:27)
my god, your kids sound smart.
Renee DiResta (1:25:30)
five year old has said that. And then we're like, my God, that was so awful, right? Because you're so used to just like there's a million things going on. You're responding, you're responding, you're responding, you're looking at this, you're looking at that, and you're
Siara Singleton (1:25:40)
Yeah.
Renee DiResta (1:25:40)
not really investing in the people who are actually in the room with you. And so I have also spent a lot more time, you know, since since having kids really trying to say, like, how can I,
How can I really prioritize those relationships even when I have a very busy schedule and a and a crazy job?
Siara Singleton (1:25:55)
Mm-hmm. That's amazing. I bet your man, your kids are gonna be such brain yaks. Well 'cause you said
Renee DiResta (1:25:59)
Ha ha ha.
Siara Singleton (1:26:01)
you said before that your twelve year old was saying like everything is money and I'm like, Yeah, everything is money. I need to I need to look at the internet the same way. Like, wow, so smart, so
Renee DiResta (1:26:10)
They they really they because they grow up with this stuff, they're sort of innately attuned to it. On the flip side though, like they do this is how they think of participation, you know,
Mom, can I get TikTok so I can do those dances? And I'm like, no. Absolutely. I have a nine-year-old also. I'm 12, 9, and 5. And I'm like, you absolutely cannot at nine-year-old at nine years old dance on TikTok. Like, I will not allow it. Nope. Mean Mom, no way. You know? So
Siara Singleton (1:26:33)
Yeah. Very wise. Yeah, very, very wise. it's
amazing. Okay, well thank you so much for joining us. There are so many great questions I needed answers and you certainly did that. So thanks for joining. Yeah.
Renee DiResta (1:26:48)
No, it was really fun. Thanks for having me on.
LOG OUT PODCAST FACT CHECK
EPISODE: #020: The Business Behind Rage Bait: Renée DiResta on Who Profits From Outrage and How to Spot Propaganda
Claim 1: Renée DiResta claimed when X revealed where accounts were based, many provocative, monetized MAGA influencers turned out to be posting from countries including Kenya, Nigeria, Pakistan, India, and Eastern Europe — TRUE
When X rolled out its “About this account” location feature in November 2025, users found that dozens of high-engagement MAGA-branded accounts presenting as American patriots were listed as based overseas. Reported examples included a roughly 390,000-follower account in Eastern Europe, a large Ivanka Trump fan account in Nigeria, and accounts in India, Bangladesh, and Thailand, while viral posts also named Kenya and Pakistan. Some account owners blamed VPNs or bugs, and X acknowledged early accuracy issues with the feature, so individual listings are worth treating with some caution.
Source: Full List of MAGA Accounts on X Exposed As Being Based Outside US - newsweek.com
Claim 2: Siara Singleton and Renée DiResta claimed Alex Jones used the “crisis actors” label against Sandy Hook families and got in serious trouble for it — TRUE
Jones told his audience for years that the 2012 Sandy Hook shooting was staged and that grieving parents were hired actors. Juries in Texas and Connecticut later ordered him to pay the families close to $1 billion combined in defamation damages, and he conceded under oath in 2022 that the shooting was real.
Source: Jury orders Alex Jones to pay nearly $1 billion in Sandy Hook defamation trial - axios.com
Claim 3: Renée DiResta claimed Facebook changed its monetization program within roughly the last year and a half in a way that rewards engagement-driven posts — TRUE
Meta announced Facebook Content Monetization on October 2, 2024, merging in-stream ads, Ads on Reels, and the Performance Bonus into one program that pays for text and photo posts in addition to video. Earnings are based on the
engagement, views, and plays a post receives. The beta launched closer to two years before this recording, with wider enrollment through 2025, so her timeline is slightly compressed and the substance holds.
Claim 4: Renée DiResta claimed Elon Musk announced that X took down a Chinese network of 200,000 fake accounts, about 200 of which were posting against AI data centers while the rest amplified them — PARTIALLY TRUE
The numbers are right. X’s Safety team disclosed on August 28, 2026 that it had found a bot farm of roughly 200,000 suspected Chinese inauthentic accounts, including 200 posting about data centers, electricity prices, and grid strain. The announcement came from X’s Global Government Affairs account rather than from Musk himself, and X called the accounts suspected Chinese operations without establishing state direction. X also did not say what the remaining accounts were doing, so the idea that they existed to amplify the 200 posters is her read of how these networks typically work.
Claim 5: Renée DiResta claimed OpenAI disclosed the same data center campaign months earlier, in July, and reported that it got almost no engagement — PARTIALLY TRUE
OpenAI’s report came out on June 10, 2026, a month earlier than she said. It described banning a likely China-origin cluster of ChatGPT accounts that produced content criticizing US data centers, rated the threat as low, and noted that most of the posts drew little or no engagement. Researchers who compared the two disclosures found that the accounts X announced in August had already been highlighted in OpenAI’s June report.
Source: How X turned old bot data into a story about a national threat from China - san.com
Claim 6: Renée DiResta claimed Bangladesh-based operators are running Facebook pages that pose as American state pages and push anti-data-center content to earn money — TRUE
A May 2026 investigation documented a Facebook spam network run from Bangladesh that posted AI-generated “life in [state]” content with recurring anti-data-center, anti-tax, and anti-Chinese-import themes aimed at rural American audiences, and several of the accounts appeared to be receiving Facebook payouts. We could not independently verify her comparison that these pages drew more engagement than the Chinese network X disclosed.
Source: Life in <U.S. state>, according to spammers from Bangladesh - conspirator0.com
Claim 7: Renée DiResta claimed Anthropic announced several new state-linked takedowns in the past week, with Russia among them — TRUE
Anthropic published its September 2026 threat intelligence report on September 10, covering misuse it disrupted between December 2025 and August 2026. The influence operations section details nine cases, originating in Russia, Iran, Turkey, and across the Gulf, South Asia, Africa, and Europe, in which actors used Claude to build fake profiles and news sites.
Source: Countering misuse of AI: September 2026 - anthropic.com
Claim 8: Renée DiResta claimed Platforms stopped sharing detailed takedown data with outside researchers, and X’s announcement gave the public nothing it could check — TRUE
X’s disclosure included no account list, technical indicators, creation dates, posting history, or engagement figures. Clemson researcher Darren Linvill described it as X’s first public acknowledgment of a foreign influence takedown in years, which stands apart from Twitter’s earlier practice of publishing takedown datasets for researchers.
Claim 9: Siara Singleton and Renée DiResta claimed X and Meta both use Community Notes, with Meta’s version running on Threads and Instagram — TRUE
Meta ended third-party fact-checking in the US and began testing Community Notes on March 18, 2025, using the open-source algorithm X built. The program runs on Facebook as well as Instagram and Threads.
Source: Testing Begins for Community Notes on Facebook, Instagram and Threads - about.fb.com
Claim 10: Renée DiResta claimed She has participated in Community Notes since 2022, when the program was still called Birdwatch — TRUE
Twitter launched Birdwatch in January 2021 and rebranded it as Community Notes in November 2022, shortly after Musk’s acquisition, so joining in 2022 places her in the Birdwatch era. The program was first rolled out by Twitter, before the company became X.
Source: Community Notes - wikipedia.org
Claim 11: Renée DiResta claimed A note is published only when raters who usually disagree with each other agree that it is helpful, based on rating history rather than political identity — TRUE
Meta’s documentation says the system looks for agreement among contributors who normally disagree on note ratings, and that this is based only on their rating history. X’s program uses the same bridging-based approach.
Source: Community Notes: A New Way to Add Context to Posts - transparency.meta.com
Claim 12: Renée DiResta claimed Most submitted community notes never get even one rating — MOSTLY TRUE
Her larger point is accurate, since very few notes ever appear. On X, 87.7% of posts with a proposed note stayed in “Needs More Ratings” between 2021 and 2025, and only 8.3% of notes were published. Meta has reported that about 6% of its notes are published. Those unpublished notes typically received some ratings, only not enough to reach a verdict, so “never even one rating” overstates the problem.
Source: Community Notes are Vulnerable to Rater Bias and Manipulation - arxiv.org
Claim 13: Meta AI generates Community Notes because Meta has had trouble getting enough people to write them, and human contributors then rate those AI-written notes — TRUE
Meta appears to be testing AI-generated Community Notes. In the contributor interface, Meta states: “When someone requests a note, AI may generate a draft note for contributors to rate,” and says the notes are only published if contributors rate them helpful. We could not find public Meta documentation describing this test, but the feature is confirmed by Renée DiResta’s contributor account and the screenshot provided below.
Source: Screenshot from Meta Community Notes contributor interface, provided by Renée DiResta
Meta Community Notes documentation - meta.com
https://www.facebook.com/share/p/19y1jgEtey/
Claim 14: Renée DiResta claimed Musk bought Twitter in 2022, monetized posting about a year later, and tied payouts to engagement from paying verified users — TRUE
Musk took over in October 2022, and X launched its Creator Ads Revenue Sharing program on July 13, 2023. Musk said payouts depended on how many ads were shown to other verified users, and eligibility required a paid subscription.
Source: Twitter creator ads revenue sharing program - socialsamosa.com
Claim 15: Renée DiResta claimed Nikita Bier was behind the location feature and has since left X, and a proposed region-based payout change was walked back — TRUE
Bier stepped down as X’s head of product in August 2026 and stayed on as an adviser. He was closely associated with the “About this account” feature that displayed accounts’ countries of origin. X also withdrew a proposed change that would have weighted creator payouts toward local impressions after backlash.
Source: X Product Chief Nikita Bier Steps Down After One Year - dataconomy.com
Source: Nikita Bier Leaves X Product Chief Role and Remains as Company Adviser - mezha.net
Claim 16: Renée DiResta claimed She published a Carnegie white paper the day before with Rachel Kleinfeld and Margot Treadwell on how experts should engage on social media — TRUE
Carnegie published “How to Engage with New Media: A Strategic Guide for Nonprofit Organizations” by DiResta, Kleinfeld, and Treadwell on September 9, 2026.
Source: How to Engage with New Media: A Strategic Guide for Nonprofit Organizations - carnegieendowment.org
Claim 17: Renée DiResta claimed She helped build MySky, a Bluesky feed with sliders that shows users why each post appears — TRUE
MySky was built by DiResta and Jonathan Stray of UC Berkeley’s Center for Human-Compatible AI. Users can log in with a Bluesky account and adjust how heavily the feed weighs engagement against a “constructiveness” score, and the tool explains the ranking choices that are normally hidden.
Source: Meet the algorithm you can actually control and understand - newpublic.substack.com
Claim 18: Renée DiResta claimed The Stanford Internet Observatory was shut down after claims that it mass-censored conservatives, following Jim Jordan’s subpoenas and a lawsuit from Stephen Miller — TRUE
Jordan’s House subcommittee subpoenaed Stanford over SIO’s work in 2023, and America First Legal, the group Stephen Miller founded, sued SIO researchers that same year. Reporting in June 2024 described the lab as being dismantled after lawsuits had cost Stanford millions, though Stanford said some of its work would continue under new leadership. The “22 million tweets” figure she references came from a mischaracterization of an Election Integrity Partnership report.
Source: The Stanford Internet Observatory is being dismantled - platformer.news
Claim 19: Renée DiResta claimed “Parasocial media” is danah boyd’s term for what social media has become — TRUE
danah boyd argued in a 2026 essay that most users now scroll rather than post, and that the content they see is strategically produced and algorithmically curated, which is why she proposes calling these platforms parasocial media.
Source: Social Media Is Now Parasocial Media, danah boyd, 2026 - jmason.ie
Subscribe to Log Out
wherever you listen.
