A former Twitter executive responsible for policing misinformation has warned Elon Musk’s takeover has made the social media platform increasingly vulnerable to foreign governments and extremists trying to influence the Voice to Parliament referendum.

Yoel Roth led Twitter’s moderation and integrity unit, which monitored and investigated abuse, inappropriate content and misinformation, until he quit shortly after Mr Musk bought the company last October.

He told ABC Investigations the Voice to Parliament vote was a “prime target” for disinformation campaigns and the platform, renamed X in June, was doing almost nothing under Mr Musk’s ownership to stop lies spreading among its users.

“Elections, and especially elections that touch on divisive social and cultural issues, are prime targets for organised disinformation campaigns and abuse, both foreign and domestic,” Mr Roth said.

“We’ve seen this playbook deployed by the governments of Russia, Iran, and China, as well as domestic extremist groups around the world. I would expect these threats are present ahead of Australia’s referendum this year.”

a man in a suit talking into a microphone looking serious

Yoel Roth, former global head of trust and safety at Twitter, has delivered a dire warning about extremists influencing the Voice to Parliament debate.(Reuters: Evelyn Hockstein)

He said that although X’s overall user base appeared to have dwindled since Mr Musk’s takeover, its enduring popularity among politicians and journalists meant it could still be used effectively to spread misinformation.

“The most successful disinformation efforts we’ve seen over the years have used social media to try to reach mainstream audiences via the media, and Twitter is a particularly effective way to reach those groups” Mr Roth said.

The former Twitter executive’s comments come as X faces criticism for quietly removing a function which allowed users to report election misinformation in posts.

Mr Roth’s intervention also coincides with new research showing X and other social media platforms have approved paid advertisements containing clearly false information about the Voice to Parliament referendum.

Lobby group Reset.Tech Australia published the results of an experiment showing X, Meta and Tiktok allowed it to submit dozens of ads featuring misleading claims, including that the date of the referendum had changed and that the Australian Electoral Commission (AEC) supported a Yes vote.

X ‘slammed the door shut’

Mr Roth has been a central figure in Twitter’s attempts to tackle misinformation.

During his seven-year tenure at the San Francisco company, he was part of the team which chose to suspend former US president Donald Trump’s Twitter account in 2020.

After quitting in the wake of the acquisition of Twitter, he said he was forced into hiding after Mr Musk falsely claimed in a tweet that Mr Roth had advocated for the sexualisation of children in his PhD thesis.

X and Twitter Logo

Twitter was renamed X in June.(Reuters: Dado Ruvic/Illustration)

Mr Roth said under Mr Musk’s leadership, the company had pulled back from “protecting the integrity of election conversation” by “laying off virtually all of the teams responsible for election security and dismantling many of the protections the company had built up since 2016”.

“Especially destructive is the elimination of Twitter’s Curation team, including a significant Sydney-based team focused specifically on Australia,” he said.

“Their efforts to elevate credible, expert sources during elections were an essential part of how Twitter combated the spread of misinformation on the platform.

Mr Roth also criticised X for ending its partnerships with civil researchers monitoring online disinformation, including groups like the Australian Strategic Policy Institute (ASPI) which he said was an “essential part” of Twitter’s operations in and around Australia.

“Twitter has slammed the door shut on these collaborations, to the detriment of both the company and the security of global elections,” Mr Roth said.

APSI misinformation and election interference analyst Albert Zhang said his organisation had “significantly less” engagement with Twitter since Mr Musk’s takeover.

“Like other researchers, our access to Twitter’s formerly free data has also been cut off and subject to prohibitive pricing,” he said.

“This has impacted our ability to track misinformation, disinformation and other threats across the platform.”

An ASPI analysis in July found multiple Twitter accounts which appeared to be linked to the Chinese Communist Party (CCP) were amplifying negative posts related to the Voice to Parliament.

“We assess this network is very likely linked to the Chinese government and is part of a broader covert campaign to undermine Australia’s social cohesion and trust in government through X,” Mr Zhang said.

Research published by Queensland University of Technology’s Timothy Graham earlier this year also revealed coordinated misinformation on X. Hundreds of accounts which had limited profile information were created in the months following Mr Musk’s takeover and have since begun promoting anti-Voice messages.

‘You could very easily believe it’

Mr Roth’s comments also come at the same time as research showing major social media companies, including X, have permitted patently false paid advertisements about the Voice to Parliament on their platforms.

Lobby group Reset.Tech Australia this week published the results of an experiment in which it mocked up dozens of images about the referendum and the AEC and submitted them for approval on TikTok, X and Meta’s Facebook.

X Fake News Ads

X approved all of the researcher’s political paid ads, even though they included the AEC logo alongside false claims about how to vote twice.(Supplied: Reset.Tech Australia)

The group has provided a submission to the Exposure Draft Bill on combating online misinformation and disinformation, arguing for stronger legislative oversight of social media platforms.

Reset.Tech Australia executive director Alice Dawkins said the ads were based on common tropes used by dishonest actors online and were designed by the researchers to be overtly, even comically, misleading.

She was shocked an overwhelming majority of them had been approved for publication.

Several of Reset.Tech Australia’s ads which were approved contained incorrect voter information alongside the AEC logo. This included changes to the date of the referendum, declarations that the AEC had supported a Yes vote, and that voting was optional.

“These [were] not just about differences of political opinion that’s ventilated in robust online debate,” Ms Dawkins said.

“It’s digital attacks on electoral processes. This is where platforms have special policies, because they know it’s important.”

fake news ads on tiktok

Approved political ads from the researchers on TikTok included false claims the Voice was rigged.(Supplied: Reset.Tech Australia)

Other false ads contained fake news that the AEC was being investigated for fraud or that the referendum had been cancelled, an apparent guide on how to cast your ballot twice, and references to conspiracy theories commonly pushed by the sovereign citizen movement.

Reset.Tech Australia reported X approved all of the fake ads, TikTok approved 70 per cent and Meta allowed all but one through.

Screenshots shared by Reset.Tech Australia showed the ads had been approved and scheduled for publication, but the researchers did not let them go live.

Ms Dawkins questioned why the social media platforms’ image-recognition software did not detect the logo and names of the AEC, given each company had explicit rules on political advertising.

X did not respond to ABC questions.

A Meta spokeswoman told ABC Investigations it had a “robust approach to safeguarding elections, including taking measures to reduce the spread of misinformation”.

“This report was based on a very small sample of ads and is not representative of the number of ads we review daily across the world,” she said.

“Our ads review process has several layers of analysis and detection, both before and after an ad goes live.

“As noted in the report, these ads never went live, and therefore our full enforcement detection technology did not have an opportunity to pick up these ads.”

Alice Dakwins Reset Australia

Reset.Tech Australia’s Alice Dawkins said she was shocked a majority of the false political ads made it through the approval process.(Supplied: Reset.Tech Australia)

A TikTok spokesperson said the platform took the claims in the Reset.Tech Australia report seriously and was conducting its own inquiries.

Ms Dawkins said the experiment indicated how harmful content could potentially be served to millions of Australians.

“What worries me is if some of these ads land in your news feed at the right time,” Ms Dawkins said, “and you’re all fired up because you’ve just been recommended all this dodgy content about how there’s all these apparent issues with the referendum process.

“If you saw an ad in that kind of context, you could very easily believe it.”


If you’re unable to load the form, you can access it here.

Get the latest health news and information from across the ABC.