Are Australia’s misinformation detection tools keeping pace with AI-enabled influence campaigns?
Australia’s bot-detection tools are being defeated. Fact-checking is too slow to matter. And the campaigns spreading influence today use real people – not bots – to hide their coordination. The AI and Misinformation Symposium Fivecast hosted this year made one thing clear: the threat has evolved, but most detection approaches have not.
Earlier this year, we hosted the Fivecast AI and Misinformation Symposium with the support of the Australian Institute of Professional Intelligence Officers. I opened the day with something I believe deeply: misinformation is no longer just a technology problem. It is a societal challenge, a national security challenge, and increasingly a business risk. What I heard from the researchers, defence practitioners, and intelligence professionals across that day confirmed that the scale of the problem has outpaced the tools most organisations are using to address it.
In a 2024 survey cited at the symposium, 76% of Australians said they believe misinformation is a problem – and 75% believe the government should act. The frustration is well-founded. But AI misinformation detection in Australia has not kept pace: the frameworks most organisations deploy are increasingly unfit for the threat they face.
The Real Problem: Real People, Not Bots
The dominant model of influence operations has shifted. It is no longer primarily automated accounts posting content at scale. It is what researchers at the symposium called “cyborg propaganda” – a model where an organising entity generates AI-tailored content for individual authentic users, who then post it through their own accounts. The campaign hides its coordination signature behind real people.
I have watched this evolution closely at Fivecast. What was once episodic and human-driven has become systemic, automated, and increasingly sophisticated. AI lowers the barrier to entry and raises the ceiling of impact. The consequence for investigators is significant: the authenticity of the accounts defeats most automated detection, the volume defeats manual review, and the campaign looks organic precisely because, at the account level, it is.
Three Things That Are No Longer Working
Fact-checking: Professor David Matthews, Program Leader – Strategic & Asymmetric Effects for Defence Science and Technology Group, was direct: “One of the things we really learned quickly was that fact-checking doesn’t work.” Misinformation only needs to be believed for a short window – hours, not weeks – to shift a decision or harden an attitude. Corrections arrive too late.
Bot-detection tools: Botometer and similar tools were trained on features from a previous era. Generative AI now produces account behaviour that evades their classifiers. One researcher noted there was limited published work on this detection gap as recently as two years ago. Tools calibrated to yesterday’s campaigns do not reliably detect today’s.
Single-platform monitoring: Coordinated campaigns migrate. They seed on high-reach platforms, amplify through smaller communities, and consolidate in less-moderated spaces. Any analytical approach operating within a single platform sees only a fragment of the picture. The adversary builds their operation across the gaps.
The Psychology That Makes It Work
Professor Emma Thomas from Flinders University presented a finding that I think reframes the entire detection challenge: AI does not change the psychology of influence, it changes the scale and speed of delivery. Campaigns that combine group identity with perceived harm to that group are dramatically more effective at mobilising action than those that simply present false information.
The Voice to Parliament referendum study – 15 months of data, more than 7,000 users – showed that reactive polarisation is conditional, not inevitable. Understanding when and why communities tip toward radicalisation requires longitudinal data and a behavioural framework, not just a content filter.
That insight matters to me because it means the problem is solvable – but only with the right tools and the right analytical approach. Polarisation that is conditional can be detected early. Trajectories that are measurable can be interrupted.
What Effective AI Misinformation Detection Actually Requires
The Sara Duterte coordination case study, presented by researchers from Fivecast and the University of Adelaide, shows what holistic, multi-signal analysis looks like in practice. Starting from 295 posts and 27,000 engagements, the investigation narrowed to 459 suspect accounts using timing coordination, character entropy analysis, and semantic distance mapping. No single signal was sufficient. As Angus Lewis concluded: “None of those things made me convincing on their own – and that’s kind of the point.”
We built Fivecast ONYX to support exactly this type of cross-platform, multi-signal investigation – persistent collection across open-source channels, AI-assisted network and semantic analysis, and traceable outputs that keep human judgement at the centre. The tool surfaces the patterns. The analyst makes the call. That principle matters to me: certainty in a world of unknowns requires human expertise working with the right intelligence, not automation working alone.
“The question is not whether misinformation will continue to evolve. It will. The question is whether we can evolve faster in how we detect, understand, and respond to it.” – Dr. Brenton Cooper
About Fivecast
Fivecast delivers intelligence solutions built for clarity, powered by AI, and trusted to surface what matters. Engineered to solve complex intelligence challenges our platform cuts through digital noise to help those protecting nations, borders, businesses, and communities uncover critical insights – before risk becomes reality.
Trusted by agencies and enterprises across national security, law enforcement, defense, corporate security, and financial crime, Fivecast was born from collaboration between governments and research institutions. Headquartered in Australia with a global footprint, we support the world’s most critical missions.
Fivecast. Engineered for Intelligence.
