Key Findings: A comprehensive analysis reveals that research on social media bots suffers from widespread methodological flaws and misconceptions, undermining scientific credibility in a field crucial for understanding online manipulation.
Why it matters: As AI-powered bots become more sophisticated and social media platforms restrict data access, flawed research methods could leave society vulnerable to manipulation while eroding trust in legitimate scientific findings.
Model drift — Detectors trained on known bot characteristics fail when applied to new, unknown bots
Unfair evaluation — Researchers cherry-pick favorable comparisons and use inconsistent definitions
Data biases — Datasets contain outdated or biased samples that don’t reflect current bot populations
Straw-man arguments — Critics oversimplify the field, ignoring advances beyond basic supervised learning
Misconceptions debunked
- Bot detection is a solved problem (it’s not — it’s an ongoing arms race)
- Bot detection can be easily improved (adversarial nature makes this extremely difficult)
- All social bots are similar (vast diversity exists across bot types)
- Any detector can catch all bots (specialized tools needed for different threats)
- Bots are the main driver of disinformation (many other factors involved)
- All bot research is useless (has led to successful platform removals and policy changes)
Why It May Not Get Better: Platform API restrictions since 2023 have severely limited researchers’ access to fresh data, while bot operators continue operating with minimal restrictions. Despite significant flaws, social bot research has produced valuable insights and real-world impact. The field needs more rigorous methods and responsible reporting — not abandonment.
Source
Cresci, S., Yang, K.-C., Spognardi, A., Di Pietro, R., Menczer, F., & Petrocchi, M. (2025). Demystifying misconceptions in social bots research. Social Science Computer Review, 0(08944393251376707), 08944393251376707.
Citation
@article{infoepi_lab2025,
author = {{InfoEpi Lab}},
publisher = {Information Epidemiology Lab},
title = {Social {Bot} {Research} {Is} {Broken} — {But} {Not}
{Hopeless}},
journal = {InfoEpi Lab},
date = {2025-09-23},
url = {https://infoepi.org/posts/2025/09/social-bot-research.html},
langid = {en}
}