Science skeptics have historically relied on anecdotes and misused data to promote false claims. Now, they’re increasingly attempting to pass off fake studies as legitimate research.
Low-quality and non-credible studies are everywhere online, presented as proven science to bolster false claims. Sometimes, they masquerade as “analyses” published on authors’ websites. Others publish papers in unreliable journals that lack editorial integrity and a strict review process.
It can be difficult to catch red flags if you don’t frequently read scientific or medical studies. Many researchers have formal training and years of practice to help them understand these papers. But you don’t need to be an expert to identify a flawed study. Here’s what to look for.
Understand the publication process
When researchers submit a study to a journal for publication, it must first pass initial screening by an editor. The study then undergoes peer review, which involves multiple independent experts in the field who review the study for accuracy and scientific rigor, or precision.
Peer reviewers give the study’s authors feedback and may recommend additional experiments to strengthen the study. An editor then decides whether the study should be accepted, rejected, or sent back for revisions before being resubmitted. Studies are rejected if there are serious issues with the research methods, data, or overall quality that cannot be resolved.
Know that preprint studies are preliminary
Because the peer review can take months or even years, many researchers use preprints to make their findings available to their peers, the media, and the public sooner. A preprint is a paper draft that hasn’t been reviewed yet by other experts.
The fact that preprints haven’t been peer reviewed doesn’t mean that they aren’t accurate; they are just preliminary and haven’t been carefully checked by independent reviewers. Preprints are important for scientists to share and get feedback on their research, but they have drawbacks.
“By omitting the peer-review process, readers are left to take the science at face value, without the gold standard of other scientists critiquing and offering suggestions before it is published,” wrote Matthew Bauer, who has a PhD in Biomedical and Biological Sciences from Harvard University, in a 2022 article.
Responsible researchers use preprints to make their findings accessible to colleagues and the public. Irresponsible researchers abuse the preprint system to promote studies that they know are unlikely to pass peer review or be published in a reputable journal.
The potential risks of preprints became apparent in the early months of the COVID-19 pandemic. The ability to upload research to preprint databases immediately was invaluable to the timely sharing of information about COVID-19 during the rapidly evolving pandemic. However, many bad actors abused the system to push out less credible preprints.
Some of those papers never made it past peer review, but the damage was done. For example, a large-scale preprint study from November 2020 set off a wave of false claims that ivermectin can treat COVID-19. The study was cited in other ivermectin studies only to be withdrawn eight months later, never having completed peer review.
“This situation further fueled the uncontrolled and unfiltered spread of misinformation surrounding COVID-19, appropriately termed an infodemic,” said Liam Brierley, an epidemiologist who now works as a research fellow at the University of Glasgow, in a 2021 Lancet article.
“Even though it is tempting to think that public misinformation can only originate from non-scientific sources, misrepresented preprint research needs to be recognised as a potential driver of such infodemics.”
Check the affiliations
The first thing to look at when assessing a study is the authors. A reputable article should list the researchers’ names, titles, affiliations, and contact information.
Credible authors are affiliated with legitimate research institutions such as universities, disease research organizations, or government agencies. A study about vaccines or infectious disease written by people affiliated with an anti-vaccine organization is a major red flag.
Next, check the authors’ area of expertise. An immunologist, microbiologist, epidemiologist, or clinical researcher publishing a study about a disease outbreak is reasonable. An economist or “entrepreneur” publishing such a study is less so.
Finally, look at who funds or sponsors the research. Any credible research will list its funders and any conflicts of interest or biases the authors may have. Research funded or sponsored by anti-science organizations is not credible.
Similarly, studies where the researchers have obvious conflicts of interest or financial stakes in the results should be read with additional skepticism. For example, someone who sells vaccine “detox” products is not a trustworthy source about vaccine safety.
Language red flags
Scientific studies are not generally written to be flashy or exciting. They often use specialized, technical language and avoid grandiose phrases. Some red flags to look for when reading a study are:
- Nonspecific language like undefined “toxins” or “chemicals.”
- Alternative health and wellness buzzwords like “miracle cure” or “clean eating.”
- Conspiratorial or inflammatory language, particularly attacking individuals and institutions instead of ideas.
- Informal or unprofessional language and poor editing.
- Unscientific terms like “turbo cancer” or “vaxxed.”
To avoid being fooled by fake studies, use trusted sources such as universities and research institutions, science organizations, and peer-reviewed research from experts.