Scientific studies are a first-class source of knowledge. However, the studies are plagued by biases, both when published and when disseminated and disseminated, which slows down progress.
In the field of social sciences we have witnessed major scandals, such as Sokal's. Or more recently, the so-called ya Sokal squared, regarding gender studies. But these problems, to a greater or lesser extent, have been detected in all areas of scientific literature.
Loss of trust in science
In short, the problem of disseminating good quality scientific knowledge is fueled by:
- Publication bias: publishing only positive studies
- Citation bias: citing only positive studies. In 2012, Anne-Sophie Jannot and a team examined 242 meta-analyses published in the Cochrane Database of Systematic Reviews between January and March 2010 that confirmed this.
- Distortion quotes. In 2010, Andreas Stang posted a review of the Newcastle-Ottawa scale, a scale used in meta-analysis to evaluate the quality of observational studies. Sometimes references are simply copied from one document to another. It's hard to know how common this is, but Pieter Kroonenberg, a Dutch statistician, discovered a nonexistent study that had been cited more than 400 times.
- Underutilization of evidence: not citing existing studies. The researchers Karen Robinson and Steven Goodman examined the frequency with which subsequent clinical trial authors reported similar clinical trials. They identified 1,523 essays and tracked how they had cited others on the same topic. Only about a quarter of the relevant trials were cited, which also constituted only about a quarter of the subjects enrolled in the relevant trials.
Within publication bias, in turn, different forms of bias have been identified:
- Time lag bias: in which trials with impressive positive results (large size, statistical significance) are published more quickly than trials with negative or equivocal results.
- Results reporting bias: reporting only statistically significant results or results that favor a particular claim, while other results have been measured but not reported.
- Location bias: publication of non-significant, equivocal, or unsupported findings in less prestigious journals, while studies reporting positive, statistically significant findings tend to be submitted to better-known journals.
A significant case was the one analyzed in 2015 by Michal Kicinski and colleagues, who examined 1,106 meta-analyses published by the Cochrane Collaboration on the effectiveness or safety of particular treatments. For meta-analyses that focused on efficacy, positive and significant trials were more likely to be included in the meta-analyses than other trials. In contrast, for meta-analyses that focused on safety, "results that provided no evidence of adverse effects were on average 78 percent more likely to enter the meta-analysis sample than statistically significant results that showed they existed." Adverse effects".
The existence of these biases is not only a problem that affects the quality of scientific literature, but is undermining the integrity of science, allowing myths or half-truths to flourish more easily. Once we have diagnosed the problem, it is imperative to get to work to mitigate it and, by extension, reinforce the foundations on which we build the edifice of science.
–
The news
The dissemination of scientific knowledge is plagued by biases and that is a problem
was originally published in
Xataka Science
by
Sergio Parra
.