What is it?
The Brazilian Reproducibility Initiative is a multicenter initiative to estimate the reproducibility of Brazilian biomedical science. Supported by the Serrapilheira Institute, our goal is to reproduce a sample of 50 to 100 experiments of Brazilian articles in 3 to 5 different areas of the life sciences.
Why focus on Brazilian articles?
Because knowing how much of our published research is reproducible is vital to improve the country’s science. As such efforts are not routinely performed by funding agencies, we believe that the scientific community must face this task as a collective duty.
What kinds of experiments will be replicated?
The selected experiments will include models and methods that are used by multiple laboratories in the country. These methods will be selected on the basis of a wide survey of the most commonly used techniques in Brazilian publications.
How will experiments be selected for replication?
We will base our work on a random sample of Brazilian articles, selecting individual relevant experiments in 50 to 100 articles. To decrease the risk of bias, the article of origin of each experiment will only be revealed at the end of the study.
How will replications be performed?
Each experiment will be reproduced in 3 different laboratories, with preregistered protocols developed to be as close as possible to those of the original study. Measures to decrease bias such as randomization of samples, blinding of experimenters and sample size calculations will be used to ensure rigor.
When will a replication be deemed successful?
We will consider that an experiment has been successfully replicated when its effect size is within the 95% prediction interval defined by the effect sizes obtained in each of the three replications.
If an experiment is not replicated, does it mean that the original result was wrong?
No. Many factors can lead to non-reproducibility, including uncontrollable factors related to individual laboratory conditions. That said, performing replications in different labs will allow us to estimate interlaboratory variation for each method, and thus judge whether the original effect is within the expected range of effects.
What will be done with the data?
Besides estimating the reproducibility of experiments in specific areas of Brazilian science, we will use the database to study if there are variables in the original studies that can predict successful replications, in order to identify factors that can increase the reproducibility of a finding.
Does the study plan to evaluate individual researchers?
No. On the contrary, the initiative will not include more than one experiment per article or research group, and is thus insufficient to judge them individually. Our goal is to estimate the general reproducibility of experiments in Brazilian biomedical science and propose ways to improve it.
How can I participate?
Starting on June 2018, we will announce the models and methods to be included in our replication efforts. At that time, we will open an online form so that labs interested in participating in the project can sign up for the specific methods they can perform. If you wish to keep in touch until then, leave us your e-mail and we will send you ongoing news about the project.
What do I gain by being a collaborator?
All participating laboratories will be coauthors in the Brazilian Reproducibility Initiative, which will sign articles derived from the project as a consortium. The costs of experiments will be financed by the Serrapilheira Institute, and we are studying the possibility of paying an overhead value to collaborating labs.
What does Brazilian science gain with this effort?
At the end of the study, we will have performed the first systematic study of research reproducibility at a national level, at least to our knowledge. Independently of the results, the process of generating these data will allow us to reflect on our individual and collective practices, helping to place Brazil at the forefront of the international effort to develop more reliable scientific practices.