Contest Winners: How to improve reproducibility


“How would you improve reproducibility in science?”  It’s not an easy question, but it’s important. The Laura and John Arnold foundation has been integral in trying to answer this by funding organizations like the Center of Open Science and others and we’re happy to have partnered with them to hold the second essay competition on the Winnower, which we’ve just concluded judging.  We had many great entries (read them here) from around the world and are happy that this is not just a competition but a resource that will continue to be useful now and in the future as we tackle improving reproducibility in the sciences.  So without further ado, we give the winners of the contest: Anne Jorstad & Konrad Hinsen.

Verifiable research: The missing link between replicability and reproducibility

Konrad Hinsen, Centre de Biophysique Moléculaire, CNRS, France

“To err is human. Scientists being human, they make mistakes. Many if not most of the rules for doing science are designed to weed out mistakes. Reproducibility and replicability are recognized as playing a central role in this process. But a lot of confusion remains about the difference between these two labels and the relation between them. In this essay, I will explain why replicability is the foundation on top of which reproducibility can be constructed, and introduce verifiability as the missing link between them, which deserves particular attention in the context of computer-aided research.” [Read more]

From the judges: “Interesting and informed, this essay very clearly follows one idea from definitions to implications.”

Leveraging Doctoral Requirements to Promote Reproducibility

Anne Jorstad, Swiss National Science Foundation, École polytechnique fédérale de Lausanne

“It is commonly accepted that there is a crisis of reproducibility in science [1].  But how can this trend be reversed when bibliometrics such as the h-index tend to encourage quantity and incremental research [2], often at the expense of quality?

In an ideal world, researchers would only publish when they have found a meaningful result, be it a new insight or a better way to solve a problem. Authors would share all relevant information to make their work as reliable and reproducible as possible, promoting transparency and open data.  They would thereby be producing fewer papers, and those papers would represent higher quality science.  These studies must be externally confirmed as part of the scientific process, and so in this ideal world, every researcher would publish a reproducibility study of another researcher’s work between every original paper of their own.  Unfortunately, the current system rewards the researchers with the most and most cited original publications, and so until the system changes, it is unreasonable to expect researchers to change on their own.” [Read more]

From the judges: “I think it has the potential for greatest impact, and it is focusing on the next generation”