‘There ain’t no rules around here! We’re trying to accomplish something!’
Thomas A. Edison (1847-1931), American Inventor
It is without any doubt that the scientific progress of continued innovation requires scientists having significant freedom to use their creativity to its full potential. This is the key to advance life sciences and to translate the development of novel drugs into clinical applications. Society should therefore abstain from imposing any regulations or policies that restrict the scientific development.
However, science progresses as hypotheses are generated and tested, building on existing findings and thereby furthering our collective knowledge. For such a process it is essential that studies are rigorous, reproducible and of sufficient quality. Each generation of scientists builds upon the theoretical foundations set by its predecessors. If those foundations are made of sand, then time and resources will be wasted in the pursuit of ideas that simply aren't right. Valuable science requires reliable, high-quality data.
The quality aspect of a research study can be divided into two parts: a fundamental scientific and a practical experimental aspect. The scientific part starts with a specific question or task. Once the question is asked, research follows and facts are gathered to either prove or disprove the hypothesis. However, even the most brilliantly reasoned working hypothesis will not lead to meaningful and reasonable results if not supported by flawlessly conducted and high quality experiments, which are bound to Good Research Practice (GRP) standards and guidelines (Figure 1).
Figure 1: Good Research Practices ensure the robustness, integrity and reproducibility of research findings when implemented at the beginning of non-regulated, basic research projects
Reproducible findings in basic research and robust pre-clinical data form the basis for future success in many different life science disciplines - a statement, that is especially relevant for drug discovery and development projects. Several steps within this process, e.g. clinical trials, have adequate ‘codes of conduct’ and are covered by GxP policies (e.g. GLP, GCP, GMP, etc.) that aim to protect trial participants and the integrity of research (Figure 2). However, these same standards cannot be applied to the non-regulated areas of drug discovery (mainly basic and pre-clinical research). There is a need for a specialized set of Good Research Practice conditions that specifically focus on study design, unbiased conduct, statistical analysis and transparent reporting.
Figure 2: Several steps within the scientific value chain are well controlled by GxP-based controls and standards (orange arrows). However, only when research study design and conduct at early stages of the research process (blue circle) are compliant with the principles of Good Research Practice will this lead to an increase in quality and integrity of preclinical and basic research data. These then can form a strong basis for successful drug discovery projects.
The lack of GRP procedures at the early phase of drug discovery projects can quite often explain their failure at later clinical stages, which makes it surprising that, even nowadays, the standards and guidelines for pre-clinical phase of drug development programs are not sufficient to ensure quality research and data robustness.
“Integrity without knowledge is weak and useless, and knowledge without
integrity is dangerous and dreadful”
Samuel Johnson (1709-1784), English writer, poet, editor and literary critic
Every step of the experimental process, from planning a study to data collection and cleaning, statistical analysis and interpretation must be monitored and controlled so that a high-quality outcome can be obtained. Crucially, this GRP-based quality guidance approach has to start at the very early planning phase of a research experiment. Only proper study design, which includes the incorporation of blinding and randomization steps, appropriate statistical power analyses and primary endpoint definitions, will ensure that meaningful and analyzable results are obtained, which may, subsequently, be used in follow-up studies.
"To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of".
Ronald A. Fisher (1890-1962), Biologist and statistician
To support scientists at academic or company-based research organizations to achieve robust and highly reproducible data, the PAASP GmbH (Partnership for Assessment and Accreditation of Scientific Practice) was founded in December 2015. PAASP provides consultancy and GRP-based solutions for the non-regulated, pre-clinical phase of research projects, with a special focus on reducing bias and increasing Good Research Practices at the initial study design phase of any project. This GRP guidance allows for the detection of problematic areas and provides unbiased information on whether the minimum data integrity standards required to obtain meaningful and relevant data have been met.
However, in the current hyper-competitive research environment, factors that work against reproducibility are strongly promoted and reinforced by our scientific culture, in which researchers feel the need to publish novel findings as often and as fast as possible, as career progression and the attraction of funding are mainly associated with publication records. This process is further exacerbated by publication-related biases, e.g. the favorable reporting of positive results over negative or inconclusive outcomes, which places an emphasis on novelty and comes at the expense of rigor and robust methods.
Ultimately, the number of publications in high impact factor journals has become the ‘currency’ to assess scientific productivity and performance - a false perception of quality.
To break this vicious circle, a fundamental question for any field of research arises: ‘How could scientists receive the recognition or appreciation they deserve for following Good Research Practices?’
An innovative alternative approach is to introduce and establish a research quality label, which seeks to promote rigor, integrity and reproducibility of experimental findings. Individual scientists, laboratories or whole institutions will receive this seal of quality when demonstrating that studies and experiments are conducted with the highest level of technical execution, accuracy and fidelity. Importantly, as unpublished research studies can form the basis of the approval procedure, this will take off the pressure from the need to publish and offers a novel way to evaluate scientists solely in the context of their quality of work and compliancy with Good Research Practice standards and guidelines. Being an integrated requirement for e.g. grant applications, quality labels will help to ensure that only meaningful basic research, worth building on, is produced. As a consequence, the ‘Number of publications’ will be replaced by ‘Quality’ as the unit to measure scientists’ achievements and performances.
The establishment of a quality label requires an independent assessment system, which can provide an unbiased verification that all essential quality criteria are fulfilled. Furthermore, specific quality standards and aspects have to be accurately and precisely defined, so that the evaluation process becomes transparent and plausible.
Here, based on its current work and function, the PAASP GmbH, as an independent authority, offers expertise, know-how and the impulse to drive this process from an idea to reality.
As a multilayer strategy that tackles the root causes around the reproducibility crisis, not just the symptoms, and together with the development of improved and more detailed global Good Research Practice standards and guidelines for non-regulated areas of life sciences, these approaches can produce credible, reproducible and translatable outcomes in biomedical life sciences.
This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.