How do we ensure that research is reproducible?

  1. 1.  School of Psychology, Newcastle University, NE1 7RU, England

TS Eliot predicted the current corruption of science by pervasive dishonesty of motive and communication when he described characteristic modern Men:  They constantly try to escape/ From the darkness outside and within/ By dreaming of systems so perfect that no one will need to be good.

Professional science has arrived at this state in which the typical researcher feels free to indulge in unrestrained careerism, while blandly assuming that the ‘systems’ of science will somehow transmute the dross of his own contribution into the gold of truth. It does not: hence the preponderance of irreproducible publications.

There is a massive evasion of responsibility at work. Scientists have allowed themselves to become mere puppets of external forces. The choice of research topic is dictated by the availability of funding, the methods are dictated by fashion, analysis and publication are dictated by considerations of security and status. The ‘scientist’ is merely a passive responder to such imposed factors.  

Science itself is now organized in groups; groups for which obedience is far more highly prized than truth-seeking and truth-speaking – indeed strict honesty is feared as likely to upset the apple cart; to derail ‘the team’ from its collective track of self-proclaimed progress.

Hype and spin are the order of the day; with even the most routine and mediocre bits of ‘stamp collecting’ medical research touting itself as a cure for cancer, ageing and (perhaps?) death itself. Such grandiosity is more-or-less enforced by the systems of professional evaluation. Of course, such claims are discounted (‘world class’ deflated to being of potentially national importance; ‘paradigm shifting’ as being understood to be just a little bit off the beaten track) – but at the cost of downgrading to insignificance those scientists who are either modest, or simply uninterested in their own prestige and instead devoted to their subject - to their work.

The most pernicious of these responsibility-evading ‘systems’ is peer-review; which is now often believed to be the very essence of science. So we have a massive, and interlinked, proliferation of voting committees that control access to all the important success-markers of modern research. Peer review for jobs and promotions, for grants and funding, for conference participation and journal publication – and for prizes and awards. All of them dished-out by ‘the usual suspects’ – those overlapping groups of ‘the great and good’ – a kind of market-fixing cartel who assume the right of deciding by consensus what counts as true and important.

Yet throughout the great days of science there was no peer review. Decisions were made by individual people. There was no closed-shop of opinion-formers who could include or exclude ideas and evidence by fiat.

Instead, science consisted of self-chosen, self-motivated, independent individuals who spontaneously grouped themselves into ‘invisible colleges’ who tacitly agreed to work together on projects and problems of mutual interest. These communicated by word of mouth, by letter, by lectures and more formal papers and books – the emphasis of all of these being on communication, not competition or reputation.

Membership of an invisible college was on the basis of being judged – by at least some of the other members - to have something worthwhile to contribute. There was no need for formal credentials - although they may be taken as prima facie evidence of competence – because one’s work was expected to speak for itself.  

They were ‘policed’ only in the sense of being self-policed – if a member was discovered to be dishonest, incompetent, careless – and he was unable satisfactorily to explain or justify his error, and if he failed to demonstrate repentance and promise reform… then that individual would be excluded from the group.

This exclusion need not be formal – but the other members knew that his work could not be relied upon; and the sanctions were withdrawal of cooperation (loss of access to new knowledge being shared among the group); and that the work of the unreliable researcher was simply ignored, as being something which did not elicit belief.

All this is very pragmatic, makes solid functional sense; but that is not its ultimate basis. What underpins this model of honest science is idealism.

Idealism in science means that pure understanding is being sought as the primary end, and everything else is merely a means to that end – and the means must not be allowed to interfere with the end. Such an ideal science is not intended to be useful (although true knowledge often is useful), it is not intended to be harnessed into new forms of power (although science has often been used, and abused, for power). The participants must have as their main motivation this ideal aspiration of knowledge about nature – as in the old phrase ‘natural’ science.

With such idealism, the absolute necessity for honesty – for truth in all things great and small (or what Jacob Bronowski termed ‘the habit of truth’) - is self-evident. Lying and misleading are simply counter-productive – a waste of everybody’s time and energies. And why would an idealistic scientist want, even for a moment, to waste his own time and energies or those of his colleagues in the search for knowledge?

So, a restoration of honest science seems to entail a fresh start, and an escape from the tyrannical constraints of ‘professionalism’. Unless he is a saint of integrity (and there are a few such persons – but never many) the true scientist cannot rely on his research as the means of generating income; but must become an amateur.

Of course, the amateur scientist needs to live – and if he does not have a private income or a patron, then he will need to generate his income by some other means: perhaps teaching, practicing medicine, law or religion, but in principle from any activity for which he has some aptitude or liking and can earn a sufficiency.

And that is the best possible future of science; that science again become the activity of those self-selected, self-motivated, honest enthusiasts who do it from pure love of knowledge.

Systems will never make science good; if dishonesty goes-in, then garbage will surely come-out. Only good scientists will make good science. And good scientists create their own informal but effective regulatory systems

Science will not then become perfect, of course! But it will become as good as good scientists can make it; and as reproducible as honest endeavour may devise.  

 

License

This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.