The cost of the rejection-resubmission cycle

ResearchBlogging.orgRejection is one of the unpleasant but inevitable components of life. There are positive components to rejection: they build character, they force you to deal with negativity and sometimes they force you to change your life to avoid future rejections. In science, if your submitted manuscript is rejected by the journal you submitted it to, Calcagno et al. reported that the way the authors change the manuscript has an effect on future citations this manuscript receives. The effect is on the order of ~0.1 citations, tiny.

So much about the benefits. How about the costs? On the whole, peer-review costs an estimated 2.2 billion € (US$ ~2.8b) annually (Research Information Network, 2008), so re-review costs money. How much, I don’t think anybody knows. However, revise, resubmit and re-review costs time as well. Time in which the article might have been cited. Çağan H. Şekercioğlu has now provided us with a rough estimate of the citation cost of the rejection-resubmission cycle (toll access). The gist of his analysis:

On average, each resubmitted paper accumulated 47.4 fewer citations by being published later, with an overall opportunity cost of 190 lost citations.

Compared to these costs, the estimated benefit of ~0.1 citations appears laughable. It is quite likely that Casey Bergman is correct in his assessment of the reason why Calcagno et al. was published in Science:

Nature and Science have a vested interest in making the case that it is in the best interest of scientists to submit their most important work to (their) highly selective journals and risk having it be rejected.  This gives Nature and Science first crack at selecting the (what authors think is their) best science and serves to maintain their hegemony in the scientific publishing marketplace.

Çağan’s analysis shows quite unequivocally: the citation costs clearly outweigh the potential benefits of the rejection-resubmission cycle. I wonder if he submitted it to Nature and Science as well, before publishing it with Current Biology and how much that might have cost him?

Çağan H. Şekercioğlu (2013). Citation opportunity cost of the high impact factor obsession Current Biology, 23 (17) DOI: 10.1016/j.cub.2013.07.065

Showing 2 Reviews

  • Placeholder
    Dorothy Bishop
    Originality of work
    Quality of writing
    Confidence in paper
    1

    Thanks for this interesting analysis. Commenters are asked to say whether the data support your conclusions, and I think they do, insofar as it is clear that the time spent submitting and resubmitting papers is a considerable burden to authors that far outweighs any measureable improvement for resubmissions.

    I have one quibble with the original analysis of Calcagno et al, whose work I had not previously seen. They argued that the (very small) effect they found for more citations of resubmissions was 'challenging to explain'. They went on to say 'Several mechanisms could be involved, but perhaps the most likely explanation is that inputs from editors, reviewers, and the greater amount of time spent working on resubmissions significantly improve the citation impact of the final product.' I didn't find this very convincing: presumably most of the papers in the Calcagno et al study had revisions in light of reviewer comments, regardless of whether they were rejected and resubmitted or accepted by the journal of first choice. This means their citation advantage can't be taken as representing 'added value' of having work reviewed - only the marginal increase from having a different set of reviewers look at the work. 

    I was more concerned that their analysis has intrinsic bias as it excludes a subset of papers: those which are rejected on first try and never resubmitted. It could simply be that an author decision to resubmit reflects their confidence in the paper. It is, after all, as these analyses emphasise, a kind of game where authors have to judge whether the extra effort of submission is worth it.

    Finally, this topic attracted my interest because I have recently blogged about a case where the publication record indicates a reciprocal relationship between editors who were accepting one another's work without review. So editor of journal A was accepting papers from editors of B without review, and they were conversely accepting his papers without review. They completely avoided the costs associated with the rejection/resubmission cycle. Since they also engaged in considerable self-citation, this was a very effective strategy for boosting the H-index of all concerned. I guess this could in principle provide a natural experiment that could be used to test whether reviewing confers benefits, as one could compare the citations (excluding self-citations) for papers accepted in these journals without review, vs. papers in the same journals that went (as far as one can tell) through a normal review process. For further information see: http://deevybee.blogspot.co.uk/2015/02/editors-behaving-badly.html.

    I confirm I have no conflict of interest.

  • Placeholder
    Ashwin kumar
    Originality of work
    Quality of writing
    Quality of figures
    Confidence in paper
    0

    Come here guys and see this website homepage delete browsing history then you are realize that this is the learning place that you looking for.Get the knowledge through this and remove your search History and keep them safe to the others.Thank you.

License

This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.