COPE (the Committee on Publication Ethics) provides advice to editors and publishers on all aspects of publication ethics. To help fulfil that remit, it produces a number of guidelines that are freely available on its website http:// publicationethics.org/. On 22 March 2013, COPE published its most recent set, Ethical Guidelines for Peer Reviewers, launching them at the annual European Seminar in London. They are available at http://publicationethics.org/ resources/guidelines under the Creative Commons Attribution-NonCommercial-NoDerivs (CC BY- NC-ND) license http://creativecommons.org/licenses/by-nc-nd/3.0/.
Why are the guidelines needed?
Peer reviewers play a central and critical part in peer review – the process couldn’t function without them – but too often they come to the role without guidance and may be unaware of their ethical obligations. Few researchers receive training in peer review, but it is something many would like to have. For example, in the large international survey on peer review carried out by Sense About Science in 2009 (Sense About Science, 2009; Mulligan et al., 2013), 56% of researchers felt there is generally a lack of guidance on how to review papers, and 68% thought formal training would improve the quality of reviews (73% of those under the age of 36 years). The lack of guidance and training becomes even more surprising when one considers the scale of scholarly journal peer review: an estimated 1.8 million articles are now published every year in about 28,000 peer-reviewed scholarly journals (Ware and Mabe, 2012), so it follows that many millions of reviews are carried out annually. Peer- reviewer reports are central to decisions on what is published and where, and because publication records play a large part in the job, promotion, and funding prospects of researchers, peer reviewers have enormous impact on the careers of other researchers.
Some recent cases of reviewer misconduct have also highlighted the need for guidelines that lay out clearly the expected standards to which peer reviewers should adhere, and what is considered unacceptable behaviour. In 2012 there were at least three cases of “fake reviewers,” where the suggested reviewers provided by authors were not what they appeared to be. Some authors provided either false identities (and emails) which were them or their colleagues, or the names of real people but created fake email accounts for them which they or their associates had access to (see the “faked emails” category on the blog Retraction Watch, http://retractionwatch.wordpress.com/, for the full stories). The reviews were done by the authors and/or their associates under the false identities and returned to the journal. Worryingly, there were not just a few isolated incidents at a single journal (although one did have a large number). The cases involved different journals, different publishers, different disciplines, and a large number of papers. In the case of one author, 28 papers had to be retracted (Oransky, 2012a), with notices that stated:
“The peer-review process for the above articles was found to have been compromised and inappropriately influenced by the corresponding author … as a result the findings and conclusions of these articles cannot be relied upon.”
Suggested reviewers can be very valuable to editors, but it is critical that journals run checks to ensure not only that those people are suitable reviewers without any conflicts, but that they actually exist, and if they do, that their contact details are legitimate. One of the publishers whose journals have been involved in the above retractions has acknowledged that “the integrity of the peer review process should have been subject to more rigorous verification to ensure the reviews provided were genuine and impartial,” and has updated its guidance for editors to advise that reviewers other than just those suggested by authors should be involved in the review of manuscripts (Oransky, 2012b), which is sound advice. The integration of ORCID (http://orcid.org/) into online manuscript submission and review systems will also help verify the identity of authors and reviewers.
Improper conduct in all types of peer review is generally viewed as unacceptable, and this includes the actions of reviewers. For example, new guidelines from Research Councils UK (RCUK, 2013) note that unacceptable research conduct includes (Section 3, p7):
“Improper conduct in peer review of research proposals or results (including manuscripts submitted for publication); this includes failure to disclose conflicts of interest; inadequate disclosure of clearly limited competence; misappropriation of the content of material; and breach of confidentiality or abuse of material provided in confidence for peer review purposes.”
“Misrepresentation” is also listed as unacceptable conduct in these guidelines, which would include the behaviour by the authors and “reviewers” in the above “faked emails” cases.
As well as providing an ethical framework for reviewers, it was felt the new COPE guidelines would be a resource junior people could use to present to more senior people – such as editorial assistants to editors, and early-career researchers to their group leaders – when they find themselves in situations they have ethical concerns about but which they don’t know how to address or rectify.
The process to the guidelines
We started by setting our aims for the guidelines. They should:
- state the basic principles and standards
- be generic – applicable to all disciplines
- all models of peer review, and all journal business and access models
- be comprehensive but easily understood, both internationally and across all career stages
- be a reference and resource for journals and editors in guiding and educating their reviewers
- be suitable as an educational resource for institutions in training their students and researchers.
The draft guidelines were ready late January 2013 and were then posted on the COPE website for a period of three weeks of community feedback. Thirty-six individuals and groups provided comments, many of which were thoughtful and valuable. The draft guidelines were then revised to address the feedback and clarify things that had caused confusion or misunderstanding. A number of issues had to be taken to Council for discussion and decision, either because there were a number of possible options or because they’d proved controversial. The guidelines start with the basic principles to which peer reviewers should adhere and are then organized into sections: expectations during the peer-review process (on being approached to review, during review, when preparing the report) and expectations post review (because obligations don’t stop when reviewers submit their reviews).
What were the main issues in the feedback period?
There were many positive and enthusiastic comments, welcoming the guidelines and stating what a great resource they would be. There were some concerns about the length of the document and that researchers wouldn’t read them, but we felt it important that the guidelines be comprehensive and spell out clearly the ethical issues and requirements. A few points were felt to be too prescriptive and so some flexibility was introduced. For example, it shouldn’t be a requirement for reviewers to declare if they become aware of the identity of the authors during double-blind review; rather they should only “notify the journal if this knowledge raises any potential conflict of interest.” It was also felt it shouldn’t be a requirement that reviewers declare if they’ve already reviewed a manuscript for another journal; rather they should “review afresh any manuscript they have previously reviewed for another journal as it may have changed between the two submissions and the journals’ criteria for evaluation and acceptance may be different.”
Other issues came up. One concerned use of the word “timely” because the interpretation of what is considered timely varies so much between journals. So use of this word was carefully assessed and textual changes made where appropriate, e.g. “peer reviewers should only agree to review a manuscript if they are fairly confident they can return a review within the proposed or mutually agreed time-frame, informing the journal promptly if they require an extension.” Another issue was the suggestion that all reviewers should be required to have an institutional email address. This wasn’t added to the guidelines because it is clear there are people actively involved in scholarly publishing and research integrity issues who work outside of the institutional framework. A good suggestion concerned reviewing work from authors not writing in their native language. Such authors sometimes feel there are issues of bias or hostility in the reviews they receive because of language issues. The following was added in response:
“Peer reviewers should be aware of the sensitivities surrounding language issues that are due to the authors writing in a language that is not their own, and phrase the feedback appropriately and with due respect.”
Journals shouldn’t send out to reviewers manuscripts in which the language is too poor for the work to be understood – it’s not fair to either the reviewers or the authors (Hames, 2007, pp38-39) – but from comments I hear from researchers, it’s clear that there isn’t always adequate screening before manuscripts are sent out to reviewers.
What were the controversial issues?
Involvement of junior researchers
The original draft stated “Peer reviewers should not involve anyone else in the review of a manuscript without first obtaining permission from the journal.” Quite a few commentators thought that the case of junior researchers should be exempt from this because supervisors frequently involve members of their groups in the review of manuscripts they’ve been sent without telling the journals involved. Even though journals know this occurs (“tolerate it,” as one commentator put it), this wasn’t felt to be ideal and concerns were voiced in the feedback about lack of transparency and appropriate accountability, and a blurring of the boundaries (“if you can delegate to someone in your lab, why not to anyone?”). COPE Council felt that it should be made explicit that permission from journals should be sought by researchers wanting to involve their junior colleagues in reviewing, especially as there are related issues with lack of appropriate credit. When junior researchers review for their principal investigators (PIs) without journals knowing about this, their efforts can’t be acknowledged. They can’t therefore receive any of the “rewards” journals give their reviewers – even the simplest and most usual one of featuring the names of their reviewers in an annual list – or be recognized in their own right in journals’ databases and so build up a reviewing record with journals. This is one of the most common issues I get asked about by young researchers: how can they stop this happening and how can they persuade their PIs to tell journals they’ve helped co-review manuscripts? Hopefully now they will be able to point to the COPE guidelines, which say:
“Peer reviewers should not involve anyone else in the review of a manuscript, including junior researches they are mentoring, without first obtaining permission from the journal; the names of any individuals who have helped them with the review should be included with the returned review so that they are associated with the manuscript in the journal’s records and can also receive due credit for their efforts.”
Editors acting as reviewers for manuscripts they are handling in blind review
This was the most controversial issue. The original draft stated that if editors decide themselves to review a manuscript they are handling, for example if they are having difficulties finding enough expert external reviewers and they have the relevant expertise, they should do this transparently and not via an anonymous review, i.e. one that looks as though it has been done by an independent external reviewer. Various reasons were given in the feedback why editors should be able to review anonymously: that cases where it’s difficult to find reviewers will take even longer; if they can’t do it anonymously they may be constrained in their review and not be as rigorous as they would otherwise be; taking away anonymity will discourage editors reviewing and result in delayed or even no decisions for some manuscripts.
But is it ethical? I don’t think so. Reviewers have editors to oversee them and make sure their reviews are appropriate, conflicting interests are taken into account and comments are evaluated and moderated. When editors act as their own reviewers there isn’t anyone to oversee and moderate them. It’s also misleading the authors, who think they’ve received, for example, two independent reviews and a set of editor comments, whereas in reality they’ve received only one independent review and two sets of editor comments. What if there are no other reviewers? When researchers find out about the practice of “ghostwritten” reviews, it erodes trust and confidence in peer review (Vines, 2012). Being an editor isn’t always easy, and there has to be transparency and accountability in all situations. If an editor needs to carry out a proper review of a manuscript they are handling, they should tell the authors this and why, putting their review either in the editorial correspondence or onto the online system, but with their name on it – this means the other reviewers can see this in those journals where reviewers don’t get to see editorial correspondence.
What we ended up with after Council discussion was the following, which recognizes that it is acceptable for editors to act as reviewers for manuscripts other editors are handling at their journal:
“Peer reviewers should if they are the editor handling a manuscript and decide themselves to provide a review of that manuscript, do this transparently and not under the guise of an anonymous review if the journal operates blind review; providing a review for a manuscript being handled by another editor at the journal can be treated as any other review.”
It is hoped that the guidelines will be widely disseminated and used by a number of communities: publishers in training and helping their editors, journals and editors in guiding their reviewers, and universities and research institutions in training their students and researchers. The guidelines are also very much a “living” document and will be revised at future stages in response to feedback received.
(This article is based on the talk given by the author at the COPE European Seminar, 22 March 2013, to launch the guidelines.)
1. COPE Ethical Guidelines for Peer Reviewers, v1 March 2013, by Irene Hames on behalf of COPE Council http://publicationethics.org/files/Ethical_guidelines_for_peer_ reviewers_0.pdf accessed 7 April 2013
2. Hames, I. (2007) Peer Review and Manuscript Management in Scientific Journals: guidelines for good practice. Blackwell Publishing in association with ALPSP http://eu.wiley.com/WileyCDA/ WileyTitle/productCd-1405131594.html
3. Mulligan, A., Hall, L. and Raphael, E. (2013) Peer review in a changing world: an international study measuring the attitudes of researchers Journal of the American Society for Information Science and Technology 64, 132-161
4. Oransky, I. (2012a) Retraction count grows to 35 for scientist who faked emails to do his own peer review, Retraction Watch blog, 17
September 2012, http://retractionwatch.wordpress.com/2012/09/17/retraction-count-for-scientist-who-faked-emails-to-do-his-own-peer-review-grows-to-35/# accessed 1 April 2013
5. Oransky, I. (2012b) South Korean plant compound researcher faked email addresses so he could review his own studies, Retraction Watch blog, 24 August 2012, http://retractionwatch.wordpress.com