Reviewer Selection Handbook
75% of journal editors say that “finding reviewers and getting them to accept review invitations” is the hardest part of their job (2016 Publons’ Editor Survey). In order to further support editors with this issue, SAGE has developed the Reviewer Selection Handbook. The Handbook provides you with:
- A guide to the various ways in which you can search for potential reviewers (including Web of Science, PubMed, Publons, and JANE Tool)
- Information on SAGE’s ethics and best practice policies and recommendations
- Time-saving tips
- The Reviewer Selection Handbook aims to minimize the number of invites an editor must send for each manuscript, improve the quality of comments received and reduce time to decision for authors.
What makes a good or bad review?
- Reviewer comments are substantial for the first review (even if Minor Revision recommended) and any Major Revision recommendations.
- Reject recommendations don’t necessarily have to be substantial if there are fundamental errors/issues which have been noted by the reviewer.
- It provides an overview of the paper’s suitability for publication, followed by more detailed feedback.
- Ideally, the review should be easy to read and written in a logical order.
- The reviewers should list any specific edits (e.g. spelling) with a page and paragraph/line number.
- Will include the good points as well as bad, especially for Major Revisions, as authors will have a better idea of the aspects of the paper which are strong.
- Comments to the Editor don’t include any useful material for the authors that isn’t already in the Comments to the Authors.
- The reviewer agrees to review again….
- Vast majority of the time – any review that recommends Acceptance at first round review.
- Lacking detail at first review stage – even if the recommendation is for Minor Revisions, the reviewer should be able to justify why the paper is already suitable.
- Any major revisions without substantial detail and or/justification.
- When the overall recommendation doesn’t reflect the reviewer’s comments.
- Purely descriptive with no evaluation of content
- Reviewer asks authors to cite own papers with no justification
- Only grammatical/spelling changes requested.
Handling Poor Quality Reviews
Blank and poor quality reviews should not be used to inform a recommendation or decision.
There are two options for dealing with insufficient reviews on SAGE Track:
- Rescind and unassign the poor quality review and unassign the reviewer
- The benefits of this option are that the reviewer is informed if their review is not used, and it's very clear to the handling editor that poor quality reviews cannot be used.
- The disadvantages of this option are it's more time consuming, the reviewer may dispute their removal from the manuscript, and there is a chance that the editor will re-invite the same reviewer, which would add unnecessary time to the process and they will likely decline or submit another blank/poor review.
- Keep the poor quality review on the system and increase the number of reviews required by one (e.g. if set to '2', increase to '3')
- The benefits of this option are that the poor quality review remains on the system, so can be used as evidence if needed in future, there are fewer steps involved on the system, and the reviewer cannot be invited back to review that manuscript again.
- The disadvantages of this option are that the editor may keep trying to make a recommendation or decision using the poor quality review, and the reviewer is not aware that their review is not used to inform the recommendation/decision.
In general, especially for larger journals, the most effective option is to keep the review on the system and increase the required number of reviews by one. Rescinding the review may be preferable in certain circumstances.
Reviewer Selection FAQ
When I am selecting reviewers, why do some say "opt out", some say "opt in", and others don't say either?
When reviewers create their accounts, they are asked whether they “opt in“ or “opt out” to receive emails form SAGE and its affiliates about their products and services. This is an “attribute” associated with their accounts like other keywords, which is why it shows up in the search results.
You don’t need to worry about this, however. It’s due to an unusual system setup that the opt in/opt out option appears next to other keywords. Additionally, you are free to invite any reviewer no matter if they’ve answered opt in, opt out, or nothing at all.
Why can't I add a reviewer to the Reviewer List? Why is the "Add" check box greyed out?
If a user does not check that they wish to be invited for peer review in their profile, then they will be marked as an Excluded Reviewer.
What is an R score?
The R-score is the average score a reviewer receives from editors. At the bottom of a completed review, you will see two brief rating scales – one for timeliness and one for quality assessment, on a scale from 1 to 3. Simply give your score for the review and click Save. Each score a reviewer receives is collected and averaged to produce the overall R-score. The R-score is attached to a person’s account, and can be used in reviewer searches and reports to find and determine the best reviewers in your site. You will get the most accurate R-score results if every editor in the site rates reviewers once they complete their reviews.