Peer Review: How to Do it Right?
Peer review: how to do it properly
Shanty Shahron shared some expert advice on how to be a useful, scientific and professional person when reviewing a journal
1) Become professional. This is called peer review for a reason. You, putative reviewer, are partners. If you don't do it for them, why do they have to do it for you? This is a core part of your work as an academic. This shows that you are part of the academy and are willing to engage in interactions that make the profession work. Reviewing is a great way to balance out the literature and a wonderful way to sharpen your own writing.
2) Fun. If this journal is really bad, suggest a rejection but don't get involved in home ad comments. Rejection must be a positive experience for all. Don't say things in a reputable peer review journal that you won't say to that person's face in the presentation or in the bar after the conference.
3) Read the invitation. When you receive an email inviting you to review a journal, most reputable international journals will provide a link to accept and or reject. Don't respond to the editor with a long apology about how you want to do it but your cat has a kitten and you have your own paper to do, plus classes to teach and however won't prof von Juntz at Miskatonic get better? Click. That. Link. An invitation tells you when. This can also give you specific instructions, so follow this.
4. Be helpful. Suggest to the author how to overcome the deficiencies you identified. This is the easiest thing in the world to make a hole in something. It's usually more difficult to suggest ways to fix it. Reviews are more than suggestions for revising, rejecting, or accepting. It must be meaningful. It should guide the writer about what is good and what is not so good as you see. If it's too short, then maybe it won't do it. So be polite. Explain what is happening in your thinking. Suggest alternative approaches.
5) Be scientific. Your role is a scientific colleague. That is not an editor either in the sense of proofreading or decision making. Don't re-fill the review with editorial and typographic issues. If the journal is full of errors, tell the editor and give an example. Concentrate more on showing the added value of your scientific knowledge and not so much on missing comas etc. If as part of your revision, you think that the journal must be edited professionally (as I sometimes do with myself), then say so. The caveat to this is that journals (and indeed reviews) are acts of communication. If it's built so badly that it fails in its communication role, then tell me that. Remember that in the end this journal is not about style but substance, unless style is blocking.
6) On time. There is no point in complaining about how slow the journal publishing process is if you are part of the problem. When you agree to review a journal with a given timeline (unless there is a very good reason), you must obey it. Believe it or not, editors keep track of who reviews what and when. We must balance the natural tendency to give more reviews to those who do the most, with the realization that people do this are basically pro bono and have limited time. So the time period that we provide is designed to be on time but a bit stressful. Deadline is good. Stay with them.
7) Be realistic. Be realistic about the work presented, the changes you suggest, and your role. You, the reviewer, are part of the process. You do not have a final decision about determining the paper. I, as an editor, have it. Sometimes editors ignore suggestions from reviewers (hopefully with good reason). You can, and in this case get involved, in a dialogue with the editor about why - ideally this is a learning opportunity for all. Sometimes this override is because the bar set by the reviewer is too high for the journal. Data may not be available, the suggested paradigms are not appropriate. This might be a useful suggestion for other journals but each journal, or should, one main idea.
8) Be empathetic. Think about the best review you get in guiding a journal forward. Then think of the worst. Which do you want on average? Then place yourself in the position of the writer whose journal you reviewed. Where along the scale will your review fall? What is around appears and therefore ensures that your review is scientific, useful, and polite is a good idea.
9) Open. Unless it is a review for the Journal of Incredible Specialization, specialists and generalists both have a role to play. Editors, especially public interest journals, will try to get specialized and more general reviewers. Saying "this is not my area" is rarely the reason, especially when you recently published a very related journal. Saying "I'm just one of the writers" in answer, don't cut it either. The editor tries to balance the review. That's why we asked a number of reviewers. We might want a generalist, a subject specialist, someone with experience in methodology and someone whose work is being criticized. If we ask you, assume you have a valid and useful role to play.
10) Organized. Reviews, like paper, are communication. Because it needed a logical structure and flow. It is impossible to criticize the paper for logical holes, grammar, poor structure etc. if your own criticism is full of these deficiencies. Draft the review as you proceed, then re-create. Most publishers provide a brief guide to preparing peer reviews on their websites. Read some of them and follow the main principles. Initially, provide an overview of one or two short sentences from your review. Then provide feedback on the following: the structure of the paper, the quality of the data sources and the investigative methods used, the specific problems regarding the methods and methodologies used (yes, there are differences), the logical argument flow (or lack thereof), and the validity of the conclusions pulled out. Then comment on the style, voice and lexical concern and choice.