Peer review: how it works

Summary

• A peer reviewer is an expert in the area covered in a manuscript that they are evaluating.
• The whole process can take days or even months to complete. There are so many factors at play that can determine the length of the peer-review process.
• There are various types of peer review. Certain individuals are kept anonymous to ensure a fair and unbiased evaluation of a manuscript.
• It’s not clear on who can be a peer reviewer. It could be a PhD student evaluating the work of an established academic.
• It’s not perfect. But without peer review, things could be much worse.
• As an author, do not take the peer review comments personally and avoid being defensive in your response.
• As a peer reviewer, treat the job as a duty to your field. There are those out there that look up to you.

In a nutshell

Peer review is the evaluation of scientific, academic and professional work done by other experts in a matching field. A majority of the big publishers and journals depend on the peer review process to maintain their high-quality output. The reputation of these giants hinges on robust and honest scrutiny of all submissions through this process.

The peer-review process has got its flaws, which I will touch upon in this blog, but it’s widely viewed as the best way to filter out the good research from the not so good. Experts from around the world are invited to do the job. Some may view the task as a duty to their field and others may see it as an inconvenience. Personally, whenever I undertook the role of a peer reviewer, I treated it as an honour and a solemn duty to evaluate someone’s work for the betterment of my own field. I saw it as giving back. But how does it work?

How it works

I discuss at length in my blog ‘Manuscript publishing: a beginner’s guide’ how to write a manuscript for submission to an academic journal. I also briefly mention the peer review process, but I don’t go into much detail. The peer-review process can last anywhere between days or even months. It all depends on how fast an editor can secure the peer reviewers, the peer reviewers’ availability, the number of issues uncovered by the peer reviewers, the speed at which the author addresses these issues and much more. In a nutshell, the peer review process looks something like this:

1. The author submits their manuscript to a single journal for publication.
2. Then, a quick assessment of the manuscript is done to determine its suitability for the journal. If it is suitable, the manuscript is passed on to the next stage. If not, it’s rejected.
3. At this stage, the editor does a quick evaluation to see if the manuscript is worth proceeding on to the peer review stage. If not, it’s rejected.
4. Peer reviewers evaluate the manuscript and send a report to the editor with a recommendation to either accept, accept subject to revision or reject.
5. If revisions are needed, the author is informed of what changes are required. The authors are then invited to resubmit a revised version of their manuscript.
6. The revised manuscript is then checked by the editor and/or peer reviewers. If they’re satisfied, the manuscript is accepted. If they want more revisions made then steps 4 to 6 are repeated. The only other outcome is rejection.

Peer review can work in different ways

Now we’ve established the three parties involved in the publication process: the author(s), editor and peer reviewers. But they don’t always know each other’s identity. Every journal has its own policy regarding this and they usually indicate in their Guide for Authors the type of peer review process they go for.

Single-blind review

The single-blind review process is one I’m quite familiar with myself and is the most common. Essentially, the names of the peer reviewers are not revealed to the author. Keeping the peer reviewers anonymous can help to make their decisions impartial and stop any unnecessary influence by the author. But the drawback is that some peer reviewers may see their anonymity as an opportunity to be excessively harsh and unfairly critical.

Double-blind review

In this scenario, both the peer reviewers and author are not privy to each other’s identity. Only the editor knows the identity of the peer reviewers and author. This helps to further alleviate any bias peer reviewers may have towards an author. It also makes sure that a manuscript is evaluated based on its merit and quality of the content as opposed to the reputation of the author.

Triple-blind review

Triple-blind review means that the peer reviewers are anonymous and the identity of the author is not revealed to the peer reviewer and editor. This is to further promote impartiality and prevent unfair evaluation of an author’s manuscript. But this level of anonymity is really difficult to keep. Both the peer reviewers and editor can figure out the identity of an author through their writing style, subject area, self-citations and various other clues. It’s not hard.

Open review

Open review makes the whole process transparent. Essentially, the peer reviewers and author know each other’s identity. In this case, the peer reviewer names can appear on the publication and can even include the peer review reports as well as the responses by the editor and author. Under certain circumstances, the manuscript can be checked by the community for feedback.

Now, this approach can be divisive, to say the least. On one hand, some feel that it can promote accountability for the peer reviewers and encourage unbiased scrutiny. On the other hand, others may see it the other way where peer reviewers feel intimidated with giving an honest evaluation for fear of backlash from the author.

Who can be a peer reviewer?

It can be vague

The prominent publisher, Wiley, describe on their website that a peer reviewer is an expert in the area of research covered in a manuscript. The word ‘expert’ is a bit arbitrary in this context, I discuss at length the definition of an expert in the role of a consultant in my blog ‘What do consultants do’. In this blog, I mention that Malcolm Gladwell believed that it takes 10,000 hours to become an expert in any field. The term expert can be subjective, but the fact of the matter is no one has a strict definition or criteria for becoming one.

So, on top of being an expert, Wiley also states that a good peer reviewer can be at any stage of their career. The requirements still seem quite vague. From my own personal experience, it seems the definition of an expert is down to the editor’s discretion. I was first invited to peer review a scientific manuscript shortly after publishing my first original research article. At the time, I was a PhD student. Perhaps publishing my first article was enough proof that I was an expert. Or maybe it’s because my contact details were on that article and I conveniently accepted when no one else would.

The reality

Normally, when an author submits their manuscript for review, they can nominate experts for the job of the peer reviewer. The editor then invites the nominees to peer review the manuscript. But there are times that these invitations are declined for whatever reason. So, the editor has to find alternatives for the job and may scour their contact list or even do an online search for potential candidates. You might end up having a PhD student reviewing your work. I know of instances where academics accept an invitation to be a peer reviewer and delegate the work to one of their research students. Regardless, you won’t always have the top names in your field scrutinising your work. Seeing as how peer review is a pro bono task, many don’t see the incentive. Others can see it as a duty.

Certification

Although it’s not mandatory, you can become a certified peer reviewer. Publons Academy offers a free online peer review training course aimed at teaching the fundamentals of the peer-review process. I do not work for Publons. You get assigned a mentor of your choosing and you must complete a series of online modules. Once you’ve completed the course you become certified. This can be a good way to learn how to peer review if you’re new to the whole thing. But editors are not exclusively looking for graduates of this course to do the honours. But perhaps certification may be a thing in the future.

My thoughts

The peer-review process is not perfect. Published work can still contain errors and inaccuracies. I have personally peer-reviewed a scientific manuscript and noted some of the mistakes to the editor. A short time later, I discovered that the article had been published without any of my comments being addressed. The mistakes remained in the article. The trouble with this is that other researchers may cite or even base their work off this article not knowing that many errors lie within. The scary thought is that there are probably many articles out there containing inaccuracies that have gone undetected or worse because of politics.

Another issue is that authors can take their rejected manuscript to another journal and have a new set of experts evaluate their work. Authors can repeat this approach until the manuscript is accepted.

Despite the imperfections of the peer review process, it can be argued that it’s not worth abandoning. It’s safe to say that things could be a lot worse without it. However, improvements can be made:

• Put measures in place to prevent authors from resubmitting their work to other journals until it is accepted. Journals could keep a record of rejected manuscripts under an author’s name that is easily accessible by editors of other journals.
• Experts could be put through formal peer review training to promote good practice.
• Recognise the efforts and contributions of peer reviewers through some kind of reward scheme.

Some of the high impact factor journals require authors to submit data files into an online repository for readers to further scrutinise the results themselves. This level of transparency really helps to create accountability for the authors. It will ensure that accurate and high-quality data is only presented.

Advice for authors

Don’t take the peer review comments personally. I know it’s easier said than done and they can sometimes come across as harsh but try to be objective in your interpretation. If you feel the criticism is unfair then seek the unbiased opinion of a more experienced and senior researcher.

I always liked to present my response to the peer reviewers in a table. I find it’s quicker and easier for the peer reviewers to see if and how you have addressed each of their suggestions. As an example, below is my responses to the single-blind peer review comments I received for my published article. I made sure my responses were respectful and professional. Avoid the urge to be defensive or even aggressive. It never helps. Be patient. One of the peer reviewers mistakenly believed that I did not reference one of my figures in the text. Peer reviewers are humans after all and can make mistakes so I didn’t see the need to make an issue of it.

Example of peer review comments and responses

Suggestion	Response
Reviewer #1 The authors can improve the Fig 3, if possible, they can make some other way.	Agreed. The problem is that the figure has been shrunk to the point that it is unclear. This has been fixed by restoring the figure to its original size. Fig. 1 has been slightly amended and Fig. 2 has been redone for improvement and Fig. 10 has been updated.  Spelling and grammar have been corrected and improved for a better read.
Reviewer #1 Mention the reference of Fig 5.	Fig. 5 has already been referenced in the text.
Reviewer #2 To be considered, it should be expanded a bit. The references should be expanded and discussed in more detail to make it more of a review.	A paragraph has been added to the beginning of the introduction to highlight to the reader the ever-growing importance of hydrogen energy and the growing need for hydrogen separation membranes. In order to expand, I have introduced existing and alternative hydrogen separation techniques. Introducing pressure swing adsorption and cryogenic distillation. Heading numbers have been amended accordingly. Table 5 has been updated with more data obtained from additional references. Sandrock reference removed for better read (originally Ref [46]). Several references have been expanded and discussed accordingly and are highlighted in yellow throughout the manuscript. Sentences which have been added have also been highlighted in yellow.

Advice for peer reviewers

Be patient. You will likely read manuscripts that will rile your emotions. Don’t get emotionally invested. Treat the whole experience as an honour since there are people out there who view you as an expert. See it also as a duty to your field of research to do your best to ensure that only the best work gets through. You are a gatekeeper.

Treat your fellow authors with the respect and dignity that you once expected. You were and have been in the position of the authors. Avoid abusing this position. We’re all in the same boat. We want to make this world a better place through our hard work. So, the author’s success is yours too. But that doesn’t mean that we should be lenient when it comes to bad research. Be fair and extra clear with your criticism. Point out the problem and suggest a possible solution. This will allow the author to quickly amend the manuscript and swiftly complete the whole peer-review process.

Whatever you do, do not correct the language. You are not a language editor. I did this once as a peer reviewer and the whole process was unnecessarily prolonged. It is the job of the author to ensure that the manuscript has been edited and proofread before undergoing the peer-review process.

What do you think?

Do you think I’ve missed something? Do you need proofreading, editing, course designing, grant proposal writing or consultancy services? I’d love to hear from you. Get your free quote today!

Follow me

Remember to follow my Facebook, Twitter and LinkedIn pages as well as subscribe to my YouTube channel for the latest updates.