Outreach & Policy

Submission to House of Commons Science and Technology inquiry: peer review

Submitted 10 March 2011  

1. The Geological Society is the national learned and professional body for Earth sciences, with 10,000 Fellows (members) worldwide. The Fellowship encompasses those working in industry, academia and government, many of whom are actively involved in the peer review process as authors, reviewers and editors. The Society is a major international Earth science publisher, producing a number of peer reviewed journals and books totalling about 10,000 pages of peer reviewed content annually.
    
2. We are grateful for the opportunity to respond to this consultation. We have responded to the specific points raised in turn.

   The strengths and weaknesses of peer review as a quality control mechanism for scientists, publishers and the public
    

3. The peer review process has become well established in the current publishing process, and is now applied to almost all scholarly content.
    
4. Peer review is generally regarded as the best method of assessing the validity of the science set out in a publication, or the merits of a research proposal. Whilst it has flaws, it is generally recognised as the best option, provided that the quality of reviewers is maintained. 
    
5. There are many benefits for different parties in the system:
   
   a) The author: can attain a level of credibility by application of a respected and trusted procedure.
   b) The journal editor: can rapidly select the highest quality papers for publication and quickly identify those without merit.
   c) The publisher: can protect the quality associated with their brand and reputation.
   d) The librarian: can ensure that library collections focus only on valid and reputable content.
   e) The reader: knows that what they read has already been vouched for by those with the highest level of expertise 
   f) The researcher: will base their literature search and review only on reliable and unique content.
    
6. Despite being a well regarded system, a number of flaws can be identified. It is important to recognise that the peer review system is subject to individual opinions, which are rarely unaffected by factors outside of the science itself, due to the complex interface between science, politics and competition for research funding. On occasion, the process appears to be unpredictable to authors, who might receive very different receptions by journals of an equal stature. 
    
7. The quality of the system relies heavily on the quality and conscience of the reviewers, which can be both a strength and a weakness. It leaves the process open to the prejudice of reviewers, and the dismissal of unconventional views. It is naturally easier to judge research that provides a new view of a well-researched topic than it is to judge the value of a novel idea that may break a current paradigm. 
    
8. There is also the possibility of plagiarism going unnoticed, and the abuse of the privilege of pre-publication access to content. Whilst some areas of specialism may have a large pool of potential reviewers, others may be very small, opening up the possibility of bias or overwork as the same reviewer may be frequently called on. In addition, the system itself risks becoming overloaded, which can slow down the process, as well as allowing for errors and individual biases to emerge.
   

   Measures to strengthen peer review
    

9. Many of the improvements that have been made in recent years have been relating to speeding up the process. This can only go so far, as the most important part of the process, reading the content and assessing its validity, cannot be rushed. Flaws relating to the biases and preferences of reviewers can never be fully avoided, as the involvement of individuals with their own set of views is essential. It has been suggested by some that by expanding the pool of reviewers, the system could be improved by increasing the statistical chances of overall accuracy. However, this introduces more complications as workload increases (see above), so a balance needs to be struck. 
   

   The value and use of peer reviewed science on advancing and testing scientific knowledge
    

10. Peer review serves to save enormous amounts of time for individual researchers, who would otherwise be required to form their own judgements about a piece of work – a process which would be time consuming and, for less experienced researchers, inconsistent in outcome. 
     
11. It is also of use in assessing grant applications. In principle, this allows for a sift by a panel of involved reviewers, followed by mail reviews, an opportunity for applicants to respond to the reviews, and panel consideration under an experienced academic chair. In practise, however, the system may be overpressured due to lack of funds. This sometimes results in decisions that appear arbitrary, and a lack of funding for first rate research.
    

    The value and use of peer reviewed science in informing public debate
     

12. While peer review is generally recognised by the scientific community as mark of valuable and credible scientific research, it is not clear that this message is fully understood by the public when engaging in scientific debate. The climate change debate is a good example of this – without a proper understanding of the difference between peer reviewed and non-peer reviewed science, a lay person may judge both equally. Biases, opinions and unsubstantiated claims, which are for the most part eliminated from peer reviewed science, can easily be presented to an audience unfamiliar with the peer review process as equally credible. To facilitate public dialogue about science, it is essential that the merits and processes of peer review are well communicated – particularly the balance and consideration of uncertainty and error which are involved.

    The extent to which peer review varies between scientific disciplines and between countries across the world
    
    No comment
    
    
    The processes by which reviewers with the requisite skills and knowledge are identified, in particular as the volume of multi-disciplinary research increases
    
    

13. Journals rely on editors, who use their experience and networking to choose the most appropriate reviewers. Editors develop a pool of experts, who are known to have been published frequently and are well respected in their field, and refer to an editorial board, who recommend who may be best suited to review each paper as it is submitted. For reviewers, there are few benefits, other than staying in touch with current research prior to publication, and recognition for their efforts by the journal in question. While this system is designed to eliminate the establishment of a ‘clique’ of researchers, for papers submitted in a narrow field there may be a very small number of suitable reviewers. In addition, and particularly for narrow fields, there is a risk that those known to do a good job, or who are recognised in a field of very narrow specialisation, can become overwhelmed by large amounts of material.  
14. The system highlights the importance of employing professional and properly qualified scientific editors, whose role it is to identify the appropriate experts and ensure that no one reviewer is overburdened.
    
    
    

    The impact of IT and greater use of online resources on the peer review process

15. Online submission and review procedures are now widespread and are helping to minimise any delays due to the review process. Publishers are also better able to monitor progress and maintain effective reviewer and author databases.
16. Advances made in IT and online resources are generally thought to have improved the process and provided opportunities for time saving. However, they are not able to speed up the most time consuming part of the process, which is reading the document and preparing a well balanced, properly thought out review.
    
    
    

    Possible alternatives to peer review

17. No preferable system to peer review has been identified to date. Any changes are more likely to be improvements to the current system. Suggestions to publish online and modify articles based on ratings/comments may speed up the process and allow for wider publication, but it is difficult to see how quality could be maintained in this way. Peer review will, most likely, continue to evolve and publishers may experiment with new ways of reviewing content. Though it has flaws, the current system is widely regarded as effective, and remains stable and well respected for good reasons.