Ted Nield ponders peer review
Geoscientist 17.8 August 2007
In the Musée du Petit Palais in Paris hangs one of Rembrandt's more unusual canvases. Self-portrait in Oriental Attire is a rare example of the artist standing – and closely resembling my late Aunt Agnes on her way to chapel. It is also remarkable for a rather preoccupied foreground poodle.
X-ray investigations of this painting have revealed that the poodle was not originally part of the artist's conception. Apparently, the great painter didn't like the way he had rendered his own legs. After a few failed attempts at putting the problem right, the master finally decided to hide the offending limbs behind some random animal.
As an allegory of the way scientific papers sometimes get written ("Crikey; that argument's a bit feeble… let's put a bit of methodological guff in there as a smokescreen and hope nobody notices") this is exactly the sort of thing that a referee's x-ray eyes should spot. Paradoxes, inconsistencies, disguised weaknesses and unanswered questions are exactly what peer reviewers are there to catch.
Whenever Presidents of this Society chair a Council meeting, they have to stare at another painting – of the infamous "Piltdown men". The Society had the honour of publishing the original papers based on this fraudulent fossil; and although the painting is not exactly Rembrandt, it reminds us that peer review is astonishingly weak when faced with a well executed fraud. "Beware of things you want to be true" may be a healthy motto for all of us; but the warning is hardly infallible, especially when faced with faked evidence. That part of the scientific process is inevitably taken on trust.
But peer review, imperfect and fallible though it may occasionally be, is what makes scientific publishing scientific. For this reason it is often appealed to in the face of the news media's habit of seizing upon a maverick paper for coverage. It irritates scientists that journalists are more interested in what is news rather than what is believed by a majority of scientists to be true (which is of course by definition not news at all).
Journalists are used to controversy, and have mechanisms to deal with it. They usually involve counter-comment; Tory by Labour. Big Oil by raffia mafia. But this Manichean approach, which many ordinary folk already find irritating, fails spectacularly to satisfy scientists. They, with justification, observe that giving "equal time" to a scientist who speaks on behalf of thousands of colleagues who all agree that the contrarian is barking mad - or a poodle on the leash of a sinister interest group - hardly constitutes "balance". On issues like climate change (see Colin Summerhayes's Global Warming Primer, last month) these arguments often become - heated.
But what happens when the peer review system itself becomes corrupt? Ruling theories can gain such a hold on both journals and funding bodies that freedom of speech is threatened, and contrary views may struggle to get an airing, even when they are well-founded. Paradoxically, when that happens, the only way out for beleaguered scientists is to gain coverage for their heresy in the otherwise despised "grey" literature, where the trammels of peer review do not apply.
This was the motivation behind two on-line debates held on www.geolsoc.org.uk – the first on mantle plumes, and in 2004, on the question of whether Chicxulub Crater, which everybody really wanted to be the dinosaurs' crater of doom, actually was. Opinion on both sides ran hot and strong, and at the time neither side seemed able to deliver a knock-out. However, in two recent papers Professor Gerta Keller (Princeton University) has presented highly persuasive evidence that the Chicxulub impact happened 300,000 years before the end of the Cretaceous.
Next month GeoNews will carry a news piece about this and a Media Monitor commentary. Or you can read all about it now on Geoscientist Online - nobody's poodle.