Talk:Peer review

Missing image Cscr-featured.png Featured article star

Peer review is a featured article, which means it has been identified as one of the best articles produced by the Wikipedia community. If you see a way this page can be updated or improved without compromising previous work, feel free to contribute.

Template:Talk Spoken Wikipedia

Talk:Peer review/Archive 1

---

That which this article, and apparently the general public, calls "peer review", is known in mathematics and statistics as "refereeing". If not for TV news broadcasts and other media addressed to the hoi polloi, I would never have heard the term "peer review". Persons whom editors of scholarly journals ask to review submitted papers are "referees". It is certainly plausible that in other fields they call it something else, but I wonder if some specificity is in order here: what are the fields in which it is called "peer review"? Michael Hardy 18:13 Feb 20, 2003 (UTC)

My understanding is that the two phrases are used in complementary ways -- a journal or an article may be called either refereed or peer-reviewed; the peer-review process involves one or more referees (that is, the reviewers are always called referees in my experience, but the process is more often called "peer review"). That said, I strongly take issue with the accuracy of this article. It is far from certain that peer-review leads to increased quality of articles; it certainly does promote conformity to general standards. And Wikipedia is most definitely not peer-reviewed in the academic sense; the whole point of Wikipedia is that anyone can be a referee, including people with widely varying levels of education in very diffeent fields. Slrubenstein

Agreed. In literature, as authors go in and out of vogue, many leave in their wake a body of books and peer-reviewed journals by and for fans of the authors. These books and journals are considered not to be authoritative by academics even though they are peer-reviewed because of the non-critical nature of the work. They reflect the qualities appreciated by the author's admirers, but often gloss over important or embarassing information about the author. --Modemx 19:59, 1 Feb 2005 (UTC)

You haven't said what field you're in. I have not heard of "peer review" in mathematics, but I've refereed some papers and had my own refereed. It's called "refereeing", not "peer review". But, if I can believe what I keep hearing, in some fields it's called "peer review". So my question is: which ones? Michael Hardy 23:24 Mar 1, 2003 (UTC)

Computer Science. See any Call for Papers from IEEE Computer society (http://www.computer.org/) and Association for Computer Machinery (http://www.acm.org/).

--- ejrrjs

Also, what about the anonymity issue? In mathematics, referees or usually anonymous, i.e., papers' authors don't know who the referees are, although they read the referees' reports, but referees generally do know who the authors are. I have heard that in some fields, they don't. Which ones? Maybe biology? Michael Hardy 23:26 Mar 1, 2003 (UTC)

This notion may be misunderstood, and thus may upset people, but I do not feel math is a science, it is more like a language, one that humans discovered and one that is self-consistent and extraordinarily useful while practicing science. Peer review would be useless in mathematics, as the proof is on the page, so to speak. There is no need to consult history (or "accepted" notions, which peer review does compare "new" ideas to as a basis for their validity) when "refereeing" a statistics or mathematics paper. - Plautus satire 14:30, 24 Feb 2004 (UTC)

I suspect that you do not know much about the workings of the mathematical community. First, it's inexact to state that the "proof is on the page". Real-life mathematical proofs are not machine-checkable (excluding fringe projects like Mizar) and generally require considerable expertise to be understood. Very subtle mistakes can arise; the author may have believed that some point should be skipped because it was "easy"; he may have made a definition that was subtly inconsistent; etc... For instance, checking whether Andrew Wiles' proof of Fermat's last theorem is correct is a job for only a few other high level mathematicians.

Plautus satire's comments are nonsense. I have refereed papers for five mathematics journals, and my published papers have been refereed (and some unpublished ones that I submitted). Obviously history is consulted; obviously proof-checking is not the main point of refereeing; obviously novel ideas are examined in the context of accepted notions. To say that mathematics is not a science but a "language" ignores the fact that hundreds of journals are devoted to publishing new discoveries in mathematics. Obviously in judging publication-worthiness one considers how new discoveries may be relevant to potential future research; one considers esthetics (which for most mathematicians is the main motive for doing mathematics or for learning mathematics). Where did you get this loony idea that there is no need to consult history? Do you not see sections on how a new discovery fits into the historical development of the subject in many research papers in mathematics? Michael Hardy 22:33, 1 Feb 2005 (UTC)

Second, there's an issue that many philosopher and sociologists that discuss science neglect: refereeing is not only about gauging whether a paper is scientifically correct, but also of gauging whether it's scientifically interesting. It's possible to write a whole paper of correct mathematics by writing 1=1, 2=2, 3=3... Yet, such a paper would be totally uninteresting, since it would not advance science, by saying evidences (trivial tautologies). One may for instance judge that a paper does not warrant publication in a major journal if it's about some gratuitous mathematical theory. Such judgment is of course somewhat subjective. David.Monniaux 19:20, 29 Apr 2004 (UTC)

A common argument I've had before with my students. My response is that people should actually look in a peer-reviewed journal of mathematics and then try to argue why math isn't a science. Math is "the mother of all sciences." --Modemx 20:43, 1 Feb 2005 (UTC)

Contents

1 Questionable para from fraud section 2 Image 3 Grammar question 4 Peer Review used to suppress opposing views 5 The picture 6 Lack of references 7 Spoken? 8 Request for references

Questionable para from fraud section

(William M. Connolley 08:37, 21 Jun 2004 (UTC)) I've moved this here for discussion:

While fraud obviously has severe negative consequences for the author of a paper, there are generally no adverse consequences to either the editor or the reviewers for recommending publication of a fraudulent paper, as detecting fraud is not a goal of the process, and the editors and reviewers almost never have enough information to detect outright fraud.

Who says that there are no consequences for editors (or reviewers - less sure). Quite likely, the editors get a black mark in their career record - what this text means probably is that nothing about that appears in the newspapers?

Hi William.

Fair point...when Nature withdrew the Schon papers, the editorial said

Nature this week finds itself in the unenviable (and unprecedented) position of formally retracting 
seven papers (see page 92).  All share the same first author, Jan Hendrik Schön (see Nature 419, 
417;  2002); in fact, this represents the entire body of work published by Schön in this journal.

which *does* look like severe negative consequences on the Journal, if not the Editor...nevertheless, the referees (being anonymous and voluntary, as much as anything) do escape censure....WMMV.

best

Robinh 10:23, 21 Jun 2004 (UTC)

Hi again. Another gem from Nature Editorial, 3 Oct 2002:

In some media reports, journalists and a few scientists who are unconnected with the
Schön investigations have taken the opportunity to make potentially damaging assertions
about journals, including Nature: that in order to compete or to publish exciting results,
journals will cut corners in peer review, overrule hostile reviewers or select sympathetic ones.

We at Nature unequivocally reject such charges. The publication history and files of these                                                                                                                                                                                         
particular papers and the editorial policies and interests of Nature are completely at odds with 
these assertions. Nature has nothing to gain by the pursuit of glamour at the expense 
of scientific quality, considering, not least, the criticisms, corrections and retractions we 
would then habitually be forced to publish.  There is more than enough rock-solid and splendid 
science to publish. Furthermore, it is a strict policy of Nature that our Letters and Articles are
selected for their outstanding scientific impact, sometimes also taking into account relevance to 
public policy issues, but never simply because the results will make headlines.

Robinh 13:16, 22 Jun 2004 (UTC)

Image

I added a picture from the Center for Scientific Review webpage (http://www.csr.nih.gov/). However, since I don't have a decent image editing program handy, the orginal that I uploaded is very big and probably should be edited for size. I'm pretty sure it's public domain, since it's from the NIH, a Federal government organization. There are several other images on the same webpage, under "images of peer review" (handy, huh?). Sayeth 19:43, Aug 4, 2004 (UTC)

Grammar question

"A chief rationale for peer review is that rarely is just one person, or one closely working group, able to spot every mistake or weakness ..." -- is that correct english? It seems to me that the "is" and "able" should be together, but english is not my native language. Could somebody enlighten me? -- stw (Talk) 09:52, 15 Aug 2004 (UTC)

I think this phenomenon is referred to as inversion (grammar). Inversion is the normal word order in questions (Must they feed the animals?) and can also be used after a "negative", restrictive adverb or phrase (Under no circumstances must they feed the animals. Never before have I heard such nonsense. Rarely do they eat meat. etc.)

However, you can altogether avoid this construction by not putting the adverb at the beginning of the sentence. -- Any good grammar will have more details. All the best, <KF> 10:44, 15 Aug 2004 (UTC)

Peer Review used to suppress opposing views

Perhaps we should mention that some scholars have charged that in environmental science, billions of dollars per year are subsidizing one viewpoint, i.e., the theory of anthropogenic global warming -- and that this financial pressure may be sufficient to corrupt the peer review process. Thus, perfectly valid results may then be dismissed from consideration on the grounds that they did not pass peer review. However, failing peer review is not proof that a paper has errors; only that the referees chosen to review it, diasagreed with it. This is science by voting, which is a self-contradiction. The only reliable criterion for the worthiness of an idea is whether others who try earnestly to verify it are able to do so. Science is replete with examples of important and valuable new ideas which were suppressed by the establishment (germ theory of disease, for example), and we should not exalt peer review as being immune to this type of abuse. --Uncle Ed 14:59, 15 Aug 2004 (UTC)

Furthermore, "Extraordinary claims require extraordinary evidence," said Carl Sagan[1] (http://www.pbs.org/wgbh/nova/aliens/carlsagan.html). If someone comes out with evidence to support an unpopular hypothesis (for example, "life exists on Mars" in 1975), they have to prove it beyond refute before it can even be considered seriously. -- ke4roh 16:04, Aug 15, 2004 (UTC)

Nonsense, if I submit the idea that it is possible to get to the moon by jumping on a pogo stick will the only criterion for "worthiness" be if several other scientists "try earnestly to verify it"? No, the worthlessness of some ideas are self-evident. The scientific process can be far more easily corrupted by, for example, the deep pockets of Washington think tanks. --Deglr6328 18:17, 15 Aug 2004 (UTC)

For an idea to be considered scientifically valid it also must be repeatable. For example, Einstein supported his general theory of relativity with a simply replicated experiment that proved the existence of quantum states of energy. "The establishment" initially rejected his findings because it was from someone who was "just a patent clerk." It was the repeatability of his experiment that helped put Einstein's ideas in the firmament of scientifically valid ideas. Contrast that with the researchers who railed against "the establishment" when their claims of being able to experimentally mass produce Fullerines (a.k.a. "Bucky Balls") were rejected because lack of repeatability. When someone actually found the process to mass produce Fullerines, some of those researchers were found out to simply be liars with economic rather than scientific motivations. Just because someone presents an arguably valid alternate theory about a matter of scientific controversy doesn't mean that it should be treated with equal weight to established theories because of "fairness." Fairness to the truth demands a body of repeatable results that gains the adherence of the scientific community and the standards should be high because the costs of bad science are high. --Modemx 20:28, 1 Feb 2005 (UTC)

The picture

(William M. Connolley 09:51, 18 Aug 2004 (UTC)) The picture at the start of this article doesn't look right to me. I don't think the chap is evaluating (in the sense of peer review) the proposals - he looks more like he is checking a pile of submitted proposals that all the boxes have been filled in.

You could be right - he/she (I can't really tell) is looking over the front page of what looks like a NRSA application. The front page has stuff like name, institution, and proposal title. However, the following pages contain the grant proposal, which is peer reveiwed. From the picture, the person looking at the grant could be just checking to make sure that boxes are filled in or he/she could be starting to read the entire grant, begining at the front page with the applicant's information. As I stated above, I got the picture from the Center for Scientific Review webpage (http://www.csr.nih.gov/). Looking over all the pictures they had there, I decided that this was probably best suited to the article, since many of the other pictures depicted reviewers debating grant proposals, which would be confusing to people reading about how peer review for journal articles is done anonymously. You're welcome to try to find another picture that better depicts peer review or just change the caption. Personally, I think this picture is okay, since it could be interpeted as someone starting to read a grant, but improvement is always welcome. Sayeth 15:44, Aug 18, 2004 (UTC)

Lack of references

Hi, this page no longer meets all of the featured article criteria. Specifically it has no references, or at least any that are properly cited. Please help add some references, use them to improve the article and cite them. Thanks - Taxman 01:19, Nov 5, 2004 (UTC)

Spoken?

(William M. Connolley 22:50, 18 Apr 2005 (UTC)) The new "spoken" thing doesn't work for me... when I click on "listen to this article" I get back to the text article again. Is something broken?

Request for references

Hi, I am working to encourage implementation of the goals of the Wikipedia:Verifiability policy. Part of that is to make sure articles cite their sources. This is particularly important for featured articles, since they are a prominent part of Wikipedia, and even moreso for an article like this one. The Fact and Reference Check Project has more information. Thank you, and please leave me a message (http://en.wikipedia.org/w/wiki.phtml?title=User_talk:Taxman&action=edit&section=new) when you have added a few references to the article. - Taxman 19:18, Apr 21, 2005 (UTC)