In reterees we trust?

In reterees we trust?

Melinda Baldwin

The imprimatur bestowed by peer review has a history that is both shorter and more complex than many scientists realize.

Melinda Baldwin is PHYSICS TODAY’s Books editor and the author of Making “Nature”: The History of a Scientific Journal (University of Chicago Press, 2015)

In the early summer of 1936, Albert Einstein and this assistant Nathan Rosen submitted a paper on gravitational waves to Physical Review. In it they argued that gravitational waves did not exist—a controversial claim that went against the prevailing scientific consensus. Six weeks after the paper’s submission, Physical Review editor-in-chief John Torrence Tate wrote back to Einstein with a copy of a critical referee report and asked for a response to the reviewer’s comments.

So far, this story will sound familiar to most PHYSICS TODAY readers.  Modern  scientists  expect  that  their  submissions  to journals will be read and criticized and will require revision before they are admiied into the corpus of published scientific literature. Einstein, however, did not share those expectations. In fact, he was surprised and offended by the idea that his paper had been sent out for external review. (See the article by Daniel Kennefick, PHYSICS TODAY, September 2005, page 43.) In his riposte to Tate, Einstein said that he and Rosen had sent you our manuscript for publication and had not authorized you to show it to specialists before it is printed. I see no reason to address the— in any case erroneous—comments of your anony- mous expert. On the basis of this incident I prefer to publish the paper elsewhere.

Einstein kept his word. He would never again submit a re- search article for publication in Physical Review.

It might be tempting to view Einstein’s reaction as a show of ego by a senior physicist who thought his fame would allow him to skip the peer review process. However, digging deeper into the history of peer review uncovers a more complicated picture. In 1936 refereeing was not a universal practice at the world’s top scientific journals. It was not even a universal prac- tice at Physical Review. Einstein’s previous submission to that journal, the famous 1936 Einstein-Podolsky-Rosen (EPR) paper, was not sent out for referee reports despite its provocative anti- quantum conclusions.

So Einstein’s bafflement at receiving an anonymous report criticizing his paper was hardly inexplicable. But 80 years later, peer review is an expected and established part of publishing for scientists and scholars in almost every academic discipline.

How did this process become so ingrained in scientific life?

The origins of journal refereeing

Many academic and popular articles about peer review assign it the same origin story. In 1665 the Royal Society gave its secretary Henry Oldenburg permis- sion to compile Philosophical Transactions of the Royal Society of London, generally regarded as the world’s first scientific jour- nal. Oldenburg immediately thought it wise to gather expert opinions on the papers he wanted to publish. Thus peer review was born and was ever aler a consistent part of scientific publishing.

Or was it?

That origin story appears to have its roots in a famous 1971 sociology  article—Harriet  Zuckerman  and  Robert  Merton’s “Paierns of evaluation in science: Institutionalization, structure and functions of the referee system.”1 Zuckerman and Merton’s article, based on an analysis of referee decisions at Physical Re- view, remains a foundational study of the sociology of peer re- view. It was so groundbreaking that PHYSICS TODAY printed a condensed version in its July 1971 issue (page 28). In crediting Oldenburg with the invention of peer review, Zuckerman and Merton implied that peer review’s form and function had changed liile since the 17th century.

More recent historical work, however, has called Zucker- man and Merton’s history into question. In reality, Oldenburg rarely consulted outside opinions on what should be published in Philosophical Transactions, and he held such close control over the journal’s contents that he occasionally referred to himself as its “author.” There was not even a formal submission process. Oldenburg would simply print what interested him and what he thought might be of value to his readers, including not only experimental papers but secondhand accounts of others’ exper- iments, discussions of recent books, and even his own personal correspondence.

Although Oldenburg was indeed a pivotal figure in the his- tory of science publishing, he was not peer review’s inventor. That honor arguably belongs to William Whewell, a Cambridge University polymath who also coined the terms “physicist” and “scientist.” In 1831 Whewell suggested that the Royal Society should commission wriien reports on papers submiied for publication in Philosophical Transactions. He thought those reports should then be pub- lished in the Society’s new journal, Proceedings of the Royal So- ciety of London, thereby fulfi the dual purpose of fostering rich scientific discussions and providing material for the new publication.

The Royal Society adopted Whewell’s suggestion of soliciting reports but shiied quickly away from his vision of print- ing them for public discussion. A handful of reports did appear in Proceedings, but by the mid 1830s that practice had ceased. Instead, the society decided that referee opinions were mainly useful for helping it avoid printing anything embarrassing in its publications. By the mid 19th century, refereeing for Philosophical Transactions was almost entirely run by two secretaries, one in the physical sciences and one in the biological sciences.

The secretaries were eminent members of the society, and they each worked with an assistant sec- retary to arrange refereeing for the papers submijed to Philosophical Transactions. The referee reports came to be seen as confidential documents for the internal use of the society. For many years, they were not made available to the authors of accepted or rejected papers.

Because authors did not see the reports, there was no real equivalent to today’s common “revise and resubmit” decision. Submissions to Philosophical Transactions were either accepted or rejected. However, the secretaries did occa- sionally encourage authors of Philosophical Transactions papers to revise their articles before they went to print. Physicist George Gabriel Stokes, who served as the society’s physical sciences secretary for more than 30 years, oien suggested changes to authors via personal correspondence. Stokes would paraphrase useful comments from the Philosophical Transactions referees, and if Stokes himself had refereed the paper, he would oien send the author a copy of his full, signed report.

Full text on this link.

La responsabilidad del contenido de los artículos y reportes incluidos en esta Colección es de sus autores y de las entidades que los publicaron. Su contenido no refleja necesariamente criterios adoptados por el COMECSO ni por sus instituciones afiliadas.