[net-gold] Roediger's Tips for Reviewers
- From: "David P. Dillard" <jwne@xxxxxxxxxx>
- To: Other Net-Gold Lists -- Educator Gold <Educator-Gold@xxxxxxxxxxxxxxx>, Educator Gold <Educator-Gold@xxxxxxxxxxxxxxxx>, net-gold@xxxxxxxxxxxxx, NetGold <netgold@xxxxxxxxxxxxxxx>, Net-Gold <net-gold@xxxxxxxxxxxxxxxx>, K-12ADMINLIFE <K12ADMIN@xxxxxxxxxxxxxxxxxxx>, K12AdminLIFE <K12AdminLIFE@xxxxxxxxxxxxxxx>, NetGold <netgold@xxxxxxxxxxxxxxxx>, Net-Platinum <net-platinum@xxxxxxxxxxxxxxx>, Net-Gold <NetGold_general@xxxxxxxxxxxxxxxxx>, Temple Gold Discussion Group <TEMPLE-GOLD@xxxxxxxxxxxxxxxxxxx>, Temple University Net-Gold Archive <net-gold@xxxxxxxxxxxxxxxxxxx>, Health Lists -- Health Diet Fitness Recreation Sports Tourism <healthrecsport@xxxxxxxxxxxxxxxx>, Health Diet Fitness Recreation Sports <healthrecsport@xxxxxxxxxxxxxxx>, HEALTH-RECREATION-SPORTS-TOURISM@xxxxxxxxxxxxxxxxxxx
- Date: Wed, 3 Aug 2011 15:00:10 -0400 (EDT)
.
.
Date: Wed, 3 Aug 2011 10:27:03 -0700
From: Richard Hake <rrhake@xxxxxxxxxxxxx>
Reply-To: Net-Gold@xxxxxxxxxxxxxxx
To: AERA-L@xxxxxxxxxxxxxxxxx
Cc: Net-Gold@xxxxxxxxxxxxxxx
Subject: [Net-Gold] Roediger's Tips for Reviewers
.
.
If you reply to this long (11 kB) post please don't hit the reply
button unless you prune the copy of this post that may appear in your
reply down to a few relevant lines, otherwise the entire already
archived post may be needlessly resent to subscribers.
.
************************************************
.
ABSTRACT: The inadequate reviewing of articles submitted for
publication in the field of Physics Education Research (PER) suggests
revisiting Henry Roeidiger's (2007) "Twelve Tips for
Reviewers"<http://bit.ly/oOR5iQ>. I list six of those TIPS which are
often ignored by PER reviewers.
.
For the few "Good Reviewers" of PER article submissions who may wish
to get in step with the PER's army of "Bad Reviewers," I strongly
recommend Mohammad Sal Moslehian's "How To Be a Bad
Referee?!"<http://bit.ly/ranWvb>.
.
************************************************
.
My two decades of experience in physics education research indicates
that one of field's major problems is the inadequate reviewing of
articles submitted for publication and subsequent apparent failure of
editors to recognize such inadequacies.
.
According to Charles Henderson (2004), Physics Education Research
Conference (PERC) organizers are currently recommending that
attendees acquaint themselves with Henry Roeidiger's (2007) "Twelve
Tips for Reviewers." As I recall, Roeidiger's TIPS were also
recommended to attendees at some past Physics Education Research
Conferences (please correct me if I'm wrong).
.
However, I've never detected any evidence that Roeidiger's TIPS have
improved PER refereeing - see e.g. "Re: Learning Physics with a
Statistical Gold Standard" [Hake (2011)].
.
Perhaps it might be helpful to revisit Roeidiger's TIPS. Roediger
perceptively wrote [bracketed by lines "RRRRR. . . ."]:
.
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
.
Many critical skills needed for becoming a successful academic are
typically not taught in graduate school, at least not in any formal
way. One of these is how to review journal articles. . .
.
. . . . . . How do people learn to review? I suspect most newly
minted PhDs learn to review papers in the same way that the children
in Albert Bandura's famous 1960s studies learned aggressive
responses, which is to say, BY IMITATION. . . . .[[My CAPS]]. . . .
.
Because reviews are often highly negative, the new researcher
implicitly learns from the negative reviews received on his or her
own submitted papers that reviews are supposed to be negative. It is
as if the implicit message is: "A reviewer's job is to criticize the
manuscript. Find any faults in the logic, method, results, and
conclusions that the paper might have and then communicate these to
the editor." Hence, the cycle of negative reviews is perpetuated
across academic generations. . . . .
.
. The book "Reviewing Scientific Works in Psychology" . . . .
.[[Sternberg (2006)]]. . . . is excellent, and I urge readers to
purchase it. . . . . The purpose of this column is not to replace
that book but simply to provide some guidelines for reviewing papers
that may be helpful to some, especially younger, readers.
.
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
.
Among Roediger's valuable TIPS for reviewers are [bracketed by
RT-RT-RT. . . . "; my inserts at ". . . . . . [[insert]]. . . ."]:
.
RT-RT-RT-RT-RT-RT-RT-RT-RT-RT-RT-RT-RT-RT-RT
.
1. KNOW YOUR MISSION. A reviewer's job is to evaluate a submitted
article, not (necessarily) to criticize it and certainly NOT TO TRASH
IT. . . . . .[[My CAPS.]]. . . .
.
2. SAY POSITIVE THINGS IN YOUR REVIEW. Many reviews I read (both
those of my papers and those of others' papers when I am one of the
companion reviewers) have nary a kind word. Rather, the review
consists of an unrelenting series of criticisms that might leave the
author ready to hang him- or herself by the end of the review.
.
3. DON'T EXHIBIT HOSTILITY OR MEAN-SPIRITEDNESS IN YOUR REVIEW. The
field suffers from too many hostile reviews, as Sternberg has noted
in his essays on this topic.
.
4. BE CAREFUL IN RECOMMENDING FURTHER EXPERIMENTATION. One of the
easiest bad habits for reviewers to develop is to routinely recommend
that further research be conducted before the paper is published. For
most papers, you will be able to think of further experiments that
might be done on the topic. However, if the author has provided
several experiments already and made a coherent contribution, the
critical question is not "can further experiments be done?" (the
answer is "yes"), but "does the author need to do further experiments
to support the conclusions being made?"
.
5. WATCH FOR EGOCENTRISM. If you received a paper to review, chances
are you yourself have published on that topic. Nearly every member of
any scientific field is subject to the feeling of citation neglect -
"The author should be citing my work more often." Somehow, we feel
compelled to remind the author that we made a similar point to hers,
even if we embedded it on page 646 in footnote 2 of that article we
published in an obscure journal in 1991.
.
6. SIGN YOUR REVIEW. . . . . . .[[IMHO this is the most important of
Roediger's tips - but almost never done by Physics Education Research
(PER) reviewers]]. . . . . . Or, if you can't bring yourself to do
that, at least write your review as if the author will learn your
identity and you wouldn't be embarrassed. I sign all of my reviews
and have done so for many years. I THINK IF EVERYONE DID, MOST OF THE
PROBLEMS OF NASTINESS IN REVIEWING WOULD DISAPPEAR. . . . .[[My
CAPS.]]. . . . As psychologists have repeatedly shown (e.g.,
Zimbardo's prison experiment), human beings do not display their best
behavior when they are cloaked behind the mask of anonymity. Signed
reviews will usually be more polite and diplomatic, with much less
tendency for brutal, unvarnished criticism. Of course, you still want
to give your honest opinion, but (as discussed above) there are
helpful and unhelpful ways of relating that opinion.
RT-RT-RT-RT-RT-RT-RT-RT-RT-RT-RT-RT-RT-RT-RT
.
My experience has been that most PER reviewers are in drastic need of
reviewing and benefiting from Roediger's TIPS, especially the above
six.
.
BTW -There may be few reviewers of PER article submissions who are
"Good Reviewers" and for whom Roediger's TIPS have already been
internalized. If such reviewers wish to get in step with the PER's
army of "Bad Reviewers," I strongly recommend Mohammad Sal
Moslehian's "How To Be a Bad Referee?!"
.
.
.
Richard Hake, Emeritus Professor of Physics, Indiana University
Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands
President, PEdants for Definitive Academic References which Recognize the
Invention of the Internet (PEDARRII)
<rrhake@xxxxxxxxxxxxx>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>
<http://HakesEdStuff.blogspot.com>
<http://iub.academia.edu/RichardHake>
.
.
.
REFERENCES [All URL's shortened by <http://bit.ly/> and accessed on 3
August 2011.]
Hake, R.R. 2011. "Re: Learning Physics with a Statistical Gold
Standard" online on the OPEN! AERA-L archives at
<http://bit.ly/nlyfCC>. Post of 24 Jul 2011 11:12:13-0700 to AERA-L
and Net-Gold. The abstract and link to the complete post were
transmitted to various discussion lists and are also on my blog
"Hake'sEdStuff" at <http://bit.ly/q9Snn6> with a provision for
comments.
.
.
.
Harvey, J. & M. Holladay. 2006. Review of Sternberg (2006), R.J.,
ed. 2006. PsycCRITIQUES 51(4), January 25; online as a 37 kB pdf at <
http://bit.ly/pSatnp >. The reviewers wrote "Although the focus of
'Reviewing Scientific Works in Psychology' is on psychology, its
advice and distinctions can be readily generalized to the other
social sciences. The book could be readily used in advanced
undergraduate and graduate seminars on methods and introductions to
the scientific enterprise in psychology. Further, it is accessible
and thorough enough that experienced and beginning reviewers alike
will find it valuable."
.
Henderson, C.R. 2011. Private communication to R.R. Hake, 2 August.
.
Roediger III, H.L. 2007. Association for Psychological Science, "The
Academic Observer: Twelve Tips for Reviewers," online at
<http://bit.ly/oOR5iQ>.
.
Moslehian, M.S. 2010a. "How To Be a Bad Referee?!"online at
<http://bit.ly/ranWvb>. Evidently derived from Moslehian (2010b).
.
Moslehian, M.S. 2010b. "Attributes of an Ideal Referee," Notices of
the American Mathematical Society, November, p. 1245; online as a 49
kB pdf at <http://bit.ly/oDBWIt>.
.
Sternberg, R.J., ed. 2006. "Reviewing scientific works in
psychology," American Psychological Association, Amazon.com
information at <http://amzn.to/p4HKwj>. See also the review by
Harvey & Holladay (2006).
.
.
Other related posts:
- » [net-gold] Roediger's Tips for Reviewers - David P. Dillard
