Back to Chapter Back to Chapter

Case 11b :

The Awesome Responsibility of the Volunteer Reviewer

Avery's tone was incredulous,

"What do you mean it was falsified? How was I supposed to know that? Do you honestly mean that you expect that all of us who voluntarily review papers for your journal will actually re-run the analyses with the review data set? Who has time for that? I have ten post-graduate researchers, teach two undergraduate classes, and have my own research to attend to. It is not my job to police bad behavior. It is my job to tell you if the research seems plausible, important, and publishable. I did that. If you insist on publishing a retraction, go ahead, but if you publish the names of those of us who you are alleging missed a serious piece of research misconduct—that we did do our jobs properly—then I will have you sued for libel".

With that, Avery hung up the phone forcefully.

After returning from a walk around the campus to calm himself, Avery opened the original files and his STATA analysis software. He put the data through statconversion, and began entering the code for analysis.

Immediately, he began to see problems: the data were perfectly normally distributed, which is almost an impossibility for survey data. The "n" on the data was also quite high, something he noted in his review, but, "hey, we all get lucky sometimes, right?", he recalled thinking.

As he probed the data, it became quite clear to Avery that with just a bit more due diligence—running merely descriptive statistics on the file—he would have found enough to be suspicious of falsification. But, Avery was not alone! He did not review this paper himself… there must have been other reviewers accept the paper as well.

Avery thought to himself,

'How did we all miss this? Did this paper confirm our intuitions so completely that we did not even feel the need to review it?"

Throughout his evening Taekwondo workout and late into the night, the idea that he had believed a plausible story without checking the facts bothered Avery.

The next day, a very tired looking Avery knocked on the office door of his colleague, Jackson, and told him the story. Jackson listened sympathetically, offering Avery a third cup of coffee. Jackson, who edits a small journal, listened sympathetically to Avery, but was disappointed, saying to Avery,

"I understand we are all very busy, but we do have to rely on you to help us find these flaws. Even in my journal, we get 35 submissions a month. I could never review those. I have to count on you. What will you do now?"

• What are the responsibilities for reviewers to supervise submission for research integrity violations?
• What responsibilities editors have for supervising research integrity in their journals?
• What are the major problems here? Are they on the side of the journal or the reviewer?
   

Imagine you are Jackson:

• What would you advise Avery to do now?
• How should Avery respond to the editor of the journal now?
• How would you further describe the editor's position to Avery?
• How would you explain Avery's position to the editor?