A Peer Review Problem

Michael Sean Quinn, Ph.D, J.D., Etc.
2630 Exposition Blvd  #115
Austin, Texas 78703
(o) 512-296-2594
(c) 512-656-0503

Expert witnesses in all sorts of areas, including insurance, as often asks in the opening qualification section whether they have published anything relevant.  If the answer is “Yes,” they are often asked whether any of their work has been “peer reviewed.” I know about this gambit–indeed, I am an expert on this sort of deposition, since I have given a lot of them and taken more than a few.  

The origin of the question is the Daubert case (generally pronounced “Doe-bear,” with the accent on the second syllable) case.  The Supreme Court of the United States case establishing tests for qualifications for a”candidate” for qualified scientific expert witness.  One of the tests was whether the relevant publications of the proposed expert had been “peer reviewed,” i.e., reviewed by anonymous already qualified scientists.  That case was replaced by Kumho Tire, and the centrality of peer review dropped out to some degree, but not quite out. 

Most active judges had already realized that peer review did not exist for many types of proposed experts, such as experts on insurance and insurance bad faith.  Not even law reviews have what is technically known as “peer review,” although some peers might review a proposed writing. 

When I am asked THE question I always answer by saying one of two things. Here

 Answer Strategy #(1)

(1) “What do you mean by “peer review?”

Some lawyers say, 

“What do you think it means? [Often they also say, “This is my deposition so I’ll ask the questions, to which I say something, “Not really. It’s actually mine.”]

My response is expansive: 

“It is a form of editorial review, sort of, used mainly in scientific and mathematical journals (and social science journals, sometime), and much less elsewhere, if at all, in which anonymous groups of similar scientists or mathematicians is are asked to review a submission and evaluate its ‘publish-ability.'”  (I have never seen it used in judging publications on, for example XYZ performance or behavior.)” 

Some lawyers ask, 

“Well, has any of your stuff been “peer reviewed”?

My answer is entirely predictable: 

“Since I have not published scientific, engineering or mathematical papers, the answer is ‘No,’ though many of my peers have read much of my stuff, sometimes at my request before I submit it for publication.  For example, when I publish something in the INSURANCE LITIGATION REPORTER the editor, John di Mugno, a leading authority on insurance law, reads the drafts and comments on them.”

Usually it stops there. 

Answer Strategy #(2)

Here’s the second way of answering. It’s quicker but not quite as good. 

L: “Has any of your stuff been peer reviewed”?

A: “In the technical sense of that term, the one used in Daubert, the answer, of course, is negative, since no one who writes on my kind of stuff is.”

One would expect an objection on the grounds that the question is nonresponsive because of saying too much.  This has never happened to me.  If it did, I’m ready, I revert back to Strategy #(1).

The point of this blog is slightly different, however. I have a a new idea for a Strategy #(3) that can be added into the other two, at least for[of maybe I should be saying, “if only for” amusement.

On Friday July 11, 2014, NYT carries a story on A3 about a scientist who had published 60 or so papers all involving fraudulent experiments and fraudulent peer reviews–conspirators, false mailboxes and more. An on the same day there was an editorial written by a scientist-&-journal-editor, in effect, that there’s more of this sort of thing than one might think. Id at A21.

So, now we get to the addition.

“Besides given what is being reported about fraud in the area of “peer reviews,” I wonder how reliable they are these days.” 

A Bonus, Though Not a Bone to Be Picked

One thing I have never seen is for counsel taking the deposition to go after the peer review files. There is two ways to do this, at least.  One is to seek it from the journal.  This could be done through a simple request, or it could be done by means of a deposition on written questions.  E-discovery or other fancy stuff will not be necessary since the lawyer will be looking only for the copies regarding the witness. There will be resistance from the journal and from the “peer,” since confidentiality has been promised. The other is to seek them, from the witness.  The chances are s/he won’t have any, or will have bits and pieces only.  Even he s/he does, there will be resistance.  

Another & Different Bonus: You Lucky Readers

There is another usage of the phrase “peer review.”  This is to be found in areas of engineering and related disciplines. It is not academic.  It is designed to see to it that the author(s) of a design or a person in change of an activity is doing it well. This could be putting something together, e.g., a jet engine, building something, like a series of networks for an intranet system, or a whole slew of other activities.  It could be an test at a pharmaceutical company or the tuning of one hell of a a piano   This entire idea could be called “a work performance review by one or more peers.”

Of course, there could be expert testimony about the performance of the peers doing the reviews in this sort of case. It is easy to image exactly this happen, e.g., if a big building of complex design tipped over. Indeed, at least in theory, this could happen in the academic case, as well, but it probably hasn’t and probably never will.