In 1996, physicist Alan Sokal decided to prove once and for all that postmodernism was a crock of shit. He submitted an entirely bogus paper to the journal “Social Text” that was subsequently published. Known as the Sokal Hoax, the fallout still effects the perception and legitimacy of critical theory and postmodernism today. Now, the “hard” sciences are having an equally embarrassing moment.
Science Magazine’s John Bohannon recently sent 304 open-access science journals a paper rife with errors and inadequacies that “were both obvious and ‘boringly bad.'” Of the 255 journals that decided to go through the editing process, 157 accepted the paper.
Many have taken the recent affair to launch attacks on the open-access movement. Because fuck making knowledge accessible to poor people, right?
The Scholarly Kitchen notes:
First, there is evidence that a large number of open access publishers are willfully deceiving readers and authors that articles published in their journals passed through a peer review process–or any review for that matter. It is simply not enough to declare that a journal abides by a rigorous review process.
Bohannon even called it the “Wild West” of academic publishing, a moniker picked up by the LA Times.
There is, to be fair, some truth to Bohannon’s assertions. Open-access journals themselves are becoming a business where author publishing fees replace subscription fees. And where there’s money, there’s assholes. But Bohannon’s experiment, as many have noted, didn’t have a control.At no point did Bohannon submit his paper to subscription-based peer-reviewed journals.
That’s pretty problematic, says Michael Eisen. Eisen notes that subscription-based journals have also published frighteningly bad papers, and a lot of it has to do, again, with money.
None of this will stop anti-open access campaigners (hello Scholarly Kitchen) from spinning this as a repudiation for enabling fraud. But the real story is that a fair number of journals who actually carried out peer review still accepted the paper, and the lesson people should take home from this story not that open access is bad, but that peer review is a joke. If a nakedly bogus paper is able to get through journals that actually peer reviewed it, think about how many legitimate, but deeply flawed, papers must also get through. Any scientist can quickly point to dozens of papers – including, and perhaps especially, in high impact journals – that are deeply, deeply flawed – the arsenic DNA story is one of many recent examples. As you probably know there has been a lot of smoke lately about the “reproducibility” problem in biomedical science, in which people have found that a majority of published papers report facts that turn out not to be true. This all adds up to showing that peer review simply doesn’t work.
And the real problem isn’t that some fly-by-night publishers hoping to make a quick buck aren’t even doing peer review (although that is a problem). While some fringe OA publishers are playing a short con, subscription publishers are seasoned grifters playing a long con. They fleece the research community of billions of dollars every year by convincing them of something manifestly false – that their journals and their “peer review” process are an essential part of science, and that we need them to filter out the good science – and the good scientists – from the bad. Like all good grifters playing the long con, they get us to believe they are doing something good for us – something we need. While they pocket our billions, with elegant sleight of hand, then get us to ignore the fact that crappy papers routinely get into high-profile journals simply because they deal with sexy topics.
So what should we do? Whether you’re a scientist, a critical theorists, or something else, peer-reviewed papers should not be read without heavy scrutiny from even the “casual” reader. Chances are, they’re full of lies.