Should We Abandon Peer Review?
Should We Abandon Peer Review?
Should We Abandon Peer Review?
Over the last several weeks' articles, I have addressed the deep problems with science, focusing on private corporations’ exploitative practices, the deep flaws in peer review, and the issues with funding sources. One of the key reasons why there hasn’t been a massive revolt from scientists is that peer review, while deeply flawed, is better than the alternative. The alternative meaning that it wasn’t reviewed at all, meaning it was not scrutinized in any way. When boiled down to the very basic proposition, peer review is essential. Without some form of peer review to weed out the fraudulent, even if not always effective, any group could fabricate any claims they want with little deterrence. Solutions should focus on cleaning up the faults the peer review system has, not replacing it with nothing.
History Favours Change
The longer an institution is held, the harder it is to tear it down or change it. Fortunately, the peer-reviewed journal business, at least in the form we recognize it today, is not that old at all. The modern era of scientific publishing was largely founded by Robert Maxwell in the post-World War II era. Maxwell created a multi-billion publishing business empire, churning out journal after journal knowing it was a business of volume and not necessarily quality, and was later found out to be an outright fraudster; misappropriating pension funds in the tune of hundreds of millions.[2,3] A corrupt, exploitative business model not even a century old, designed by a known fraudster, is the most powerful force in science today. Not only that, but the cost to print and distribute journals, previously the biggest investment in the model, has been rendered obsolete by the internet.
It may be too difficult to sway the majority of professors who’ve spent their entire career romanticizing the notion of publicizing in a high impact factor journal that said journals no longer offer any tangible value, however, that is typically the case with any change. As Max Planck said, “Science advances one funeral at a time.” As it becomes more and more clear how technology and the internet are rapidly altering our society, we must look to the future to develop strategies in order radically to fix these broken and corrupt systems, not just slightly adapt them to a new electronic platform.
Sci-Hub is a resource I often use. Started by a Kazhaki scientist named Alexandra Elbakyan, Sci-Hub is a platform that can best be described as akin to a torrent site, or the original platforms like Napster and LimeWire, for scientific papers. Participants all over the world upload papers to Sci-Hub, violating copyright laws, where other users can search and download them free of charge, effectively bypassing the paywalls. In an article in The Verge which interviewed Elbakyan via email, they noted,
“Elbakyan is in hiding, facing charges of hacking and copyright infringement in the US. Elsevier recently obtained a $15m injunction (the maximum allowable amount) against her.
Elbakyan is an unabashed utopian. ‘Science should belong to scientists and not the publishers,’ she told me in an email. In a letter to the court, she cited Article 27 of the UN’s Universal Declaration of Human Rights, asserting the right ‘to share in scientific advancement and its benefits.’”
Sci-Hub was a needed first step, but not a long-term solution. Elbakyan’s illegal activities gave more attention to the greater problem and provide a sort-of band-aid-like solution to a small part of the issue. Unfortunately, she likely will not escape consequences for these actions, even if it means permanently remaining in “hiding.”
Open access is a necessary first step, but far from the revolution, it was originally expected to be. The NIH started the trend in requiring that research they fund to be open access within a year of publication, a move which Germany later followed suit in supporting via a “soft request” rather than a hard mandate. Now, 11 European nations including the UK, France, and the Netherlands are accounting for an annual scientific funding budget of $8.8 billion have moved to adopt radical open-access policies, stating that any work they fund must be immediately made open-access. This solves the massive issue of science sitting behind a paywall, however, it may simply (and has so far been the case) transfer costs from the public, and institutions library funds, to the teams conducting the research via increased publication fees.
Support Institutions that Fight Publishers
The University of California network, as discussed earlier in this series, moved to end their relationship with Elsevier. Eight years ago, around when Harvard was declaring that the price of journal subscriptions was unsustainable, thousands of scientists vowed to boycott Elsevier due to the rising costs to publish and access research. Despite the calls of boycott, motivating senior investment analysts to predict a depreciation in the valuation of companies like, and specifically, Elsevier, the business has continued to thrive and even grow. This suggests that researchers have not stood behind their own words, at least not sufficiently enough to have made an impact. It is interesting that in a set up where the academics hold all the cards, they cannot motivate to enact change; they conduct the work, to pay a fee to have it published, in which their counterparts do the work for free, to ensure its quality, and then they all purchase the end product back again. Literally, the publishers are providing no tangible service and should have no real power in this situation. Academics, in this isolated example, are behaving as if they are common serfs managing their master’s land and buying their own tools in order to do it. Researchers must stand together and fight against the abhorrent practices of publishers that are bad for science. They must stand in support with university networks like The University of California.
Innovative Platforms on the Right Track
As it stands, most referees prefer to remain anonymous. In fact, only 8% of referees agreed to let their identity be known in a recent pilot on open peer review. Some novel new systems have sprung up in the publishing world, on top of open access. For instance, as noted previously in this series, the BMJ has an open review process. Going a step further, the online open-access journal F1000 has made tremendous strides to fix some of the backlogs I have discussed. They are open access, are very low cost with short articles costing $150 and typical length articles $500, and publish referee reports. Some other journals have started publishing referee reports, such as those under Nature Publishing Group, eLife, and Cell Systems., although referees can “opt-out”, and there is no confirmed dedication to offering this service.
One major issue is that articles are initially published in F1000 before any referee reports, and articles may be online indefinitely until a reviewer comes along. Another interesting model, PeerJ, is open accessed and has no publishing fees per paper; authors must simply pay a one-time fee of $259 and are free to publish as many articles as they please, forever. All of these solutions are a step in the right direction, however, more drastic change is needed.
Solution: Government-Run Open Access Publication
Anyone who knows me well or has done business with me knows that I am highly critical of virtually any governmentally run department. A large majority of government departments are ineffective, slow, bogged down by tedious procedures and when pertaining to regulating business practices, even when mistakes are made, there are no safeguards to rectify- meaning the business simply loses time and money with no recourse. The issue is that the private set-up is more horrifying than anything I can imagine a first-world government planning up, and that is saying a lot. First and foremost, the governments of the leading nations (or at least the USA, to start), collectively stepping in to take action on this scandal is the lesser of the evils.
Secondly, it will take a massive step in order to bring down the publication business and there is not business situated to do this more effectively than the government, with any business with the capacity to do it exposed to the same risks of exploitation that currently exist. A great start would be that instead of the NIH, and other government-supported research grant organizations, mandating open access while ignoring the costs associated with publishing, they begin their own platform that researchers utilizing their grant funds must publish in. This step would be an immediate blow to the publishing business and would allow for radical change, paving the way to clean up many of the issues plaguing peer review.
NIH, and their international counterparts, are already footing the cost to publish in research they are funding. As detailed previously in this serious, the shift from the entrenched publishers to open access publishers reduces the fees by as much as 90%, with real costs being roughly half of the charged rates. Even if the government system is not as efficient as private open-access publishers, it will still amount to savings of 90% or more, and billions in freed up capital that can be redirected to actual scientific research. Further, if this massive shift of publication practice is diverted away from the entrenched publishers, with stated policy making it clear that the reason is abhorrent business practices coupled with lack of effectiveness, the remaining researchers will be stigmatized for continuing to do business with these groups when better options arise. Effectively, the allure of prestige will be replaced with the likelihood of being shamed and questioned.
Tit for Tat
In speeding up the massive bottleneck of scientific papers awaiting review and publication, an article published in Slate had an interesting proposition that could be immediately implemented into this hypothetical government-run, open-access platform.
“Right now anyone can submit an article or book to any journal or press, and if the beleaguered (often unpaid) editor likes it, she begs friends or grad students or total strangers to look at it for peer review. But what if in order to be eligible to submit an academic article to a journal, a scholar had first to volunteer to review someone else’s article for that same journal? What if that review had only two requirements: It has to be timely (in academese, by the way, this means three months). And that review has to be constructive. You want to publish and not perish? First you have to earn that right by making a punctual, non-petty investment into the publishing enterprise. Journals get better, more motivated reviewers; authors are more invested in actually reading and contributing the journals. Everybody wins. Call it ‘peer review review.’“
What’s great is this forced review could allow a double-blind initial review process. As is shown in the F1000 model, where articles are immediately published and then invite referees to review, this model, with an open review, should be adopted after the initial referee reviews. The peer-review process need not stop at publication. The proposed platform could allow anyone who requests access to be able to review the article and submit a report. If the referee either a) is qualified to review the article, or b) finds a fault in the data or conclusion then the referee report will be published as an addendum to the publication page. Additionally, all data typically available for reviewers to request, such as raw data, should be available on any post-publication review. Often, authors do not properly submit all required data knowing that it is rarely asked by many publications. Usually, this is simply time management, and referees do not have time to fully review unless significant issues are uncovered. However, sometimes information is not available (and if it is, it is fabricated) due to outright fraud. By knowing that this information needs to be kept available at the request of any subsequent post-publication review, it would dramatically increase the risk of those conducting fraud to be caught. Additionally, it would make it far easier for those doubting the findings of their competitors to review and assess their research, further increasing the risk of detecting fraud.
Some considerations would be in identifying which papers are the most important, a task that the “prestigious journals” are arbitrarily tasked with identifying now. First, editors would still have a role. Editors could keep track of new publications and craft out newsletters- free for all scientists or non-scientists alike to subscribe- and focus on papers they find are particularly intriguing, or important. Additionally, merit on quality of work can be assessed via referee scores as well as citation counts. For instance, individual papers would still be cited, and removing the façade of a journal's perceived importance would lead to better papers being cited on their merits alone. Second, referees could submit scorecards for the paper, judging on the quality of work, importance, etc. As is the case with Yelp and other review platforms focused on consumer goods, a combination of a good score, and more reviews, contributes to the total perceived quality.
Even with all these changes, new issues will undoubtedly arise. No system is perfect, and there are undoubtedly flaws to this as well; some may be able to be assessed immediately, while others would be sussed out in real-time, once instituted. Peer review can be fixed, and solutions do exist to improve both the quality and the cost, diverting resources away from corrupt publishers and back to where it is needed, the research.
Next week, solutions for the funding fiascos.