Why the Incessant Quest for Academic Excellence Leads to Bad Science

We tend to reward excellence in all realms of life. We especially tend to reward excellence in academia. Now, a team of highly credentialed academics from diverse disciplines are striking a blow against that very excellence, which they are paid to represent and embody. Excellence, they say, is bad, and our undying pursuit of academic excellence in particular has become counterproductive.

The five scholars behind the treatise “Excellence R Us: University Research and the Fetishisation of Excellence” have a credibility problem — but only insofar as they are so credible it’s counterintuitive that they would rally to this cause. Daniel O’Donnell is a University of Lethbridge English professor. Samuel Moore is the Managing Editor of Ubiquity Press. Damian Pattinson is the Vice President of Publishing Innovation at Research Square with a background in chemistry. Cameron Neylon is Professor of Research Communications at Curtin University’s CCAT lab, the former Advocacy Director for PLOS ONE, a chemist by trade. Martin Paul Eve is Professor of Literature, Technology, and Publishing at the University of London. They form an odd collective, but they have unified around a shared concern: emphasis on brilliant work is jeopardizing a scientific system designed to ensure constant progress.

“If you focus on trying to only fund the revolution, you end up impoverishing your science,” University of Lethbridge English professor Daniel O’Donnell tells Inverse. “If you’re aiming at the back fence all the time, you’re going to strike out an awful lot.”

These academics don’t want to disincentivize excellence. The scholars still acknowledge that we need to support excellent scientists enough for them to see their visionary ideas through to completion. The goal is to make sure that science doesn’t go the way of Hollywood, which has all but divested from the mid-tier movies that once constituted the bulk of output and invested that capital in blockbusters.

An example of the problem, O’Donnell says, is the United Kingdom’s “Research Excellence Framework,” a systematic attempt to grade all university departments and researchers. With the REF, the UK was attempting “to reward excellence disproportionately, and to not reward nearly as proportionally the non-excellent.” The framework makes sense from a political perspective, but the people it’s being applied to can clearly see that rewarding research that is bound to be rewarded anyway is flawed.

O’Donnell is ultimately concerned about scientific capacity. The more we increase scientific capacity, the greater the odds are that excellent work will emerge. “Our point is that if we focus entirely on the top, we’re going to end up impoverishing what we know about the world, and doing bad science,” he says. While, in theory, policies like the REF seem beneficial, they’re detrimental in practice. If the UK succeeded in discouraging all unexciting work, “all they would have is the geniuses,” he explains. “And there just aren’t that many of them.”

The UK would be left with very, very small scientific capacity. There might be a handful more groundbreaking discoveries, but, overall, the community would suffer.

Instead of fiending for excellence, O’Donnell and his fellow scholars believe that check writers should be worried about soundness and credibility. “The rhetoric of excellence is always about paradigm shifting,” he says. Thomas Kuhn, the author of The Structure of Scientific Revolutions, has always been credited with this ideology because of his hagiographic descriptions of Galileo rolling out his heliocentric model of the solar system. O’Donnell and his fellow agitators argue that Kuhn actually emphasizes the importance of normal science. Every so often, in science, someone comes along and makes an incredible discovery, a discovery powerful enough that it breaks ground. But once all that metaphorical ground is broken up, normal scientists are needed to analyze the soil samples. In a sense, revolutions begin with revolutionaries and inevitably end with normal science.

In fiction, as well as many other fields, it’s impossible to quantify excellence. We can recognize excellent writing — somewhere halfway between our gut and our noggin, normally — but there’s no numerical value that explains a novel’s greatness. “Could you imagine if there were a bar that you had to cross as a fiction writer, where you had to show — you had to show — that you were measurably more excellent than Faulkner or Joyce?” O’Donnell asks. “How would you do that? There’s no way it would be good for fiction writing.” The fact that objective criticism is much harder in literature cushions writers from some of the blows scientists routinely take. Good experiments don’t always lead to world-changing results, and world-changing results are quantifiable. It’s possible to be a good scientist while remaining inconsequential.

“The problem probably lies with the fact that you’re trying to meter scientific excellence — to measure it in some way,” O’Donnell says. Academics and researchers are held to publication and citation counts. You get the REF literally rating universities’ and researchers’ importance as if leaving an Amazon review. And you get both government and private organizations doling out cash to those who make their scientific endeavors seem important. Universities contrive rules, and those who abide can call themselves excellent. This incentivizes self-aggrandizing, not research.

The ultimate “token of excellence” right now is publication count. If a scientist exceeds publication expectations, then his or her higher-ups will deem them excellent. What’s really happening, O’Donnell says, is that scientists break down experiments into far too many pieces, and publish each part as its own paper. There’s very little synthesis, because synthesis is takes time and resources. But without synthesis, you’re unlikely to accomplish much that’s legitimately excellent.

And it’s not even clear that there is an objective excellence.

“When you ask that question of an academic, you have to ask them what department they’re from,” O’Donnell says. Economists, physicists, mathematicians, and the like will all tell you there is. Anthropologists, English scholars, and historians will not. In O’Donnell’s view, “excellence is an empty term. It’s really a way of comparing things that are incomparable.” The solution is not to clarify and redefine what excellence means, nor is it to continue raising the bar. The solution is to dispense rewards proportional to work done, not gains accomplished.