There is a quiet crisis unfolding in universities around the world. It does not make headlines, but it shapes careers, distorts research agendas, and is steadily hollowing out entire disciplines. The crisis is this: academic institutions have outsourced their judgment to databases.
Scopus. Web of Science. h-indices. Impact factors. These tools were designed to help measure scholarly output. Instead, they have become the output itself. What gets indexed is what counts. What does not appear in a database does not exist — at least not institutionally.
The consequences are not abstract. A monograph that takes a decade to write, a translation that makes ideas accessible across languages, a poem that distills a cultural moment with more precision than any survey instrument — none of these registers meaningfully in the metrics that now govern hiring, promotion, and funding decisions at most universities globally.
The Database as Arbiter of Knowledge
The problem begins with a category error. Citation databases were built to track scientific literature — journal articles, primarily in STEM fields, following standardised formats, published in indexed outlets. They do a reasonable job of that. The error is in assuming that this infrastructure can be extended to evaluate all forms of scholarly contribution.
It cannot.
The Humanities, the Arts, and practice-based disciplines do not produce knowledge in the same form as molecular biology. A historian's argument unfolds over three hundred pages. A choreographer's research is embodied in performance. A literary scholar's contribution might be a critical edition that restores a text to its proper context. None of these fits neatly into a journal article, and none accumulates citations the way a methods paper in a high-throughput field does.
Yet university ranking systems — QS, Times Higher Education, Shanghai — rely heavily on bibliometric data. And because institutions chase rankings, internal promotion criteria follow the same logic. The database becomes the de facto arbiter of scholarly value, even in fields it was never designed to assess.
What Gets Lost
The distortion runs deeper than unfairness to individual scholars. It changes what research gets done in the first place.
Academics are rational actors. When promotion depends on indexed publications, they write for indexed journals. When grant funding follows citation metrics, they frame research questions to fit formats that generate citations. When creative or interdisciplinary work carries no institutional reward, they abandon it — or never pursue it.
This is not a failure of individual courage. It is a structural problem. The metrics create incentives, and the incentives reshape research cultures over time. Fields that were once broad and intellectually adventurous become narrower. Questions that cannot be answered in a standard empirical format quietly disappear from the agenda.
There is also a quality problem. The pressure to publish in high-impact indexed journals has contributed to a well-documented proliferation of marginal, incremental work. A paper that confirms a minor variation of an established finding in a Scopus-indexed journal counts. A book that reframes an entire field's understanding of a problem, published by a university press, often counts for very little.
The Non-Traditional Research Output Gap
Several national research assessment frameworks have already recognised this problem and begun to address it. The UK's Research Excellence Framework explicitly evaluates Non-Traditional Research Outputs (NTROs), including creative works, performances, and practice-led research. Australia's Excellence in Research for Australia system assigns codes to creative outputs, requiring researchers to submit a Research Statement that contextualises the scholarly contribution. In the United States, many tenure committees at research-intensive universities routinely evaluate books, translations, and critical editions as primary outputs, assessed by peer experts in the relevant field rather than by citation counts.
These frameworks share a common insight: the form of a research output does not determine its intellectual value. What matters is the rigour of the inquiry, the originality of the contribution, and the significance of the work within its disciplinary context.
The gap between these national frameworks and institutional practice at most universities globally remains wide. Rankings still drive behaviour, and rankings still favour bibliometric data. Until institutions align their internal reward structures with a more pluralistic understanding of research, individual researchers will continue to face a stark choice between doing meaningful work and doing promotable work.
The Integrity Dimension
This is, at its core, a research integrity issue — though it is rarely framed that way.
Research integrity is usually discussed in terms of fabrication, falsification, and plagiarism. These are real problems, and they deserve serious attention. But there is a broader sense in which integrity means alignment between what an institution claims to value and what it actually rewards. When universities describe themselves as centres of knowledge creation while simultaneously reducing knowledge to what appears in Scopus, that is a form of institutional dishonesty.
It also creates downstream integrity problems. The pressure to publish in high-impact journals has been linked to questionable research practices: salami slicing, guest authorship, citation manipulation, and the strategic framing of null results. These are not random individual failures. They are predictable responses to a metrics environment in which the appearance of productivity is rewarded over the substance of contribution.
A More Honest Accounting
The solution is not to abandon measurement. Research accountability matters. Public and institutional investment in research deserves transparent assessment. The question is whether the tools used for measurement are adequate to the thing being measured.
A more honest accounting of scholarly contribution would do several things. It would treat different output types as legitimate on their own terms, assessed by criteria appropriate to each form. It would separate research quality evaluation from ranking exercises built on bibliometric proxies. It would require institutions to articulate, explicitly, what kinds of knowledge they value — and then build reward structures that reflect that articulation.
It would also require the academic community to take seriously the difference between measuring research and reducing it. Citation counts capture something real: the extent to which a piece of work circulates within a defined literature. They do not capture originality, public impact, disciplinary transformation, or the slower, deeper forms of influence that often matter most.
A poem read by a hundred thousand people and a paper cited by twelve specialists are not the same kind of contribution. Neither is superior in the abstract. But a system that counts only the second and ignores the first is not measuring knowledge. It is measuring indexability.
The Credibility Cost
There is a final irony worth noting. The global push for metrics-based research assessment was, in part, motivated by a desire to increase objectivity and reduce the influence of personal networks and institutional prestige on academic evaluation. The goal was credibility.
The result has been a different kind of credibility problem. A system that systematically undervalues entire traditions of scholarly work, that rewards form over substance, and that creates incentives for gaming rather than genuine contribution, is not objective. It is simply biased in a different direction — toward the measurable, the indexed, and the easily counted.
Rebuilding trust in research evaluation will require more than adding new database categories. It will require institutions to reassert their own judgment, to define what they value in specific and disciplinary terms, and to build assessment processes that can distinguish genuine contribution from the efficient accumulation of metrics.
That is harder than checking a database. It is also the only way to ensure that what universities reward bears some relationship to what scholarship is actually for.
Zero Citation welcomes commentary on academic publishing, research integrity, and the systems that shape scholarly work.