Implicated in the Indicator Game? An Experimental Debate

The rise of new modes evaluating academic work has substantially changed institutions and
cultures of knowledge production. This has been reflected and criticized in the literature in STS
and beyond. For STS scholars, these debates (should) however have an even more specific
dimension. Many of us are experts on aspects of these changes. But at the same time, we too are
part of the processes we are analyzing, and often criticizing. To put it slightly provocatively,
often we cannot avoid playing the very same game that we scrutinize. This creates tensions that
many of us reflect on, and it certainly has created many implicit and explicit normative stances
on how to deal with them. Yet it seems that so far there has been little room in our field to reflect
on and exchange this particular kind of experience-based knowledge. There are many different
ways to engage with the dynamics of evaluation, measurement and competition in contemporary
academia, or to play what we refer to colloquially here as the “indicator game.” With this debate,
we would like to give room to the expression and discussion of some of these ways. This text is
the introduction and prompt to an experimental debate. We discuss the state of the academic
discussion on the impact of indicator-based evaluation on academic organization, epistemic work
and identities. We use insights from these debates to raise questions for how STS and STSers
themselves deal with the indicator game. In conclusion, we summarize our contributors’
arguments and propose the concept of “evaluative inquiry” as a new way of representing the
quality of STS work in evaluative contexts.