Can research quality be measured quantitatively? On quality of scholarship, numerical research indicators and academic publishing - experiences from Norway
DOI:
https://doi.org/10.11143/fennia.66602Abstract
In this article I reflect on ways in which the neoliberal university and its administrative counterpart, new public management (NPM), affect academic publishing activity. One characteristic feature of NPM is the urge to use simple numerical indicators of research output as a tool to allocate funding and, in practice if not in theory, as a means of assessing research quality. This ranges from the use of journal impact factors (IF) and ranking of journals to publication points to determine what types of work in publishing is counted as meritorious for funding allocation. I argue that it is a fallacy to attempt to assess quality of scholarship through quantitative measures of publication output. I base my arguments on my experiences of editing a Norwegian geographical journal over a period of 16 years, along with my experiences as a scholar working for many years within the Norwegian university system.
Norwegian translation of the article is published in open access book Universitetskamp.
References
Aagaard, K., Bloch,C., Schneider, J.W., Henriksen, D., Ryan, T.K. & Lauridsen, P.S. (2014) Evaluering af den norske publiceringsindikator. Dansk Center for Forskningsanalyse, Aarhus Universitet, Aarhus. <https://npi.nsd.no/dok/eval2014/Evaluering_af_den_norske_publiceringsindikator_2014.pdf> 2.11.2017.
Aksnes, D.W. (2017) Artikler I nivå 2-tidsskrifter blir mest sitert. Forskerforum. 5.10.2017. <http://www.forskerforum.no/artikler-i-niva-2-tidsskrifter-blir-mest-sitert/> 2.11.2017.
Andersen, H., Ariew, R., Feingold, M., Bag, A.K., Barrow-Green, J., Dalen, B., Benson, K., Beretta, M. Blay, M., Bleker, J., Borck, C., Bowker, G., Leigh Star, S., Buccianti, M., Camerota, M., Buchwald, J, Gray, J., Cappelletti, V., Cimino, G., Carson, C., Clark, M., Keller, A., Cline, R., Clucas, S, Gaukroger, S., Cook, H., Hardy, A., Corry, L., Metraux, A., Renn, J., Dolan, B., Luckin, B., Duerbeck, H., Orchiston, W., Epple, M., Hård, M., Rheinberger H-J., Roelcke, V., Farber, P., Fissell, M., Packard, R., Fox, R., Frasca Spada, M., French, S., Good, J., Hackmann, W., Hllieux, R., Holmqvist, B., Home, R., HOskin, M., Inkster, I., Jardine, N., Levere, T., Lightman, B., Lüthy, C., Lynch, M., McCluskey, S., Ruggles, C., Morris, P., Rhys Morus, I., Nelson, E.C., Perez, L., Rigden, J., Stuewer, R.H., Samsó, J., Schaffer, S., Schappacher, N., Staudenmaier SJ, J., Strom, C., Unschuld, P, Weingart, P., Zamecki, S. & Zuidervaart, H. (2009) Journals under threat: A joint response from history of science, technology and medicine editors. Centaurus 51 1–4. https://doi.org/10.1111/j.1600-0498.2008.00140.x
CRIStin [Det nasjonale forskningsinformasjons-systemet] (2017) Reporting of academic publications in the health, institute and HE sectors. 28.3.2017. CRIStin – Current Research Information System in Norway, Oslo. <http://www.cristin.no/english/resources/reporting-instructions/> 1.11.2017.
Fossum-Raunehaug, S. (2017) Publication points and reward of publications at level 1 and 2. 3.7.2017. NMBU – Norwegian University of Life Sciences, Ås. <https://www.nmbu.no/en/research/for_researchers/publishing-abc/node/25300> 2.11.2017.
Gjengedal, K. (2017) Kvalitet er meir enn siteringar. Forskerforum 49(8) 6–7.
Halffman, W. & Radder, H. (2015) The academic manifesto: From an occupied to a public university. Minerva 53(2) 165–187. http://dx.doi.org/10.1007/s11024-015-9270-9
Halffman, W. & Radder, H. (eds.) (2017) International responses to the Academic Manifesto: Reports from 14 countries. Social Epistemology Review and Reply Collective, Special Report 2017. <http://wp.me/p1Bfg0-3FV> 2.11.2017.
Hansen, T.I. (2015) Tellekantsystemet. In Store norske leksikon. 20.2.2015. <https://snl.no/tellekantsystemet> 2.11.2017.
Haugstad, B. (2017) Om Aarebrot og tellekanter. Khrono. 27.9.2017. <https://khrono.no/debatt/snodig-om-aarebrot-og-tellekanter> 2.11.2017.
Hay, I. (2015) ‘Why edit a scholarly journal? Academic irony and paradox. The Professional Geographer 68(1) 159–165. https://doi.org/10.1080/00330124.2015.1062704
Kallio, K.P. (2017) Subtle radical moves in scientific publishing. Fennia 195(1) 1–4. https://doi.org/10.11143/fennia.63678
Kjelstadli, K. (2010) Akademisk kapitalismen. Forlaget Res Publica, Oslo.
Kunnskapsdepartementet (2016) Orientering om statsbudsjettet 2017 for universitet og høgskolar: Etter vedtak i Stortinget 17. desember 2016: Mål for universitet og høgskolar, budsjett og endringar i løyving og finansieringssystemet. Kunnskapsdepartementet, Oslo. <https://www.regjeringen.no/contentassets/31af8e2c3a224ac2829e48cc91d89083/orientering-om-statsbudsjettet-2017-for-universiteter-og-hoegskolar_ny-versjon160217.pdf> 2.11.2017.
Lamont, M. & Guetzkow, J. (2016) How quality is recognized by peer review panels: The case of the humanities. In Ochsner, M., Hug, S.E. & Daniel, H.-D. (eds.) Research assessment in the humanities: Towards criteria and procedures, 31–41. Springer Open. https://doi.org/10.1007/978-3-319-29016-4_4
Lund, R.W.B. (2015) Doing the ideal academic: Gender, excellence and changing academia. Doctoral dissertations 98/2015. Aalto University, Helsinki.
Myklebust, J.P. (2017a) In search of a new form of university governance. University World News 450. 10.03.2017. <http://www.universityworldnews.com/article.php?story=2017030918094136> 7.9.2017.
Myklebust, J.P. (2017b) Should universities be run like businesses? University World News 473. 8.9.2017. <http://www.universityworldnews.com/article.php?story=20170908102945748> 7.9.2017
NSD [Norsk senter for forskningsdata] (2017) Register over vitenskapelige publiseringskanaler: Kriterier for godkjenning av publiseringskanaler. NSD – Norsk senter for forskningsdata, Bergen. <https://dbh.nsd.uib.no/publiseringskanaler/OmKriterier> 2.11.2017.
Ochsner, M., Hug, S.E. & Daniel, H.-D. (2016) Research assessment in the humanities: Introduction. In Ochsner, M., Hug, S.E. & Daniel, H.-D. (eds.) Research assessment in the humanities: Towards criteria and procedures, 1–10. Springer Open. https://doi.org/10.1007/978-3-319-29016-4_1
Østerud, Ø. (2009) Forord. In Østerud, Ø. (ed.) Hvordan måle vitenskap? Søkelys på bibliometriske metoder, 5–7. Det Norske Videnskaps-Akademi – Novus forlag, Oslo. <http://www.dnva.no/binfil/download.php?tid=41358> 2.11.2017.
Paasi, A. (2013). Fennia: Positioning a ‘peripheral’ but international journal under conditions of academic capitalism. Fennia 191(1) 1–13. https://doi.org/10.11143/7787
van Reekum, R. (ed.) (2015) The new university: A special issue on the future of the university. Krisis 2015(2). <http://krisis.eu/the-new-university/> 2.11.2017.
Roll-Hansen, N. (2009) Om å “måle” kvalitet av forskning. In Østerud, Ø. (ed.) Hvordan måle vitenskap? Søkelys på bibliometriske metoder, 71–80. Det Norske Videnskaps-Akademi – Novus forlag, Oslo. <http://www.dnva.no/binfil/download.php?tid=41358> 2.11.2017.
Sandnes, F.E. (2016) Hvordan melke nye tellekanter. Khrono 27.4.2016. <https://khrono.no/debatt/hvordan-melke-tellekanter-i-2016> 2.11.2017.
Seglen, P.O. (1997) Why the impact factor of journals should not be used for evaluating research. BMJ 314, 498–502. https://doi.org/10.1136/bmj.314.7079.497
Seglen, P.O. (2009) Er tidsskrift-renommé og artikkeltelling adekvate mål for vitenskapelig kvalitet og kvantitet? In Østerud, Ø. (ed.) Hvordan måle vitenskap? Søkelys på bibliometriske metoder, 39–70. Det Norske Videnskaps-Akademi – Novus forlag, Oslo. <http://www.dnva.no/binfil/download.php?tid=41358> 2.11.2017.
Sivertsen, G. (2009) Publiseringsindikatoren. In Østerud, Ø. (ed.) Hvordan måle vitenskap? Søkelys på bibliometriske metoder, 11–37. Det Norske Videnskaps-Akademi – Novus forlag, Oslo. <http://www.dnva.no/binfil/download.php?tid=41358> 2.11.2017.
Sivertsen, G. (2016) Publication-based funding: The Norwegian model. In Ochsner, M., Hug, S.E. & Daniel, H.-D. (eds.) Research assessment in the humanities: Towards criteria and procedures, 79–90. Springer Open. https://doi.org/10.1007/978-3-319-29016-4_7
Sjøberg, S. (2017) Null poeng til Aarebrot? Khrono. 19.9.2017. <https://khrono.no/debatt/null-poeng-til-frank-aarebrot> 2.11.2017.
UHR [Universitets- og høgskolerådet] (n.d.) Publiseringskanaler. Universitets- og høgskolerådet, Oslo. <http://www.uhr.no/rad_og_utvalg/utvalg/det_nasjonale_publiseringsutvalget/publiseringskanaler> 2.11.2017.
Waarenperä, U. (ed.) (2011) Universitetsrankning och bibliometriska mätningar: Konsekvenser för forskning och kunskapsutveckling. Konferenser 74. Kungl. Vitterhets Historie och Antikvitets Akademien, Stockholm. <https://vitterhetsakad.bokorder.se/sv-SE/article/2103/universitetsrankning-och-bibliometriska-matni>
Downloads
Published
Issue
Section
License
Copyright (c) 2017 Michael Richard Handley Jones

This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with Fennia agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal. The license of the published metadata is Creative Commons CC0 1.0 Universal (CC0 1.0).
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a doctoral dissertation or book compilation), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
.png)



