Can research quality be measured quantitatively? On quality of scholarship, numerical research indicators and academic publishing - experiences from Norway

Authors

  • Michael Richard Handley Jones

DOI:

https://doi.org/10.11143/fennia.66602

Abstract

In this article I reflect on ways in which the neoliberal university and its administrative counterpart, new public management (NPM), affect academic publishing activity. One characteristic feature of NPM is the urge to use simple numerical indicators of research output as a tool to allocate funding and, in practice if not in theory, as a means of assessing research quality. This ranges from the use of journal impact factors (IF) and ranking of journals to publication points to determine what types of work in publishing is counted as meritorious for funding allocation. I argue that it is a fallacy to attempt to assess quality of scholarship through quantitative measures of publication output. I base my arguments on my experiences of editing a Norwegian geographical journal over a period of 16 years, along with my experiences as a scholar working for many years within the Norwegian university system.

 

Norwegian translation of the article is published in open access book Universitetskamp.

References

Aagaard, K, Bloch, C. & Schneider, J.W. (2015) Impacts of performance-based research funding systems: The case of the Norwegian publication indicator. Research Evaluation 24, 106–117. https://doi.org/10.1093/reseval/rvv003

Aagaard, K., Bloch,C., Schneider, J.W., Henriksen, D., Ryan, T.K. & Lauridsen, P.S. (2014) Evaluering af den norske publiceringsindikator. Dansk Center for Forskningsanalyse, Aarhus Universitet, Aarhus. <https://npi.nsd.no/dok/eval2014/Evaluering_af_den_norske_publiceringsindikator_2014.pdf> 2.11.2017.

Aksnes, D.W. (2017) Artikler I nivå 2-tidsskrifter blir mest sitert. Forskerforum. 5.10.2017. <http://www.forskerforum.no/artikler-i-niva-2-tidsskrifter-blir-mest-sitert/> 2.11.2017.

Andersen, H., Ariew, R., Feingold, M., Bag, A.K., Barrow-Green, J., Dalen, B., Benson, K., Beretta, M. Blay, M., Bleker, J., Borck, C., Bowker, G., Leigh Star, S., Buccianti, M., Camerota, M., Buchwald, J, Gray, J., Cappelletti, V., Cimino, G., Carson, C., Clark, M., Keller, A., Cline, R., Clucas, S, Gaukroger, S., Cook, H., Hardy, A., Corry, L., Metraux, A., Renn, J., Dolan, B., Luckin, B., Duerbeck, H., Orchiston, W., Epple, M., Hård, M., Rheinberger H-J., Roelcke, V., Farber, P., Fissell, M., Packard, R., Fox, R., Frasca Spada, M., French, S., Good, J., Hackmann, W., Hllieux, R., Holmqvist, B., Home, R., HOskin, M., Inkster, I., Jardine, N., Levere, T., Lightman, B., Lüthy, C., Lynch, M., McCluskey, S., Ruggles, C., Morris, P., Rhys Morus, I., Nelson, E.C., Perez, L., Rigden, J., Stuewer, R.H., Samsó, J., Schaffer, S., Schappacher, N., Staudenmaier SJ, J., Strom, C., Unschuld, P, Weingart, P., Zamecki, S. & Zuidervaart, H. (2009) Journals under threat: A joint response from history of science, technology and medicine editors. Centaurus 51 1–4. https://doi.org/10.1111/j.1600-0498.2008.00140.x

CRIStin [Det nasjonale forskningsinformasjons-systemet] (2017) Reporting of academic publications in the health, institute and HE sectors. 28.3.2017. CRIStin – Current Research Information System in Norway, Oslo. <http://www.cristin.no/english/resources/reporting-instructions/> 1.11.2017.

Fossum-Raunehaug, S. (2017) Publication points and reward of publications at level 1 and 2. 3.7.2017. NMBU – Norwegian University of Life Sciences, Ås. <https://www.nmbu.no/en/research/for_researchers/publishing-abc/node/25300> 2.11.2017.

Gjengedal, K. (2017) Kvalitet er meir enn siteringar. Forskerforum 49(8) 6–7.
Halffman, W. & Radder, H. (2015) The academic manifesto: From an occupied to a public university. Minerva 53(2) 165–187. http://dx.doi.org/10.1007/s11024-015-9270-9

Halffman, W. & Radder, H. (eds.) (2017) International responses to the Academic Manifesto: Reports from 14 countries. Social Epistemology Review and Reply Collective, Special Report 2017. <http://wp.me/p1Bfg0-3FV> 2.11.2017.

Hansen, T.I. (2015) Tellekantsystemet. In Store norske leksikon. 20.2.2015. <https://snl.no/tellekantsystemet> 2.11.2017.

Haugstad, B. (2017) Om Aarebrot og tellekanter. Khrono. 27.9.2017. <https://khrono.no/debatt/snodig-om-aarebrot-og-tellekanter> 2.11.2017.

Hay, I. (2015) ‘Why edit a scholarly journal? Academic irony and paradox. The Professional Geographer 68(1) 159–165. https://doi.org/10.1080/00330124.2015.1062704

Kallio, K.P. (2017) Subtle radical moves in scientific publishing. Fennia 195(1) 1–4. https://doi.org/10.11143/fennia.63678

Kjelstadli, K. (2010) Akademisk kapitalismen. Forlaget Res Publica, Oslo.

Kunnskapsdepartementet (2016) Orientering om statsbudsjettet 2017 for universitet og høgskolar: Etter vedtak i Stortinget 17. desember 2016: Mål for universitet og høgskolar, budsjett og endringar i løyving og finansieringssystemet. Kunnskapsdepartementet, Oslo. <https://www.regjeringen.no/contentassets/31af8e2c3a224ac2829e48cc91d89083/orientering-om-statsbudsjettet-2017-for-universiteter-og-hoegskolar_ny-versjon160217.pdf> 2.11.2017.

Lamont, M. & Guetzkow, J. (2016) How quality is recognized by peer review panels: The case of the humanities. In Ochsner, M., Hug, S.E. & Daniel, H.-D. (eds.) Research assessment in the humanities: Towards criteria and procedures, 31–41. Springer Open. https://doi.org/10.1007/978-3-319-29016-4_4

Lund, R.W.B. (2015) Doing the ideal academic: Gender, excellence and changing academia. Doctoral dissertations 98/2015. Aalto University, Helsinki.

Myklebust, J.P. (2017a) In search of a new form of university governance. University World News 450. 10.03.2017. <http://www.universityworldnews.com/article.php?story=2017030918094136> 7.9.2017.

Myklebust, J.P. (2017b) Should universities be run like businesses? University World News 473. 8.9.2017. <http://www.universityworldnews.com/article.php?story=20170908102945748> 7.9.2017

NSD [Norsk senter for forskningsdata] (2017) Register over vitenskapelige publiseringskanaler: Kriterier for godkjenning av publiseringskanaler. NSD – Norsk senter for forskningsdata, Bergen. <https://dbh.nsd.uib.no/publiseringskanaler/OmKriterier> 2.11.2017.

Ochsner, M., Hug, S.E. & Daniel, H.-D. (2016) Research assessment in the humanities: Introduction. In Ochsner, M., Hug, S.E. & Daniel, H.-D. (eds.) Research assessment in the humanities: Towards criteria and procedures, 1–10. Springer Open. https://doi.org/10.1007/978-3-319-29016-4_1

Østerud, Ø. (2009) Forord. In Østerud, Ø. (ed.) Hvordan måle vitenskap? Søkelys på bibliometriske metoder, 5–7. Det Norske Videnskaps-Akademi – Novus forlag, Oslo. <http://www.dnva.no/binfil/download.php?tid=41358> 2.11.2017.

Paasi, A. (2013). Fennia: Positioning a ‘peripheral’ but international journal under conditions of academic capitalism. Fennia 191(1) 1–13. https://doi.org/10.11143/7787

van Reekum, R. (ed.) (2015) The new university: A special issue on the future of the university. Krisis 2015(2). <http://krisis.eu/the-new-university/> 2.11.2017.

Roll-Hansen, N. (2009) Om å “måle” kvalitet av forskning. In Østerud, Ø. (ed.) Hvordan måle vitenskap? Søkelys på bibliometriske metoder, 71–80. Det Norske Videnskaps-Akademi – Novus forlag, Oslo. <http://www.dnva.no/binfil/download.php?tid=41358> 2.11.2017.

Sandnes, F.E. (2016) Hvordan melke nye tellekanter. Khrono 27.4.2016. <https://khrono.no/debatt/hvordan-melke-tellekanter-i-2016> 2.11.2017.

Seglen, P.O. (1997) Why the impact factor of journals should not be used for evaluating research. BMJ 314, 498–502. https://doi.org/10.1136/bmj.314.7079.497

Seglen, P.O. (2009) Er tidsskrift-renommé og artikkeltelling adekvate mål for vitenskapelig kvalitet og kvantitet? In Østerud, Ø. (ed.) Hvordan måle vitenskap? Søkelys på bibliometriske metoder, 39–70. Det Norske Videnskaps-Akademi – Novus forlag, Oslo. <http://www.dnva.no/binfil/download.php?tid=41358> 2.11.2017.

Sivertsen, G. (2009) Publiseringsindikatoren. In Østerud, Ø. (ed.) Hvordan måle vitenskap? Søkelys på bibliometriske metoder, 11–37. Det Norske Videnskaps-Akademi – Novus forlag, Oslo. <http://www.dnva.no/binfil/download.php?tid=41358> 2.11.2017.

Sivertsen, G. (2016) Publication-based funding: The Norwegian model. In Ochsner, M., Hug, S.E. & Daniel, H.-D. (eds.) Research assessment in the humanities: Towards criteria and procedures, 79–90. Springer Open. https://doi.org/10.1007/978-3-319-29016-4_7

Sjøberg, S. (2017) Null poeng til Aarebrot? Khrono. 19.9.2017. <https://khrono.no/debatt/null-poeng-til-frank-aarebrot> 2.11.2017.

UHR [Universitets- og høgskolerådet] (n.d.) Publiseringskanaler. Universitets- og høgskolerådet, Oslo. <http://www.uhr.no/rad_og_utvalg/utvalg/det_nasjonale_publiseringsutvalget/publiseringskanaler> 2.11.2017.

Waarenperä, U. (ed.) (2011) Universitetsrankning och bibliometriska mätningar: Konsekvenser för forskning och kunskapsutveckling. Konferenser 74. Kungl. Vitterhets Historie och Antikvitets Akademien, Stockholm. <https://vitterhetsakad.bokorder.se/sv-SE/article/2103/universitetsrankning-och-bibliometriska-matni>

Downloads

Published

2017-12-15

Issue

Section

Reflections

How to Cite

Can research quality be measured quantitatively? On quality of scholarship, numerical research indicators and academic publishing - experiences from Norway. (2017). Fennia - International Journal of Geography, 195(2), 164-174. https://doi.org/10.11143/fennia.66602