Understanding Research Quality as Basis of Research Assessment
Dr Michael Ochsner, Senior researcher, FORS, Lausanne, Switzerland
In his presentation at the 17:15 Colloquium by the ETH Library, Dr Michael Ochsner will talk about the results from a decade of research on research quality in the social sciences and humanities and what the sciences can learn from it.
There is currently a lot of discussion about the reform of research assessment. What seems to have started with DORA and the Leiden Manifesto, and is currently developing rapidly in the context of the Coalition for Advancing Research Assessment (CoARA), actually has a long history in the social sciences and humanities (SSH).
The metric approach to research evaluation, based on mainly English-language journal articles indexed in one or two commercial databases and ignoring any societal relevance and other tasks of scientific work, has long been criticised.
But there are also problems with traditional methods of evaluation. While academics evaluate research all the time when they read, teach, write, etc., they struggle when they have to define research quality, as the famous quote "I know it when I see it" shows. There are many reasons for this, e.g. judging quality involves implicit knowledge, quality is multidimensional and therefore different criteria need to be weighed against each other, or quality is highly contextual and therefore judging quality needs to be adaptive.
In his presentation, Michael Ochsner draws on his research projects on research quality in the social sciences and humanities. He compares how scholars assess quality criteria in different evaluation situations and across disciplines and country contexts. He shows that research quality is a complex construct and that responsible evaluation of research requires a broad set of criteria. While the studies he presents are based in the humanities, social sciences and arts, he discusses how the results are relevant to science, technology, engineering and medicine disciplines.
Michael Ochsner is Senior Researcher at FORS, the Swiss Centre of Expertise in Social Sciences, in Lausanne, Switzerland and at the Centre for Reproducible Research at the University of Zurich.
His work focuses on conceptual frameworks for research evaluation, paying special attention to cross-national differences in research policy and evaluation as well as how to identify research quality. He also specialises in survey methodology and open data.
He is president of the European Network for Research Evaluation in the Social Science and Humanities (ENRESSH), chair of the Scientific Committee of the European Values Study and a member of the Editorial Advisory Boards of Research Evaluation and Nature’s Humanities and Social Sciences Communications.