The way to evaluate publications has always been to look at in which journals it had been published. The Journal Impact Factor (JIF) was key indicator while writing for a grant application or promotion. To take the next step on the career ladder you had to publish x papers in Q1 journals. In the end, it was not important if the publications in those Q1 journals got any citations,
the fact that they were published there was enough. In 2012, the San Francisco Declaration on Research Assessment (DORA) was signed, the signatories promised to eliminate the use of journal-based metrics such as the JIF in funding, appointment and promotion considerations. The Association of Universities in the Netherlands (VSNU) also signed DORA, and in 2018 the main research funders in the Netherlands followed. Therefore, the University Medical Center has to stop using the traditional metrics. The Central Medial Library (CMB) took the initiative to introduce a new way of evaluation to the UMCG.
Aims and objectives
Further develop a fair way to measure the impact of research based on article-level-metrics. In the end, research assessment should be a fair one, where every researcher, or a group can be evaluated by the quality of their work.
The method was first developed by a senior adviser research policy of the University of Groningen in June of 2019. The analysis
is named “Profile analysis” and aims to help the researcher to get an insight into their research quality by creating a ‘profile’ and to help the Research Office assess research staff and discover talent. For now, the tools used
are Elsevier’s Pure, Scopus, SciVal and Microsoft Excel. The researcher’s Scopus profile is imported into SciVal or a validated publication set from Pure for a department or group is used. From SciVal the publications, including all kinds of metrics as the ‘field weighted citation impact’ and topic clusters, are exported to Excel and there a profile is created based on article metrics and topics. Moreover, all this data gives the opportunity to show how well a researcher is performing and show performance within different topics. Because of the use of normalized metrics, this method is also very suitable for benchmarking.
The results so far have been positive, because of the high demand for the analysis by researchers. The CMB created this “profile analysis” for 20 researchers and for three departments in the first four months after presenting it to the Research Office in the UMCG. Researchers used the results to write narratives for their grant applications; departments used it to know their strong research areas and how they compare to others.
While this method is still under development, the first results seem promising. Because the method looks at article level metrics and different topics, it is considered fairer than the old system with JIF. More discussion and development is needed in the design and creation of this profile analysis and how it can be best used by researchers and departments
|Event title||European Association for Health Information and Libraries (EAHIL): Be Open Act Together|
|Degree of Recognition||International|