This paper shows how bibliometric assessment can be implemented at individual level. This has been successfully done at the University of Vienna carried out by the Bibliometrics and Publication Strategies Department of the Vienna University Library. According to the department's philosophy, bibliometrics is not only a helpful evaluation instrument in order to complement the peer review system. It is also meant as a compass for researchers in the "publish or perish" dilemma in order to increase general visibility and to optimize publication strategies. The individual assessment comprises of an interview with the researcher under evaluation, the elaboration of a bibliometric report of the researcher's publication output, the discussion and validation of the obtained results with the researcher under evaluation as well as further optional analyses. The produced bibliometric reports are provided to the researchers themselves and inform them about the quantitative aspects of their research output. They also serve as a basis for further discussion concerning their publication strategies. These reports are eventually intended for informed peer review practices, and are therefore forwarded to the quality assurance and the rector's office and finally sent to the peers. The most important feature of the generated bibliometric report is its multidimensional and individual character. It relies on a variety of basic indicators and further control parameters in order to foster comprehensibility. Researchers, administrative staff and peers alike have confirmed the usefulness of this bibliometric approach. An increasing demand is noticeable. In total, 33 bibliometric reports have been delivered so far. Moreover, similar reports have also been produced for the bibliometric assessment of two faculties with great success.
The scientific output 1994-2014 of the University Centre in Svalbard (UNIS) was bibliometrically analysed. It was found that the majority of the papers have been published as international cooperations and rank above world average. Analysis of the content of the papers reveals that UNIS works and publishes in a wide variety of scientific topics.
The scientific paper output of the United Nations University (UNU) was bibliometrically analysed. It was found that (i) a noticeable continous paper output starts in 1995, (ii) about 65% of the research papers have been published as international cooperations and 18% as single-authored papers, (iv) the research papers rank above world average according to Pudovkin-Garfield Percentile Rank Index, and (v) paper content indicate the wide variety of scientific topics UNU has been and is working on.
The present work proposes a bibliometric methodology for measuring the grade of correspondence between regional industry's demand for research collaboration and supply from public laboratories. The methodology also permits measurement of the intensity and direction of the regional flows of knowledge in public-private collaborations. The aim is to provide a diagnostic instrument for regional and national policy makers, which could add to existing ones to plan interventions for re-balancing sectorial public supply of knowledge with industrial absorptive capacity, and maximizing appropriability o…
Although bibliometric studies for the assessment of scientific output at university-level are relatively common, data for performance at department-level rarely exist. In this paper we develop a methodology and tools for conducting department-level assessments, and conduct for the first time a complete bibliometric study of scientific output of all university departments in Greece. The study is based on data from Scopus about the number of scientific publications and respective citations of all faculty members in each department for the period 2017-2021. Code scripts were developed using R to …
University rankings by fields are usually based on the research output of universities. However, research managers and rankings consumers expect to see in such fields a reflection of the structure of their own organizational institution. In this study we address such misinterpretation by developing the research profile of the organizational units of two Spanish universities: University of Granada and Pompeu Fabra University. We use two classification systems, the subject categories offered by Thomson Scientific which are commonly used on bibliometric studies, and the 37 disciplines displayed b…
This work investigates public-private research collaboration between Italian universities and domestic industry, applying a bibliometric type of approach. The study is based on an exhaustive listing of all co-authored publications in international journals that are jointly realized by Italian university scientists and researchers in the private sector: this listing permits the development of a national mapping system for public-private collaboration, which results unique for its extensive and representative character. It is shown that, in absolute terms, most collaborations occur in medicine a…
Monitoring bibliometric indicators of University rankings is considered as a subject of a University library activity. In order to fulfill comparative assessment of research activities of the universities of Ukraine and Belarus the authors introduced a set of bibliometric indicators. A comparative assessment of the research activities of corresponding universities was fulfilled; the data on the leading universities are presented. The sensitivity of the one of the indicators to rapid changes of the research activity of universities and the fact that the other one is normalized across the fields…
In this research, we examine the hypothesis that gender disparities in international research collaboration differ by collaboration intensity, academic position, age, and academic discipline. The following are the major findings: (1) while female scientists exhibit a higher rate of general, national, and institutional collaboration, male scientists exhibit a higher rate of international collaboration, a finding critically important in explaining gender disparities in impact, productivity, and access to large grants. (2) An aggregated picture of gender disparities hides a more nuanced cross-dis…
The use of bibliometric indicators would simplify research assessments. The 2014 Research Excellence Framework (REF) is a peer review assessment of UK universities, whose results can be taken as benchmarks for bibliometric indicators. In this study we use the REF results to investigate whether the ep index and a top percentile of most cited papers could substitute for peer review. The probability that a random university's paper reaches a certain top percentile in the global distribution of papers is a power of the ep index, which can be calculated from the citation-based distribution of unive…
In this article, we conduct data mining to discover the countries, universities and companies, produced or collaborated the most research on Covid-19 since the pandemic started. We present some interesting findings, but despite analysing all available records on COVID-19 from the Web of Science Core Collection, we failed to reach any significant conclusions on how the world responded to the COVID-19 pandemic. Therefore, we increased our analysis to include all available data records on pandemics and epidemics from 1900 to 2020. We discover some interesting results on countries, universities an…
We applied a set of standard bibliometric indicators to monitor the scientific state-of-arte of 500 universities worldwide and constructed a ranking on the basis of these indicators (Leiden Ranking 2010). We find a dramatic and hitherto largely underestimated language effect in the bibliometric, citation-based measurement of research performance when comparing the ranking based on all Web of Science (WoS) covered publications and on only English WoS covered publications, particularly for Germany and France.
This paper presents a method for classifying the varying degrees of interdependency between academic hospitals and universities in the context of the Leiden Ranking. A key question for ranking universities is whether or not to allocate the publication output of affiliated hospitals to universities. Hospital nomenclatures vary worldwide to denote some form of collaboration with a university: academic hospitals, teaching hospitals, university hospitals, and academic medical centres do not correspond to universally standard definitions. Thus, rather than seeking a normative definition of academic…
National exercises for the evaluation of research activity by universities are becoming regular practice in ever more countries. These exercises have mainly been conducted through the application of peer-review methods. Bibliometrics has not been able to offer a valid large-scale alternative because of almost overwhelming difficulties in identifying the true author of each publication. We will address this problem by presenting a heuristic approach to author name disambiguation in bibliometric datasets for large-scale research assessments. The application proposed concerns the Italian universi…
Unlike competitive higher education systems, non-competitive systems show relatively uniform distributions of top professors and low performers among universities. In this study, we examine the impact of unproductive and top faculty members on overall research performance of the university they belong to. Furthermore, we analyze the potential relationship between research productivity of a university and the indexes of concentration of unproductive and top professors. Research performance is evaluated using a bibliometric approach, through publications indexed on the Web of Science between 200…
This is the first detailed study on the coverage of Microsoft Academic (MA). Based on the complete and verified publication list of a university, the coverage of MA was assessed and compared with two benchmark databases, Scopus and Web of Science (WoS), on the level of individual publications. Citation counts were analyzed, and issues related to data retrieval and data quality were examined. A Perl script was written to retrieve metadata from MA based on publication titles. The script is freely available on GitHub. We find that MA covers journal articles, working papers, and conference items t…
Blockchain is a shared, immutable ledger that has attracted the attention of researchers and practitioners across innumerable sectors, with its implications for modernizing payment systems having the possibility of inciting a digital revolution. In the scope of this study, 1,511 publications were obtained from Scopus to conduct a systematic review of the research space through bibliometric and network analyses. The main aim of this study was to determine key authors, significant studies, and collaboration patterns, to reveal the distributions and impacts of publications in the blockchain-based…
Bibliometric analysis has firmly conquered its place as an instrument for evaluation and international comparison of performance levels. Consequently, differences in coverage by standard bibliometric databases installed a dichotomy between on the one hand the well covered 'exact' sciences, and on the other hand most of the social sciences and humanities with a more limited coverage (Nederhof, 2006). Also the latter domains need to be able to soundly demonstrate their level of performance and claim or legitimate funding accordingly. An important part of the output volume in social sciences appe…
World University rankings have become well-established tools that students, university managers and policy makers read and use. Each ranking claims to have a unique methodology capable of measuring the 'quality' of universities. The purpose of this paper is to analyze to which extent these different rankings measure the same phenomenon and what it is that they are measuring. For this, we selected a total of seven world-university rankings and performed a principal component analysis. After ensuring that despite their methodological differences, they all come together to a single component, we …
The practice of collaboration, and particularly international collaboration, is becoming ever more widespread in scientific research, and is likewise receiving greater interest and stimulus from policy-makers. However, the relation between research performance and degree of internationalization at the level of single researchers still presents unresolved questions. The present work, through a bibliometric analysis of the entire Italian university population working in the hard sciences over the period 2001-2005, seeks to answer some of these questions. The results show that the researchers wit…
This paper employs data envelopment analysis (DEA) to assess both technical and cost efficiency of research activities of the Italian university system. Differently from both peer review and the top-down discipline-invariant bibliographic approaches used elsewhere, a bottom-up bibliometric methodology is applied. Publications are assigned first to authors and then to one of nine scientific and technical university disciplinary areas. Inputs are specified in terms of the numbers of full, associate and assistant professors and outputs as the number of publications, contributions to publications …
Google Scholar has been well received by the research community. Its promises of free, universal and easy access to scientific literature as well as the perception that it covers better than other traditional multidisciplinary databases the areas of the Social Sciences and the Humanities have contributed to the quick expansion of Google Scholar Citations and Google Scholar Metrics: two new bibliometric products that offer citation data at the individual level and at journal level. In this paper we show the results of a experiment undertaken to analyze Google Scholar's capacity to detect citati…
The current work proposes an application of DEA methodology for measurement of technical and allocative efficiency of university research activity. The analysis is based on bibliometric data from the Italian university system for the five year period 2004-2008. Technical and allocative efficiency is measured with input being considered as a university's research staff, classified according to academic rank, and with output considered as the field-standardized impact of the research product realized by these staff. The analysis is applied to all scientific disciplines of the so-called hard scie…
It is widely recognized that collaboration between the public and private research sectors should be stimulated and supported, as a means of favoring innovation and regional development. This work takes a bibliometric approach, based on co-authorship of scientific publications, to propose a model for comparative measurement of the performance of public research institutions in collaboration with the domestic industry collaboration with the private sector. The model relies on an identification and disambiguation algorithm developed by the authors to link each publication to its real authors. An…
In this study, we combine bibliometric techniques with a machine learning algorithm, the sequential Information Bottleneck, to assess the interdisciplinarity of research produced by the University of Hawaii NASA Astrobiology Institute (UHNAI). In particular, we cluster abstract data to evaluate Thomson Reuters Web of Knowledge subject categories as descriptive labels for astrobiology documents, assess individual researcher interdisciplinarity, and determine where collaboration opportunities might occur. We find that the majority of the UHNAI team is engaged in interdisciplinary research, and s…
There has been ample demonstration that bibliometrics is superior to peer-review for national research assessment exercises in the hard sciences. In this paper we examine the Italian case, taking the 2001-2003 university performance rankings list based on bibliometrics as benchmark. We compare the accuracy of the first national evaluation exercise, conducted entirely by peer-review, to other rankings lists prepared at zero cost, based on indicators indirectly linked to performance or available on the Internet. The results show that, for the hard sciences, the costs of conducting the Italian ev…
Most Performance-based Research Funding Systems (PRFS) draw on peer review and bibliometric indicators, two different methodologies which are sometimes combined. A common argument against the use of indicators in such research evaluation exercises is their low correlation at the article level with peer review judgments. In this study, we analyse 191,000 papers from 154 higher education institutes which were peer reviewed in a national research evaluation exercise. We combine these data with 6.95 million citations to the original papers. We show that when citation-based indicators are applied a…
Revolutions in science: The proposal of an approach for the identification of most important researchers, institutions, and countries based on Reference Publication Year Spectroscopy (RPYS)
Objective. To perform a modern bibliometric analysis of the research based on the Roy Adaptation Model, a founding nursing model proposed by Sor Callista Roy in the1970s. Method. A descriptive and longitudinal study. We used information from the two dominant scientific databases, Web Of Science and SCOPUS. We obtained 137 publications from the Core Collection of WoS, and 338 publications from SCOPUS. We conducted our analysis using the software Bibliometrix, an R-package specialized in creating bibliometric analyses from a perspective of descriptive statistics and network analysis, including c…
The Leiden Ranking 2011/2012 is a ranking of universities based on bibliometric indicators of publication output, citation impact, and scientific collaboration. The ranking includes 500 major universities from 41 different countries. This paper provides an extensive discussion of the Leiden Ranking 2011/2012. The ranking is compared with other global university rankings, in particular the Academic Ranking of World Universities (commonly known as the Shanghai Ranking) and the Times Higher Education World University Rankings. Also, a detailed description is offered of the data collection methodo…
The 2004-2010 VQR, completed in July 2013, was Italy's second national research assessment exercise. The VQR performance evaluation followed a pattern also seen in other nations, in being based on a selected subset of products. In this work we identify the exercise's methodological weaknesses and measure the distortions that result from them in the university performance rankings. First we create a scenario in which we assume the efficient selection of the products to be submitted by the universities and from this simulate a set of rankings applying the precise VQR rating criteria. Next we com…
The field of Energy System Analysis (ESA) has experienced exponential growth in the number of publications since at least the year 2000. This paper presents a comprehensive bibliometric analysis on ESA by employing different algorithms in Matlab and R. The focus of results is on quantitative indicators relating to number and type of publication outputs, collaboration links between institutions, authors and countries, and dynamic trends within the field. The five and twelve most productive countries have 50% and 80% of ESA publications respectively. The dominant institutions are even more conce…
Accurate measurement of research productivity should take account of both the number of co-authors of every scientific work and of the different contributions of the individuals. For researchers in the life sciences, common practice is to indicate such contributions through position in the authors list. In this work, we measure the distortion introduced to bibliometric ranking lists for scientific productivity when the number of co-authors or their position in the list is ignored. The field of observation consists of all Italian university professors working in the life sciences, with scientif…
There is an evident and rapid trend towards the adoption of evaluation exercises for national research systems for purposes, among others, of improving allocative efficiency in public funding of individual institutions. However the desired macroeconomic aims could be compromised if internal redistribution of government resources within each research institution does not follow a consistent logic: the intended effects of national evaluation systems can result only if a "funds for quality" rule is followed at all levels of decision-making. The objective of this study is to propose a bibliometric…
In the context of a comprehensive Microsoft Academic (MA) study, we explored in an initial step the quality of linked references data in MA in comparison with Web of Science (WoS). Linked references are the backbone of bibliometrics, because they are the basis of the times cited information in citation indexes. We found that the concordance of linked references between MA and WoS ranges from weak to non-existent for the full sample (publications of the University of Zurich with less than 50 linked references in MA). An analysis with a sample restricted to less than 50 linked references in WoS …
International scientific mobility is acknowledged to be a key mechanism for the diffusion of knowledge, particularly tacit or 'sticky' knowledge that cannot be transferred without geographical proximity and personal contact, for the incorporation of young researchers into elite transnational scientific networks, and for accessing additional resources or infrastructures that are essential to the research process but located in other places. The inadequacy and lack of appropriate data to assess the phenomenon of researcher mobility has been repeatedly pointed out by scholars and policy makers. T…
One of the most problematic aspects in the creation of spin-offs by university personnel concerns the relationship between entrepreneurial activity and research activity by researcher-entrepreneurs. The literature has expressed varying and opposing views as to the nature of the relationship but very little has been produced to empirically legitimate one position or another. The present work proposes to address this shortcoming by exploring the relationship existing between academic spin-off generation and the research performance of enterprise founders. The study investigates whether, and to w…
We are witnessing a rapid trend towards the adoption of exercises for evaluation of national research systems, generally based on the peer review approach. They respond to two main needs: stimulating higher efficiency in research activities by public laboratories, and realizing better allocative efficiency in government funding of such institutions. However the peer review approach is typified by several limitations that raise doubts for the achievement of the ultimate objectives. In particular, subjectivity of judgment, which occurs during the step of selecting research outputs to be submitte…
