We have developed a method to obtain robust quantitative bibliometric indicators for several thousand scientists. This allows us to study the dependence of bibliometric indicators (such as number of publications, number of citations, Hirsch index...) on the age, position, etc. of CNRS scientists. Our data suggests that the normalized h index (h divided by the career length) is not constant for scientists with the same productivity but differents ages. We also compare the predictions of several bibliometric indicators on the promotions of about 600 CNRS researchers. Contrary to previous publications, our study encompasses most disciplines, and shows that no single indicator is the best predictor for all disciplines. Overall, however, the Hirsch index h provides the least bad correlations, followed by the number of papers published. It is important to realize however that even h is able to recover only half of the actual promotions. The number of citations or the mean number of citations per paper are definitely not good predictors of promotion.
Bibliometric indicators, citation counts and/or download counts are increasingly being used to inform personnel decisions such as hiring or promotions. These statistics are very often misused. Here we provide a guide to the factors which should be considered when using these so-called quantitative measures to evaluate people. Rules of thumb are given for when begin to use bibliometric measures when comparing otherwise similar candidates.
We use confirmatory factor analysis to derive a unifying measure of comparison of scientists based on bibliometric measurements, by utilizing the h-index, some similar h-type indices as well as other common measures of scientific performance. We use a real data example from nine well-known departments of statistics to demonstrate our approach and argue that our combined measure results in a better overall evaluation of a researchers' scientific work.
Citation numbers and other quantities derived from bibliographic databases are becoming standard tools for the assessment of productivity and impact of research activities. Though widely used, still their statistical properties have not been well established so far. This is especially true in the case of bibliometric indicators aimed at the evaluation of individual scholars, because large-scale data sets are typically difficult to be retrieved. Here, we take advantage of a recently introduced large bibliographic data set, Google Scholar Citations, which collects the entire publication record o…
Quantitative bibliometric indicators are widely used to evaluate the performance of scientists. However, traditional indicators do not much rely on the analysis of the processes intended to measure and the practical goals of the measurement. In this study, I propose a simple framework to measure and predict an individual researcher's scientific performance that takes into account the main regularities of publication and citation processes and the requirements of practical tasks. Statistical properties of the new indicator - a scientist's personal impact rate - are illustrated by its applicatio…
It has recently been claimed by Baccini and coauthors that due to ANVUR's bibliometric evaluations of individuals, departments, and universities, in Italy there has been a surge in self-citations in the last ten years, thus increasing the "inwardness" of Italian research more than has happened abroad. We have studied the database of Ioannidis et al. published on 12 August 2019 of the one hundred thousand most "highly cited" scientists, including about two thousand Italians, and we found that the problem of self-citations in relation to this scientific elite is not significant in Italy, while p…
This Brief Communication discusses the benefits of citation analysis in research evaluation based on Galton's "Wisdom of Crowds" (1907). Citations are based on the assessment of many which is why they can be ascribed a certain amount of accuracy. However, we show that citations are incomplete assessments and that one cannot assume that a high number of citations correlate with a high level of usefulness. Only when one knows that a rarely cited paper has been widely read is it possible to say (strictly speaking) that it was obviously of little use for further research. Using a comparison with '…
Why it has become more difficult to predict Nobel Prize winners: a bibliometric analysis of Nominees and Winners of the Chemistry and Physics Prizes (1901-2007)
The h-index is a popular bibliometric indicator for assessing individual scientists. We criticize the h-index from a theoretical point of view. We argue that for the purpose of measuring the overall scientific impact of a scientist (or some other unit of analysis) the h-index behaves in a counterintuitive way. In certain cases, the mechanism used by the h-index to aggregate publication and citation statistics into a single number leads to inconsistencies in the way in which scientists are ranked. Our conclusion is that the h-index cannot be considered an appropriate indicator of a scientist's …
Many discussions have enlarged the literature in Bibliometrics since the Hirsh proposal, the so called $h$-index. Ranking papers according to their citations, this index quantifies a researcher only by its greatest possible number of papers that are cited at least $h$ times. A closed formula for $h$-index distribution that can be applied for distinct databases is not yet known. In fact, to obtain such distribution, the knowledge of citation distribution of the authors and its specificities are required. Instead of dealing with researchers randomly chosen, here we address different groups based…
How to quantify the impact of a researcher's or an institution's body of work is a matter of increasing importance to scientists, funding agencies, and hiring committees. The use of bibliometric indicators, such as the h-index or the Journal Impact Factor, have become widespread despite their known limitations. We argue that most existing bibliometric indicators are inconsistent, biased, and, worst of all, susceptible to manipulation. Here, we pursue a principled approach to the development of an indicator to quantify the scientific impact of both individual researchers and research institutio…
The growing importance of citation-based bibliometric indicators in shaping the prospects of academic careers incentivizes scientists to boost the numbers of citations they receive. Whereas the exploitation of self-citations has been extensively documented, the impact of reciprocated citations has not yet been studied. We study reciprocity in a citation network of authors, and compare it with the average reciprocity computed in an ensemble of null network models. We show that obtaining citations through reciprocity correlates negatively with a successful career in the long term. Nevertheless, …
The Hirsch's $h$-index is perhaps the most popular citation-based measure of the scientific excellence. In 2013 G. Ionescu and B. Chopard proposed an agent-based model for this index to describe a publications and citations generation process in an abstract scientific community. With such an approach one can simulate a single scientist's activity, and by extension investigate the whole community of researchers. Even though this approach predicts quite well the $h$-index from bibliometric data, only a solution based on simulations was given. In this paper, we complete their results with exact, …
Rather than "measuring" a scientist impact through the number of citations which his/her published work can have generated, isn't it more appropriate to consider his/her value through his/her scientific network performance illustrated by his/her co-author role, thus focussing on his/her joint publications, - and their impact through citations? Whence, on one hand, this paper very briefly examines bibliometric laws, like the $h$-index and subsequent debate about co-authorship effects, but on the other hand, proposes a measure of collaborative work through a new index. Based on data about the pu…
The basic indicators of a researcher's productivity and impact are still the number of publications and their citation counts. These metrics are clear, straightforward, and easy to obtain. When a ranking of scholars is needed, for instance in grant, award, or promotion procedures, their use is the fastest and cheapest way of prioritizing some scientists over others. However, due to their nature, there is a danger of oversimplifying scientific achievements. Therefore, many other indicators have been proposed including the usage of the PageRank algorithm known for the ranking of webpages and its…
