Bibliometrics provides accurate, cheap and simple descriptions of research systems and should lay the foundations for research policy. However, disconnections between bibliometric knowledge and research policy frequently misguide the research policy in many countries. A way of correcting these disconnections might come from the use of simple indicators of research performance. One such simple indicator is the number of highly cited researchers, which can be used under the assumption that a research system that produces and employs many highly cited researchers will be more successful than others with fewer of them. Here, we validate the use of the number of highly cited researchers (Ioannidis et al. 2020; PLoS Biol 18(10): e3000918) for research assessment at the country level and determine a country ranking of research success. We also demonstrate that the number of highly cited researchers reported by Clarivate Analytics is also an indicator of the research success of countries. The formal difference between the numbers of highly cited researchers according to Ionannidis et al. and Clarivate Analytics is that evaluations based on these two lists of highly cited researchers are approximately equivalent to evaluations based on the top 5% and 0.05% of highly cited papers, respectively. Moreover, the Clarivate Analytics indicator is flawed in some countries.
A few weeks ago, Thomson Reuters published a list of the highly cited researchers worldwide (highlycited.com). Since the data is freely available for downloading and includes the names of the researchers' institutions, we produced a ranking of the institutions on the basis of the number of highly cited researchers per institution. This ranking is intended to be a helpful amendment of other available institutional rankings.
Various studies have attempted to assess the amount of free full text available on the web and recent work have suggested that we are close to the 50% mark for freely available articles (Archambault et al. 2013; Bjork et al. 2010; Jamali and Nabavi 2015). It is natural to wonder if this might reduce researchers' reliance on library subscriptions for access. To do so, we need to determine not just what papers researchers are citing to that are free today, but to estimate if the papers they were citing were freely available at the time they were citing it. We attempt to do so for a sample of cit…
Nowadays impact factor is the significant indicator for journal evaluation. In impact factor calculation is used number of all citations to journal, regardless of the prestige of cited journals, however, scientific units (paper, researcher, journal or scientific organization) cited by journals with high impact factor or researchers with high Hirsch index are more important than objects cited by journals without impact factor or unknown researcher. In this paper was offered weighted impact factor for getting more accurate rankings for journals, which consider not only quantity of citations, but…
We propose a new index, the $j$-index, which is defined for an author as the sum of the square roots of the numbers of citations to each of the author's publications. The idea behind the $j$-index it to remedy a drawback of the $h$-index $-$ that the $h$-index does not take into account the full citation record of a researcher. The square root function is motivated by our desire to avoid the possible bias that may occur with a simple sum when an author has several very highly cited papers. We compare the $j$-index to the $h$-index, the $g$-index and the total citation count for three subject a…
Nations can be distinguished in terms of whether domestic or international research is cited. We analyzed the research output in natural sciences of three leading European research economies (Germany, the Netherlands, and the UK) and ask where their researchers look for the knowledge that underpins their most highly-cited papers. Is one internationally oriented or is citation limited to national resources? Do the citation patterns reflect a growing differentiation between the domestic and international research enterprise? To evaluate change over time, we include natural-sciences papers publis…
Clarivate Analytics announces highly cited researchers (HCRs) every November to recognise true pioneers in their respective fields over the last decade who are one in 1,000 according to citation analysis based on the Web of Science database. However, the scientometric rules underlying HCR selections have constantly evolved over the years; thus, a comparative study between HCRs' academic relevance before and after 2018, when the cross field started to be included in HCR statistics, is essential. This paper evaluated the consistency of measurements in 2017 and 2018 by analysing HCR distributions…
We propose a simple new index, named the $CI$-index, based on the Choquet integral to characterize the scientific output of researchers. This index is an improvement of the $A$-index and $R$-index and has a notable feature that highly cited papers have highly weights and lowly cited papers have lowly weights. In applications many researchers may have the same $h$-index, $g$-index or $R$-index. The $CI$-index can be provided an effective method of distinguish among such researchers.
Recently, Thomson Reuters has published its 2014 list of highly cited researchers (HCRs)[1]. Initial studies over the list [2] suggested that some universities (for instance, King Abdulaziz University in Saudi Arabia) may have been manipulating its world ranking by contracting with highly cited researchers. In this work, we analyse the ranking of other Saudi universities based solely on the list. Our analysis suggests that other universities in Saudi Arabia do not follow the steps of King Abdulaziz University when it comes to contracting with HCRs.
h-index retrieved by citation indexes (Scopus, Google scholar, and Web of Science) is used to measure the scientific performance and the research impact studies based on the number of publications and citations of a scientist. It also is easily available and may be used for performance measures of scientists, and for recruitment decisions. The aim of this study is to investigate the difference between the outputs and results from these three citation databases namely Scopus, Google Scholar, and Web of Science based upon the h-index of a group of highly cited researchers (Nobel Prize winner sci…
Do highly productive researchers have significantly higher probability to produce top cited papers? Or does the increased productivity in science only result in a sea of irrelevant papers as a perverse effect of competition and the increased use of indicators for research evaluation and accountability focus? We use a Swedish author disambiguated data set consisting of 48,000 researchers and their WoS-publications during the period of 2008 2011 with citations until 2014 to investigate the relation between productivity and production of highly cited papers. As the analysis shows, quantity does m…
The research assessments of countries or institutions should reveal their contribution to the advancement of science. Taking into consideration the correlation that exists between scientific impact and number of citations, research assessments can be based on citation counts. However, it is crucial to perform the counts in the heavy tail of citation distributions. Following this heavy tail approach, all measurements performed in Spanish universities show that their research is inefficient because the proportion of highly cited papers is much lower than in the most scientifically advanced count…
Citation analysis does not generally take the quality of citations into account: all citations are weighted equally irrespective of source. However, a scholar may be highly cited but not highly regarded: popularity and prestige are not identical measures of esteem. In this study we define popularity as the number of times an author is cited and prestige as the number of times an author is cited by highly cited papers. Information Retrieval (IR) is the test field. We compare the 40 leading researchers in terms of their popularity and prestige over time. Some authors are ranked high on prestige …
Is more always better? We address this question in the context of bibliometric indices that aim to assess the scientific impact of individual researchers by counting their number of highly cited publications. We propose a simple model in which the number of citations of a publication depends not only on the scientific impact of the publication but also on other 'random' factors. Our model indicates that more need not always be better. It turns out that the most influential researchers may have a systematically lower performance, in terms of highly cited publications, than some of their less in…
This technical report presents a bibliometric analysis of the top 30 cited researchers from USA, UK and China. The analysis is based on Google Scholar data using CIDS. The researchers were identified using their email suffix: edu, uk and cn. This naïve approach was able to produce rankings consistent with the SCImago country rankings using mininal resources in a fully automated way.
PP(top x%) is the proportion of papers of a unit (e.g. an institution or a group of researchers), which belongs to the x% most frequently cited papers in the corresponding fields and publication years. It has been proposed that x% of papers can be expected which belongs to the x% most frequently cited papers. In this Letter to the Editor we will present the results of an empirical test whether we can really have this expectation and how strong the deviations from the expected values are when many random samples are drawn from the database.
Recently, a vast number of scientific publications have been produced in cities in emerging countries. It has long been observed that the publication output of Beijing has exceeded that of any other city in the world, including such leading centres of science as Boston, New York, London, Paris, and Tokyo. Researchers have suggested that, instead of focusing on cities' total publication output, the quality of the output in terms of the number of highly cited papers should be examined. However, in the period from 2014 to 2016, Beijing produced as many highly cited papers as Boston, London, or Ne…
The COVID-19 pandemic requires a fast response from researchers to help address biological, medical and public health issues to minimize its impact. In this rapidly evolving context, scholars, professionals and the public may need to quickly identify important new studies. In response, this paper assesses the coverage of scholarly databases and impact indicators during 21 March to 18 April 2020. The results confirm a rapid increase in the volume of research, which particularly accessible through Google Scholar and Dimensions, and less through Scopus, the Web of Science, PubMed. A few COVID-19 …
Bibliometrics such as the number of papers and times cited are often used to compare researchers based on specific criteria. The criteria, however, are different in each research domain and are set by empirical laws. Moreover, there are arguments, such that the simple sum of metric values works to the advantage of elders. Therefore, this paper attempts to constitute features from time series data of bibliometrics, and then classify the researchers according to the features. In detail, time series patterns are extracted from bibliographic data sets, and then a model to classify whether the rese…
We quantify the long term impact that the coauthorship with established top-cited scientists has on the career of junior researchers in four different scientific disciplines. Through matched pair analysis, we find that junior researchers who coauthor work with top scientists enjoy a persistent competitive advantage throughout the rest of their careers with respect to peers with similar early career profiles. Such a competitive advantage materialises as a higher probability of repeatedly coauthoring work with top-cited scientists, and, ultimately, as a higher probability of becoming one. Notabl…
Empirical analysis results about the possible causes leading to non-citation may help increase the potential of researchers' work to be cited and editorial staffs of journals to identify contributions with potential high quality. In this study, we conduct a survey on the possible causes leading to citation or non-citation based on a questionnaire. We then perform a statistical analysis to identify the major causes leading to non-citation in combination with the analysis on the data collected through the survey. Most respondents to our questionnaire identified eight major causes that facilitate…
Historically, papers have been physically bound to the journal in which they were published but in the electronic age papers are available individually, no longer tied to their respective journals. Hence, papers now can be read and cited based on their own merits, independently of the journal's physical availability, reputation, or Impact Factor. We compare the strength of the relationship between journals' Impact Factors and the actual citations received by their respective papers from 1902 to 2009. Throughout most of the 20th century, papers' citation rates were increasingly linked to their …
Science is a cumulative activity, which can manifest itself through the act of citing. Citations are also central to research evaluation, thus creating incentives for researchers to cite their own work. Using a dataset containing more than 63 million articles and 51 million disambiguated authors, this paper examines the relative importance of self-citations and self-references in the scholarly communication landscape, their relationship with the age and gender of authors, as well as their effects on various research evaluation indicators. Results show that self-citations and self-references ev…
Citations are commonly held to represent scientific impact. To date, however, there is no empirical evidence in support of this postulate that is central to research assessment exercises and Science of Science studies. Here, we report on the first empirical verification of the degree to which citation numbers represent scientific impact as it is actually perceived by experts in their respective field. We run a large-scale survey of about 2000 corresponding authors who performed a pairwise impact assessment task across more than 20000 scientific articles. Results of the survey show that citatio…
Science has become more collaborative over the past years, a phenomenon that is related to the increase in the number of authors per paper and the emergence of interdisciplinary works featuring specialists of different fields. In such a environment, it is not trivial to quantify the individual impact of researchers. Here we analyze how the most prolific collaboration tie (in terms of co-produced works) of an established researcher influences their productivity and visibility metrics. In particular, we check how the number of produced works, citations and h-index rank of an researcher changes w…
Unraveling the Ai2 Asta Scholarly Research Assistant Citation System
The iMetrics Lab. Department of Audiovisual Communication, Documentation and History of Art, Universitat Politècnica de València（瓦伦西亚理工大学）；Departament de Biblioteconomia, Documentació i Comunicació Audiovisual, Universitat de Barcelona（巴塞罗那大学）
Researchers are often evaluated by citation-based metrics. Such metrics can inform hiring, promotion, and funding decisions. Concerns have been expressed that popular citation-based metrics incentivize researchers to maximize the production of publications. Such incentives may not be optimal for scientific progress. Here we present a citation-based measure that rewards both productivity and taste: the researcher's ability to focus on impactful contributions. The presented measure, CAP, balances the impact of publications and their quantity, thus incentivizing researchers to consider whether a …
It has recently been claimed by Baccini and coauthors that due to ANVUR's bibliometric evaluations of individuals, departments, and universities, in Italy there has been a surge in self-citations in the last ten years, thus increasing the "inwardness" of Italian research more than has happened abroad. We have studied the database of Ioannidis et al. published on 12 August 2019 of the one hundred thousand most "highly cited" scientists, including about two thousand Italians, and we found that the problem of self-citations in relation to this scientific elite is not significant in Italy, while p…
Referenced Publication Year Spectroscopy (RPYS) was recently introduced as a method to analyze the historical roots of research fields and groups or institutions. RPYS maps the distribution of the publication years of the cited references in a document set. In this study, we apply this methodology to the œuvre of an individual researcher on the occasion of a Festschrift for András Schubert's 70th birthday. We discuss the different options of RPYS in relation to one another (e.g. Multi-RPYS), and in relation to the longer-term research program of algorithmic historiography (e.g., HistCite) base…
The study of science at the individual micro-level frequently requires the disambiguation of author names. The creation of author's publication oeuvres involves matching the list of unique author names to names used in publication databases. Despite recent progress in the development of unique author identifiers, e.g., ORCID, VIVO, or DAI, author disambiguation remains a key problem when it comes to large-scale bibliometric analysis using data from multiple databases. This study introduces and validates a new methodology called seed+expand for semi-automatic bibliographic data collection for a…
As science advances, the academic community has published millions of research papers. Researchers devote time and effort to search relevant manuscripts when writing a paper or simply to keep up with current research. In this paper, we consider the problem of citation recommendation by extending a set of known-to-be-relevant references. Our analysis shows the degrees of cited papers in the subgraph induced by the citations of a paper, called projection graph, follow a power law distribution. Existing popular methods are only good at finding the long tail papers, the ones that are highly connec…
Revolutions in science: The proposal of an approach for the identification of most important researchers, institutions, and countries based on Reference Publication Year Spectroscopy (RPYS)
We test 16 bibliometric indicators with respect to their validity at the level of the individual researcher by estimating their power to predict later successful researchers. We compare the indicators of a sample of astrophysics researchers who later co-authored highly cited papers before their first landmark paper with the distributions of these indicators over a random control group of young authors in astronomy and astrophysics. We find that field and citation-window normalisation substantially improves the predicting power of citation indicators. The two indicators of total influence based…
Recently, two new indicators (Equalized Mean-based Normalized Proportion Cited, EMNPC; Mean-based Normalized Proportion Cited, MNPC) were proposed which are intended for sparse scientometrics data. The indicators compare the proportion of mentioned papers (e.g. on Facebook) of a unit (e.g., a researcher or institution) with the proportion of mentioned papers in the corresponding fields and publication years (the expected values). In this study, we propose a third indicator (Mantel-Haenszel quotient, MHq) belonging to the same indicator family. The MHq is based on the MH analysis - an establish…
We introduce a new centrality index for bipartite network of papers and authors that we call $K$-index. The $K$-index grows with the citation performance of the papers that cite a given researcher and can seen as a measure of scientific social recognition. Indeed, the $K$-index measures the number of hubs, defined in a self-consistent way in the bipartite network, that cites a given author. We show that the $K$-index can be computed by simple inspection of the Web of Science platform and presents several advantages over other centrality indexes, in particular Hirsch $h$-index. The $K$-index is…
POLAREZ ENGINEERING（POLAREZ工程公司）；TU Bergakademie Freiberg（弗赖贝格工业大学）；University of Bielefeld（比勒费尔德大学）
The number of scientific papers grows exponentially in many disciplines. The share of online available papers grows as well. At the same time, the period of time for a paper to loose at chance to be cited anymore shortens. The decay of the citing rate shows similarity to ultradiffusional processes as for other online contents in social networks. The distribution of papers per author shows similarity to the distribution of posts per user in social networks. The rate of uncited papers for online available papers grows while some papers 'go viral' in terms of being cited. Summarized, the practice…
The definition of the g-index is as arbitrary as that of the h-index, because the threshold number g^2 of citations to the g most cited papers can be modified by a prefactor at one's discretion, thus taking into account more or less of the highly cited publications within a dataset. In a case study I investigate the citation records of 26 physicists and show that the prefactor influences the ranking in terms of the generalized g-index less than for the generalized h-index. I propose specifically a prefactor of 2 for the g-index, because then the resulting values are of the same order of magnit…
Predicting the number of coauthors for researchers contributes to understanding the development of team science. However, it is an elusive task due to diversity in the collaboration patterns of researchers. This study provides a learning model for the dynamics of this variable; the parameters are learned from empirical data that consist of the number of publications and the number of coauthors at given time intervals. The model is based on relationship between the annual number of new coauthors and time given an annual number of publications, the relationship between the annual number of publi…
Many factors could affect the number of citations to a paper. Citations have an important role in research policy and in measuring the excellence of research and researchers. This work is the first study in software engineering (SE) to assess multiple factors affecting the number of citations to SE papers. We use (a) negative binomial regression and (b) quantile regression to study arithmetic mean and median expected citations of a paper. Our dataset includes all the 25,113 papers which have been published in a set of 16 main SE journals, between 1970 and 2018. Our results indicate that public…
Researchers or students entering a emerging research area are particularly interested in what newly published papers will be most cited and which young researchers will become influential in the future, so that they can catch the most recent advances and find valuable research directions. However, predicting the future importance of scientific articles and authors is challenging due to the dynamic nature of literature networks and evolving research topics. Different from most previous studies aiming to rank the current importance of literatures and authors, we focus on \emph{ranking the future…
In this paper we review the socalled altmetrics or alternative metrics. This concept raises from the development of new indicators based on Web 2.0, for the evaluation of the research and academic activity. The basic assumption is that variables such as mentions in blogs, number of twits or of researchers bookmarking a research paper for instance, may be legitimate indicators for measuring the use and impact of scientific publications. In this sense, these indicators are currently the focus of the bibliometric community and are being discussed and debated. We describe the main platforms and in…
