The lack of predictability of citation-based measures frequently used to gauge impact, from impact factors to short-term citations, raises a fundamental question: Is there long-term predictability in citation patterns? Here, we derive a mechanistic model for the citation dynamics of individual papers, allowing us to collapse the citation histories of papers from different journals and disciplines into a single curve, indicating that all papers tend to follow the same universal temporal pattern. The observed patterns not only help us uncover basic mechanisms that govern scientific impact but also offer reliable measures of influence that may have potential policy implications.
The paper comments on "Quantifying long-term scientific impact". It indicates that there is a mistake of [D. S. Wang , C. Song, A. L. Barabasi, Quantifying long-term scientific impact, Science 342, 127 (2013), arXiv:1306.3293].
Science is a social process with far-reaching impact on our modern society. In the recent years, for the first time we are able to scientifically study the science itself. This is enabled by massive amounts of data on scientific publications that is increasingly becoming available. The data is contained in several databases such as Web of Science or PubMed, maintained by various public and private entities. Unfortunately, these databases are not always consistent, which considerably hinders this study. Relying on the powerful framework of complex networks, we conduct a systematic analysis of t…
h-index has become the most popular indicator for quantifying a scientist's scientific impact in various scientific fields. h-index is defined as the largest number of papers with citation number larger than or equal to h and it treats each citation equally. However, different citations usually come from different papers with different influence and quality, and a citation from a highly influential paper is a greater recognition of the target paper than a citation from an ordinary paper. Based on this assumption, we proposed a new indicator named x-index to quantify a scientist's scientific im…
We examine the innovation of researchers with long-lived careers in Computer Science and Physics. Despite the epistemological differences between such disciplines, we consistently find that a researcher's most innovative publication occurs earlier than expected if innovation were distributed at random across the sequence of publications in their career, and is accompanied by a peak year in which researchers publish other work which is more innovative than average. Through a series of linear models, we show that the innovation achieved by a researcher during their peak year is higher when it is…
Citation impact is commonly assessed using direct, first-order citation relations. We consider here instead the indirect influence of publications on new publications via citations. We present a novel method to quantify the higher-order citation influence of publications, considering both direct, or first-order, and indirect, or higher-order citations. In particular, we are interested in higher-order citation influence at the level of disciplines. We apply this method to the whole Web of Science data at the level of disciplines. We find that a significant amount of influence -- 42% -- stems fr…
This paper provides a detailed description of the data collection and machine learning model used in our recent PNAS paper "Flat Teams Drive Scientific Innovation" Xu et al. [2022a]. Here, we discuss how the features of scientific publication can be used to estimate the implicit hierarchy in the corresponding author teams. Besides, we also describe the method of evaluating the impact of team hierarchy on scientific outputs. More details will be updated in this article continuously. Raw data and Readme document can be accessed in this GitHub repository Xu et al. [2022b].
Team chemistry is the holy grail of understanding collaborative human behavior, yet its quantitative understanding remains inconclusive. To reveal the presence and mechanisms of team chemistry in scientific collaboration, we reconstruct the publication histories of 560,689 individual scientists and 1,026,196 duos of scientists. We identify ability discrepancies between teams and their members, enabling us to evaluate team chemistry in a way that is robust against prior experience of collaboration and inherent randomness. Furthermore, our network analysis uncovers a nontrivial modular structure…
We analyze the reaction of academic communities to a particular urgent topic which abruptly arises as a scientific problem. To this end, we have chosen the disaster that occurred in 1986 in Chornobyl (Chernobyl), Ukraine, considered as one of the most devastating nuclear power plant accidents in history. The academic response is evaluated using scientific-publication data concerning the disaster using the Scopus database to present the picture on an international scale and the bibliographic database "Ukrainika naukova" to consider it on a national level. We measured distributions of papers in …
Citation count is a popular index for assessing scientific papers. However, it depends on not only the quality of a paper but also various factors, such as conventionality, team size, and gender. Here, we examine the extent to which the conventionality of a paper is related to its citation percentile in a discipline by using our measure, topic disparity. The topic disparity is the cosine distance between a paper and its discipline on a neural embedding space. Using this measure, we show that the topic disparity is negatively associated with the citation percentile in many disciplines, even aft…
Tracing the evolution of specific topics is a subject area which belongs to the general problem of mapping the structure of scientific knowledge. Often bibliometric data bases are used to study the history of scientific topic evolution from its appearance to its extinction or merger with other topics. In this chapter the authors present an analysis of the academic response to the disaster that occurred in 1986 in Chornobyl (Chernobyl), Ukraine, considered as one of the most devastating nuclear power plant accidents in history. Using a bibliographic database the distributions of Chornobyl-relat…
Our current societies increasingly rely on electronic repositories of collective knowledge. An archetype of these databases is the Web of Science (WoS) that stores scientific publications. In contrast to several other forms of knowledge -- e.g., Wikipedia articles -- a scientific paper does not change after its "birth". Nonetheless, from the moment a paper is published it exists within the evolving web of other papers, thus, its actual meaning to the reader changes. To track how scientific ideas (represented by groups of scientific papers) appear and evolve, we apply a novel combination of alg…
Citations are commonly held to represent scientific impact. To date, however, there is no empirical evidence in support of this postulate that is central to research assessment exercises and Science of Science studies. Here, we report on the first empirical verification of the degree to which citation numbers represent scientific impact as it is actually perceived by experts in their respective field. We run a large-scale survey of about 2000 corresponding authors who performed a pairwise impact assessment task across more than 20000 scientific articles. Results of the survey show that citatio…
Quantifying success in science plays a key role in guiding funding allocations, recruitment decisions, and rewards. Recently, a significant amount of progresses have been made towards quantifying success in science. This lack of detailed analysis and summary continues a practical issue. The literature reports the factors influencing scholarly impact and evaluation methods and indices aimed at overcoming this crucial weakness. We focus on categorizing and reviewing the current development on evaluation indices of scholarly impact, including paper impact, scholar impact, and journal impact. Besi…
There is an overall perception of increased interdisciplinarity in science, but this is difficult to confirm quantitatively owing to the lack of adequate methods to evaluate subjective phenomena. This is no different from the difficulties in establishing quantitative relationships in human and social sciences. In this paper we quantified the interdisciplinarity of scientific journals and science fields by using an entropy measurement based on the diversity of the subject categories of journals citing a specific journal. The methodology consisted in building citation networks using the Journal …
Classifying researchers according to the quality of their published work rather than the quantity is a curtail issue. We attempt to introduce a new formula of the percentage range to be used for evaluating qualitatively the researchers' production. The suggested equation depends on the number of the single-author published papers and their citations to be added as a new factor to the known h-index. These factors give an advantage and make a clear evidence of innovative authors and reduce the known h-index for authors who are gaining citations by adding their names to multi-author papers. It is…
In a critical and provocative paper, Abramo and D'Angelo claim that commonly used scientometric indicators such as the mean normalized citation score (MNCS) are completely inappropriate as indicators of scientific performance. Abramo and D'Angelo argue that scientific performance should be quantified using indicators that take into account the productivity of a research unit. We provide a response to Abramo and D'Angelo, indicating where we believe they raise important issues, but also pointing out where we believe their claims to be too extreme.
Inspired by the social and economic benefits of diversity, we analyze over 9 million papers and 6 million scientists to study the relationship between research impact and five classes of diversity: ethnicity, discipline, gender, affiliation, and academic age. Using randomized baseline models, we establish the presence of homophily in ethnicity, gender and affiliation. We then study the effect of diversity on scientific impact, as reflected in citations. Remarkably, of the classes considered, ethnic diversity had the strongest correlation with scientific impact. To further isolate the effects o…
Scientists are frequently faced with the important decision to start or terminate a creative partnership. This process can be influenced by strategic motivations, as early career researchers are pursuers, whereas senior researchers are typically attractors, of new collaborative opportunities. Focusing on the longitudinal aspects of scientific collaboration, we analyzed 473 collaboration profiles using an ego-centric perspective which accounts for researcher-specific characteristics and provides insight into a range of topics, from career achievement and sustainability to team dynamics and effi…
The number of citations is a widely used metric to evaluate the scientific credit of papers, scientists and journals. However, it does happen that a paper with fewer citations from prestigious scientists is of higher influence than papers with more citations. In this paper, we argue that from whom the paper is being cited is of higher significance than merely the number of received citations. Accordingly, we propose an interactive model on author-paper bipartite networks as well as an iterative algorithm to get better rankings for scientists and their publications. The main advantage of this m…
Despite a long history of use of citation count as a measure to assess the impact or influence of a scientific paper, the evolution of follow-up work inspired by the paper and their interactions through citation links have rarely been explored to quantify how the paper enriches the depth and breadth of a research field. We propose a novel data structure, called Influence Dispersion Tree (IDT) to model the organization of follow-up papers and their dependencies through citations. We also propose the notion of an ideal IDT for every paper and show that an ideal (highly influential) paper should …
The ongoing growth in the volume of scientific literature available today precludes researchers from efficiently discerning the relevant from irrelevant content. Researchers are constantly interested in impactful papers, authors and venues in their respective fields. Moreover, they are interested in the so-called recent "rising stars" of these contexts which may lead to attractive directions for future work, collaborations or impactful publication venues. In this work, we address the problem of quantifying research impact in each of these contexts, in order to better direct attention of resear…
Quantifying the impact of a scholarly paper is of great significance, yet the effect of geographical distance of cited papers has not been explored. In this paper, we examine 30,596 papers published in Physical Review C, and identify the relationship between citations and geographical distances between author affiliations. Subsequently, a relative citation weight is applied to assess the impact of a scholarly paper. A higher-order weighted quantum PageRank algorithm is also developed to address the behavior of multiple step citation flow. Capturing the citation dynamics with higher-order depen…
Newton's centuries-old wisdom of standing on the shoulders of giants raises a crucial yet underexplored question: Out of all the prior works cited by a discovery, which one is its giant? Here, we develop a novel, discipline-independent method to identify the giant for any individual paper, allowing us to systematically examine the role and characteristics of giants in science. We find that across disciplines, about 95% of papers stand on the shoulders of giants, yet the weight of scientific progress rests on relatively few shoulders. Defining a new measure of giant index, we find that, while p…
The issue of duplicate publications has received a lot of attention in the medical literature, but much less in the information science community. This paper aims at analyzing the prevalence and scientific impact of duplicate publications across all fields of research between 1980 and 2007, using a definition of duplicate papers based on their metadata. It shows that in all fields combined, the prevalence of duplicates is one out of two-thousand papers, but is higher in the natural and medical sciences than in the social sciences and humanities. A very high proportion (>85%) of these papers ar…
Traditionally, the number of citations that a scholarly paper receives from other papers is used as the proxy of its scientific impact. Yet citations can come from domains outside the scientific community, and one such example is through patented technologies---paper can be cited by patents, achieving technological impact. While the scientific impact of papers has been extensively studied, the technological aspect remains less known in the literature. Here we aim to fill this gap by presenting a comparative study on how 919 thousand biomedical papers are cited by U.S. patents and by other pape…
University of Electronic Science and Technology of China（电子科技大学）；Iowa State University（爱荷华州立大学）；University of Maryland（马里兰大学）
Quantifying and predicting the long-term impact of scientific writings or individual scholars has important implications for many policy decisions, such as funding proposal evaluation and identifying emerging research fields. In this work, we propose an approach based on Heterogeneous Dynamical Graph Neural Network (HDGNN) to explicitly model and predict the cumulative impact of papers and authors. HDGNN extends heterogeneous GNNs by incorporating temporally evolving characteristics and capturing both structural properties of attributed graph and the growing sequence of citation behavior. HDGN…
Nanjing University of Science and Technology（南京理工大学）；Indiana University（印第安纳大学）；Shandong Normal University（山东师范大学）；Nanjing University（南京大学）；Wuhan University（武汉大学）；Université de Montréal（蒙特利尔大学）
The number of publications and the number of citations received have become the most common indicators of scholarly success. In this context, scientific writing increasingly plays an important role in scholars' scientific careers. To understand the relationship between scientific writing and scientific impact, this paper selected 12 variables of linguistic complexity as a proxy for depicting scientific writing. We then analyzed these features from 36,400 full-text Biology articles and 1,797 full-text Psychology articles. These features were compared to the scientific impact of articles, groupe…
Comments are special types of publications whose aim is to correct or criticize previously published papers. For this reason, comments are believed to make commented papers less worthy or trusty to the eyes of the scientific community, and thus predestined to have low scientific impact. Here, we show that such belief is not supported by empirical evidence. We consider thirteen major publication outlets in science, and perform systematic comparisons between the citations accumulated by commented and non commented articles. We find that (i) commented papers are, on average, much more cited than …
We consider distributions of scientific journals impact factor. Analysing 9028 scientific journals with the largest impact factors, we found that the distribution of them is year-to-year stable (at least for analysed 2011-2013 years), and it has the character of the exponential Boltzmann distribution with the power law asymptotic (tail).
This paper analyzes the effect of interdisciplinarity on the scientific impact of individual papers. Using all the papers published in Web of Science in 2000, we define the degree of interdisciplinarity of a given paper as the percentage of its cited references made to journals of other disciplines. We show that, although for all disciplines combined there is no clear correlation between the level of interdisciplinarity of papers and their citation rates, there are nonetheless some disciplines in which a higher level of interdisciplinarity is related to a higher citation rates. For other disci…
Problems for evaluation and impact of published scientific works and their authors are discussed. The role of citations in this process is pointed out. Different bibliometric indicators are reviewed in this connection and ways for generation of new bibliometric indices are given. The influence of different circumstances, like self-citations, number of authors, time dependence and publication types, on the evaluation and impact of scientific papers are considered. The repercussion of works citations and their content is investigated in this respect. Attention is paid also on implicit citations …
The empirical and theoretical justification of Gartner hype curves is a very relevant open question in the field of Technological Life Cycle analysis. The scope of the present paper is to introduce a simple model describing the growth of scientific/technological research impact, in the specific case where science is the main source of a new idea driving a technological development, leading to hype-type evolution curves. The main idea of the model is that, in a first stage, the growth of the scientific interest of a new specific field (as can be measured by publication numbers) basically follow…
The impact factor (IF) of scientific journals has acquired a major role in the evaluations of the output of scholars, departments and whole institutions. Typically papers appearing in journals with large values of the IF receive a high weight in such evaluations. However, at the end of the day one is interested in assessing the impact of individuals, rather than papers. Here we introduce Author Impact Factor (AIF), which is the extension of the IF to authors. The AIF of an author A in year $t$ is the average number of citations given by papers published in year $t$ to papers published by A in …
A well established agenda on the research output, impact, and structure of global scientific elites such as Nobel Prize laureates has generated interest in the scientific elites from developing countries. This study deploys science mapping techniques to provide a comprehensive analysis of the output, impact, and structure of the Colombian scientific elite, i.e., researchers awarded with the Alejandro Angel Escobar Foundation National Prize 1990 2020, known locally as the Colombian Nobel. Findings showed that the Colombian scientific elite has a broader agenda than indexing titles in internatio…
Quantifying the impact of clinical-academic collaborations
UCL Queen Square Institute of Neurology, University College London（伦敦大学学院）；NIHR University College London Hospitals Biomedical Research Centre（英国国家卫生与临床优化研究所伦敦大学学院医院生物医学研究中心）；UCL Cancer Institute（伦敦大学学院癌症研究所）
Nowadays impact factor is the significant indicator for journal evaluation. In impact factor calculation is used number of all citations to journal, regardless of the prestige of cited journals, however, scientific units (paper, researcher, journal or scientific organization) cited by journals with high impact factor or researchers with high Hirsch index are more important than objects cited by journals without impact factor or unknown researcher. In this paper was offered weighted impact factor for getting more accurate rankings for journals, which consider not only quantity of citations, but…
The growth rate of the number of scientific publications is constantly increasing, creating important challenges in the identification of valuable research and in various scholarly data management applications, in general. In this context, measures which can effectively quantify the scientific impact could be invaluable. In this work, we present BIP! DB, an open dataset that contains a variety of impact measures calculated for a large collection of more than 100 million scientific publications from various disciplines.
The author of this letter to the editor would like to set forth the argument that scientometrics is currently in a phase in which a taxonomic change, and hence a revolution, is taking place. One of the key terms in scientometrics is scientific impact which nowadays is understood to mean not only the impact on science but the impact on every area of society.
The citation impact of a scientific publication is usually seen as a one-dimensional concept. We introduce a multi-dimensional framework for characterizing the citation impact of a publication. In addition to the level of citation impact, quantified by the number of citations received by a publication, we also conceptualize and operationalize the depth and breadth and the dependence and independence of the citation impact of a publication. The proposed framework distinguishes between publications that have a deep citation impact, typically in a relatively narrow research area, and publications…
The broad coverage of the search for the Higgs boson in the mainstream media is a relative novelty for high energy physics (HEP) research, whose achievements have traditionally been limited to scholarly literature. This paper illustrates the results of a scientometric analysis of HEP computing in scientific literature, institutional media and the press, and a comparative overview of similar metrics concerning representative particle physics measurements. The picture emerging from these scientometric data documents the scientific impact and social perception of HEP computing. The results of thi…
The impact of scientific publications has traditionally been expressed in terms of citation counts. However, scientific activity has moved online over the past decade. To better capture scientific impact in the digital era, a variety of new impact measures has been proposed on the basis of social network analysis and usage log data. Here we investigate how these new measures relate to each other, and how accurately and completely they express scientific impact. We performed a principal component analysis of the rankings produced by 39 existing and proposed measures of scholarly impact that wer…
A widely used measure of scientific impact is citations. However, due to their heavy-tailed distribution, citations are fundamentally difficult to predict. Instead, to characterize scientific impact, we address two analogous questions asked by many scientific researchers: "How will my h-index evolve over time, and which of my previously or newly published papers will contribute to it?" To answer these questions, we perform two related tasks. First, we develop a model to predict authors' future h-indices based on their current scientific impact. Second, we examine the factors that drive papers-…
The present study aims to analyze relationship between Citations Normalized Score (NCS) of scientific publications and Article Processing Charges (APCs) amounts of Gold Open access publications. To do so, we use APCs information provided by OpenAPC database and citations scores of publications in the Web of Science database (WoS). Database covers the period from 2006 to 2019 with 83,752 articles published in 4751 journals belonging to 267 distinct publishers. Results show that contrary to this belief, paying dearly does not necessarily increase the impact of publications. First, large publishe…
This paper provides the first historical analysis of the relationship between collaboration and scientific impact, using three indicators of collaboration (number of authors, number of addresses, and number of countries) and including articles published between 1900 and 2011. The results demonstrate that an increase in the number of authors leads to an increase in impact--from the beginning of the last century onwards--and that this is not simply due to self-citations. A similar trend is also observed for the number of addresses and number of countries represented in the byline of an article. …
