Percentiles are statistics pointing to the standing of a paper's citation impact relative to other papers in a given citation distribution. Percentile Ranks (PRs) often play an important role in evaluating the impact of scholars, institutions, and lines of study. Because PRs are so important for the assessment of scholarly impact, and because citation practices differ greatly across time and fields, various percentile approaches have been proposed to time- and field-normalize citations. Unfortunately, current popular methods often face significant problems in time- and field-normalization, including when papers are assigned to multiple fields or have been published by more than one unit (e.g., researchers or countries). They also face problems for estimating citation counts (CCs) for pre-defined PRs (e.g., the 90th PR). We offer a series of guidelines and procedures that, we argue, address these problems and others and provide a superior means to make the use of percentile methods more accurate and informative. In particular, we introduce two approaches, CP-IN and CP-EX, that should be preferred in bibliometric studies because they consider the complete citation distribution. Both approaches are based on cumulative frequencies in percentages (CPs). The paper further shows how bar graphs and beamplots can present PRs in a more meaningful and accurate manner.
The substantive and practical significance of citation impact differences between institutions: Guidelines for the analysis of percentiles using effect sizes and confidence intervals
Using empirical data I demonstrate that the result of performance evaluations by percentiles can be drastically influenced by the proper choice of the journal in which a manuscript is published.
Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P100)
When comparing the citation impact of nations, departments or other groups of researchers within individual fields, three approaches have been proposed: arithmetic means, geometric means, and percentage in the top X%. This article compares the precision of these statistics using 97 trillion experimentally simulated citation counts from 6875 sets of different parameters (although all having the same scale parameter) based upon the discretised lognormal distribution with limits from 1000 repetitions for each parameter set. The results show that the geometric mean is the most precise, closely fol…
The percentile-based rating scale P100 describes the citation impact in terms of the distribution of unique citation values. This approach has recently been refined by considering also the frequency of papers with the same citation counts. Here I compare the resulting P100' with P100 for an empirical dataset and a simple fictitious model dataset. It is shown that P100' is not much different from standard percentile-based ratings in terms of citation frequencies. A new indicator P100'' is introduced.
How to improve the prediction based on citation impact percentiles for years shortly after the publication date?
Bornmann、Leydesdorff和Wang的研究结果表明，考虑期刊影响力可以提高对长期引文影响力的预测。本文进一步探讨了在短引文窗口的基础上，通过考虑期刊影响力和其他变量，如作者数、被引文献数和页数，改进引文影响力测量的可能性。该数据集包含1980年发表并在Web of Science（WoS，Thomson Reuters）中索引的475，391篇期刊论文，以及这些论文的所有年度引用计数（从1980年到2010年）。作为引文影响力的指标，我们使用了使用Hazen（1914）方法计算的引文数量。我们的研究结果表明，引文影响测量确实可以改进：如果在统计分析中考虑影响引文影响的一般因素，长期引文影响的解释方差可以大大增加。然而，这种增长只有在使用出版后不久的年份时才能看到，而在使用以后的年份时则看不到。
How much do different ways of calculating percentiles influence the derived performance indicators? - A case study
文献计量指标可以通过比较特定的引文记录与参考文献集的数量来确定。然而，在计算引文数量时存在着一种模糊性，因为通常在百分位数等级之间的边界处会发现大量具有相同引用计数的论文。本案例研究的引用期刊欧洲物理学快报（EPL）在比较所有的物理论文从Web of Science显示的偏差，由于不同的方式来处理并列论文的高被引出版物的百分比的评估。如果被引用次数达到阈值的论文都被认为是高引用或都被认为不是高引用，则可能会出现强烈的偏倚。
Using percentile shares, one can visualize and analyze the skewness in bibliometric data across disciplines and over time. The resulting figures can be intuitively interpreted and are more suitable for detailed analysis of the effects of independent and control variables on distributions than regression analysis. We show this by using percentile shares to analyze so-called "factors influencing citation impact" (FICs; e.g., the impact factor of the publishing journal) across year and disciplines. All articles (n= 2,961,789) covered by WoS in 1990 (n= 637,301), 2000 (n= 919,485), and 2010 (n= 1,…
How to analyse percentile impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes and top-cited papers
In this paper, we analyse the current availability of open citations data in one particular dataset, namely COCI (the OpenCitations Index of Crossref open DOI-to-DOI citations; http://opencitations.net/index/coci) provided by OpenCitations. The results of these analyses show a persistent gap in the coverage of the currently available open citation data. In order to address this specific issue, we propose a strategy whereby the community (e.g. scholars and publishers) can directly involve themselves in crowdsourcing open citations, by uploading their citation data via the OpenCitations infrastr…
We have studied the efficiency of research in the EU by a percentile-based citation approach that analyzes the distribution of country papers among the world papers. Going up in the citation scale, the frequency of papers from efficient countries increases while the frequency from inefficient countries decreases. In the percentile-based approach, this trend, which is permanent at any citation level, is measured by the ep index that equals the Ptop 1%/Ptop 10% ratio. By using the ep index we demonstrate that EU research on fast-evolving technological topics is less efficient than the world aver…
The two most used citation impact indicators in the assessment of scientific journals are, nowadays, the impact factor and the h-index. However, both indicators are not field normalized (vary heavily depending on the scientific category) which makes them incomparable between categories. Furthermore, the impact factor is not robust to the presence of articles with a large number of citations, while the h-index depends on the journal size. These limitations are very important when comparing journals of different sizes and categories. An alternative citation impact indicator is the percentage of …
Recently, the integrated impact indicator (I3) indicator was introduced where citations are weighted in accordance with the percentile rank class of each publication in a set of publications. I3 can also be used as a field-normalized indicator. Field-normalization is common practice in bibliometrics, especially when institutions and countries are compared. Publication and citation practices are so different among fields that citation impact is normalized for cross-field comparisons. In this study, we test the ability of the indicator to discriminate between quality levels of papers as defined …
Remaining problems with the "New Crown Indicator" (MNCS) of the CWTS
Waltman等人（2010年; arXiv：1003.2167）在其题为“Towards a new crown indicator：some theoretical considerations”的文章中指出，莱顿CWTS的“旧冠指标”在数学上不一致，应该采用“新冠指标”中应用的标准化。“尽管我们现在同意统计标准化，但“新冠指标”继承了“旧”指标的科学计量学问题，即把期刊的学科类别作为标准，对科学领域之间的引用行为差异进行标准化。我们进一步注意到，“平均值”不是测量偏态分布之间差异的适当统计量。在不改变“MNS”的缩写的情况下，可以定义“中位数标准化引文评分”。“这将使新冠指标与百分位数方法直接相关，例如，美国国家科学委员会的科学和工程指标（2010年）中使用的百分位数方法。根据定义，中位数等于第50百分位数。因此，该指标可以很容易地扩展为1%（=第99百分位）最高引用论文（Bornmann等人，印刷中）。必须使用非参数统计的表面上的缺点被可能的精度增益所补偿。
In this paper, a new field-normalized indicator is introduced, which is rooted in early insights in bibliometrics, and is compared with several established field-normalized indicators (e.g. the mean normalized citation score, MNCS, and indicators based on percentile approaches). Garfield (1979) emphasizes that bare citation counts from different fields cannot be compared for evaluative purposes, because the "citation potential" can vary significantly between the fields. Garfield (1979) suggests that "the most accurate measure of citation potential is the average number of references per paper …
Bibliometricians have long recurred to citation counts to measure the impact of publications on the advancement of science. However, since the earliest days of the field, some scholars have questioned whether all citations should be worth the same, and have gone on to weight them by a variety of factors. However sophisticated the operationalization of the measures, the methodologies used in weighting citations still present limits in their underlying assumptions. This work takes an alternative approach to resolving the underlying problem: the proposal is to value citations by the impact of the…
We combine the Integrated Impact Indicator (I3) and the h-index into the I3-type framework and introduce the publication vector X = (X1, X2, X3) and the citation vector Y = (Y1, Y2, Y3) , the publication score I3X=X1+X2+X3 and the citation score I3Y=Y1+Y2+Y3, and alternative indicators based on percentile classes generated by the h-index. These multivariate indicators can be used for academic evaluation. The empirical studies show that the h-core distribution is suitable to evaluate scholars, the X1 and Y1 are applied to measure core impact power of universities, and I3X and I3Y are alternativ…
How Proposal Novelty, Topical Diversity, and Theory-Practice Balance Shape Scholarly Outcomes in Funded Education Research
School of Economics and Management, East China Normal University（华东师范大学经济与管理学院）；Department of Education, Amity Global Institute（Amity全球学院教育学系）；School of Mathematics and Statistics, The University of Glasgow（格拉斯哥大学数学与统计学院）；Business School, The University of Edinburgh（爱丁堡大学商学院）
There are different citation habits in the research fields that influence the obsolescence of the research literature. We analyze the distinctive obsolescence of research literature in disciplinary journals in eight scientific subfields based on cited references distribution, as a synchronous approach. We use both Negative Binomial (NB) and Poisson distributions to capture this obsolescence. The corpus being examined is published in 2019 and covers 22,559 papers citing 872,442 references. Moreover, three measures to analyze the tail of the distribution are proposed: (i) cited reference surviva…
Although altmetrics and other web-based alternative indicators are now commonplace in publishers' websites, they can be difficult for research evaluators to use because of the time or expense of the data, the need to benchmark in order to assess their values, the high proportion of zeros in some alternative indicators, and the time taken to calculate multiple complex indicators. These problems are addressed here by (a) a field normalisation formula, the Mean Normalised Log-transformed Citation Score (MNLCS) that allows simple confidence limits to be calculated and is similar to a proposal of L…
The top 1 percent most highly cited articles are watched closely as the vanguards of the sciences. Using Web of Science data, one can find that China had overtaken the USA in the relative participation in the top 1 percent in 2019, after outcompeting the EU on this indicator in 2015. However, this finding contrasts with repeated reports of Western agencies that the quality of Chinese output in science is lagging other advanced nations, even as it has caught up in numbers of articles. The difference between the results presented here and the previous results depends mainly upon field normalizat…
Mapping Science across countries is a challenging task in the field of Scientometrics. A number of efforts trying to cope with this task has been discussed in the state of the art, addressing this challenge by processing collections of scientific digital libraries and visualizing author-based measures (for instance, the h-index) or document-based measures (for instance, the averaged number of citations per document). A major drawback of these approaches is related to the presence of bias. The bigger the country, the higher the measure value. We explore the use of an econometric index to tackle…
University of Electronic Science and Technology of China（电子科技大学）；Iowa State University（爱荷华州立大学）；University of Maryland（马里兰大学）
Quantifying and predicting the long-term impact of scientific writings or individual scholars has important implications for many policy decisions, such as funding proposal evaluation and identifying emerging research fields. In this work, we propose an approach based on Heterogeneous Dynamical Graph Neural Network (HDGNN) to explicitly model and predict the cumulative impact of papers and authors. HDGNN extends heterogeneous GNNs by incorporating temporally evolving characteristics and capturing both structural properties of attributed graph and the growing sequence of citation behavior. HDGN…
