This paper presents a statistical analysis of the relationship between three science indicators applied in earlier bibliometric studies, namely research leadership based on corresponding authorship, international collaboration using international co-authorship data, and field-normalized citation impact. Indicators at the level of countries are extracted from the SIR database created by SCImago Research Group from publication records indexed for Elsevier's Scopus. The relationship between authorship and citation-based indicators is found to be complex, as it reflects a country's phase of scientific development and the coverage policy of the database. Moreover, one should distinguish a genuine leadership effect from a purely statistical effect due to fractional counting. Further analyses at the level of institutions and qualitative validation studies are recommended.
Studies of bibliographic data suggest a strong correlation between the growth of citation networks and their corresponding co-authorship networks. We explore the interdependence between evolving citation and co-authorship networks focused on the publications, by Indian authors, in American Physical Society journals between 1970 and 2013. We record interactions between each possible pair of authors in two ways: first, by tracing the change in citations they exchanged and, second, by tracing the shortest path between authors in the co-authorship network. We create these data for every year of th…
We collected and cleaned a large data set on publications in statistics. The data set consists of the coauthor relationships and citation relationships of 83, 331 papers published in 36 representative journals in statistics, probability, and machine learning, spanning 41 years. The data set allows us to construct many different networks, and motivates a number of research problems about the research patterns and trends, research impacts, and network topology of the statistics community. In this paper we focus on (i) using the citation relationships to estimate the research interests of authors…
Sberbank（俄罗斯联邦储蓄银行）；National University of Science and Technology “MISIS”（国立研究型技术大学莫斯科钢铁合金学院）
The problem of co-authors selection in the area of scientific collaborations might be a daunting one. In this paper, we propose a new pipeline that effectively utilizes citation data in the link prediction task on the co-authorship network. In particular, we explore the capabilities of a recommender system based on data aggregation strategies on different graphs. Since graph neural networks proved their efficiency on a wide range of tasks related to recommendation systems, we leverage them as a relevant method for the forecasting of potential collaborations in the scientific community.
Researchers affiliated with multiple institutions are increasingly seen in current scientific environment. In this paper we systematically analyze the multi-affiliated authorship and its effect on citation impact, with focus on the scientific output of research collaboration. By considering the nationality of each institutions, we further differentiate the national multi-affiliated authorship and international multi-affiliated authorship and reveal their different patterns across disciplines and countries. We observe a large share of publications with multi-affiliated authorship (45.6%) in res…
Single journal studies are reviewed and measures used in the studies are highlighted. The following quantitative measures are used to study 272 articles published in Malaysian Journal of Computer Science, (1) the article productivity of the journal from 1985 to 2007, (2) the observed and expected authorship productivity tested using Lotka's Law of author productivity, identification and listing of core authors; (3) the authorship, co-authorship pattern by authors' country of origin and institutional affiliations; (4) the subject areas of research; (5) the citation analysis of resources referen…
We propose ReviewerNet, an online, interactive visualization system aimed to improve the reviewer selection process in the academic domain. Given a paper submitted for publication, we assume that good candidate reviewers can be chosen among the authors of a small set of relevant and pertinent papers; ReviewerNet supports the construction of such set of papers, by visualizing and exploring a literature citation network. Then, the system helps to select reviewers that are both well distributed in the scientific community and that do not have any conflict-of-interest, by visualising the careers a…
Academic citation and social attention measure different dimensions of the impact of research results. Both measures do not correlate with each other, and they are influenced by many factors. Among these factors are the field of research, the type of access, and co-authorship. In this study, the increase in the impact due to co-authorship in scientific articles disaggregated by field of research and access type, was quantified. For this, the citations and social attention accumulated until the year 2021 by a total of 244,880 research articles published in the year 2018, were analyzed. The data…
This paper introduces a simple agglomerative clustering method to identify large publishing consortia with at least 20 authors and 80% shared authorship between articles. Based on Scopus journal articles 1996-2018, under these criteria, nearly all (88%) of the large consortia published research with citation impact above the world average, with the exceptions being mainly the newer consortia for which average citation counts are unreliable. On average, consortium research had almost double (1.95) the world average citation impact on the log scale used (Mean Normalised Log Citation Score). At l…
Disparities in authorship and citations across gender can have substantial adverse consequences not just on the disadvantaged genders, but also on the field of study as a whole. Measuring gender gaps is a crucial step towards addressing them. In this work, we examine female first author percentages and the citations to their papers in Natural Language Processing (1965 to 2019). We determine aggregate-level statistics using an existing manually curated author--gender list as well as first names strongly associated with a gender. We find that only about 29% of first authors are female and only a…
The paper analyses Russian Index of Science Citation (RISC), a national citation database. We continue our previous study (Moskaleva et al., 2018) and focus on difference between bibliometric indicators calculated on, so to say, "the best" journals, so called RISC Core, and those which take into account all Russian journals available. Such a difference may show focuses of insitutional actors on different document types, publication strategies etc.
Bi-National Academic Funding and Collaboration Dynamics: The Case of the German-Israeli Foundation
Bar-Ilan University（巴伊兰大学）；University of Haifa（海法大学）；Jönköping University（延雪平大学）；GESIS – Leibniz-Institute for the Social Sciences（GESIS – 莱布尼茨社会科学研究所）
This study examines the correlational relationships between local journal authorship, local and external citation counts, full-text downloads, link-resolver clicks, and four global journal impact factor indices within an all-disciplines journal collection of 12,200 titles and six subject subsets at the University of Illinois at Urbana-Champaign (UIUC) Library. While earlier investigations of the relationships between usage (downloads) and citation metrics have been inconclusive, this study shows strong correlations in the all-disciplines set and most subject subsets. The normalized Eigenfactor…
Exploring whether different patterns emerge across networks generated by organizational structures, co-authorships and citations for characterizing and evaluating cooperative relationships is particularly important for transferring the research results into practice. This research-in-progress paper focuses on using the structure of scientific collaborations and mapping knowledge transfer to gain insight into the influence of collaborative research centres linked to the German Research Foundation (DFG) funding. Within the German Centre for Integrative Biodiversity Research (iDiv), the DFG spons…
The prediction of the long-term impact of a scientific article is challenging task, addressed by the bibliometrician through resorting to a proxy whose reliability increases with the breadth of the citation window. In the national research assessment exercises using metrics the citation window is necessarily short, but in some cases is sufficient to advise the use of simple citations. For the Italian VQR 2011-2014, the choice was instead made to adopt a linear weighted combination of citations and journal metric percentiles, with weights differentiated by discipline and year. Given the strateg…
Governments sometimes need to analyse sets of research papers within a field in order to monitor progress, assess the effect of recent policy changes, or identify areas of excellence. They may compare the average citation impacts of the papers by dividing them by the world average for the field and year. Since citation data is highly skewed, however, simple averages may be too imprecise to robustly identify differences within, rather than across, fields. In response, this article introduces two new methods to identify national differences in average citation impact, one based on linear modelli…
National research impact indicators derived from citation counts are used by governments to help assess their national research performance and to identify the effect of funding or policy changes. Citation counts lag research by several years, however, and so their information is somewhat out of date. Some of this lag can be avoided by using readership counts from the social reference sharing site Mendeley because these accumulate more quickly than citations. This article introduces a method to calculate national research impact indicators from Mendeley, using citation counts from older time p…
Nations can be distinguished in terms of whether domestic or international research is cited. We analyzed the research output in natural sciences of three leading European research economies (Germany, the Netherlands, and the UK) and ask where their researchers look for the knowledge that underpins their most highly-cited papers. Is one internationally oriented or is citation limited to national resources? Do the citation patterns reflect a growing differentiation between the domestic and international research enterprise? To evaluate change over time, we include natural-sciences papers publis…
Research collaboration is promoted by governments and research funders but if the relative prevalence and merits of collaboration vary internationally different national and disciplinary strategies may be needed to promote it. This study compares the team size and field normalised citation impact of research across all 27 Scopus broad fields in the ten countries with the most journal articles indexed in Scopus 2008-2012. The results show that team size varies substantially by discipline and country, with Japan (4.2) having two thirds more authors per article than the UK (2.5). Solo authorship …
We analyze international co-authorship relations in the Social Science Citation Index 2011 using all citable items in the DVD-version of this index. Network statistics indicate four groups of nations: (i) an Asian-Pacific one to which all Anglo-Saxon nations (including the UK and Ireland) are attributed; (ii) a continental European one including also the Latin-American countries; (iii) the Scandinavian nations; and (iv) a community of African nations. Within the EU-28 (including Croatia), eleven of the EU-15 states have dominant positions. Collapsing the EU-28 into a single node leads to a bi-…
The need for open scientific knowledge graphs is ever increasing. While there are large repositories of open access articles and free publication indexes, there are still few free knowledge graphs exposing citation networks, and often their coverage is partial. Consequently, most evaluation processes based on citation counts rely on commercial citation databases. Things are changing thanks to the Initiative for Open Citations (I4OC, https://i4oc.org) and the Initiative for Open Abstracts (I4OA, https://i4oa.org), whose goal is to campaign for scholarly publishers to open the reference lists an…
Emerging and established topics in drone research: Citation impact and knowledge flows across China, the United States, the EU, Ukraine, and Russia (2020-2025)
Centre for R&D Monitoring (ECOOM), University of Antwerp（安特卫普大学研发监测中心（ECOOM））
What Large-Scale Publication and Citation Data Tell Us About International Research Collaboration in Europe: Changing National Patterns in Global Contexts
Devising an index to measure the quality of research is a challenging task. In this paper, we propose a set of indices to evaluate the quality of research produced by an author. Our indices utilize a policy that assigns the weights to multiple authors of a paper. We have considered two weight assignment policies: positionally weighted and equally weighted. We propose two classes of weighted indices: weighted h-indices and weighted citation h-cuts. Further, we compare our weighted h-indices with the original h-index for a selected set of authors. As opposed to h-index, our weighted h-indices ta…
The importance of collaboration in research is widely accepted, as is the fact that articles with more authors tend to be more cited. Nevertheless, although previous studies have investigated whether the apparent advantage of collaboration varies by country, discipline, and number of co-authors, this study introduces a more fine-grained method to identify differences: the geometric Mean Normalized Citation Score (gMNCS). Based on comparisons between disciplines, years and countries for two million journal articles, the average citation impact of articles increases with the number of authors, e…
Over the past decade, national research evaluation exercises, traditionally conducted using the peer review method, have begun opening to bibliometric indicators. The citations received by a publication are assumed as proxy for its quality, but they require standardization prior to use in comparative evaluation of organizations or individual scientists: the citation data must be standardized, due to the varying citation behavior across research fields. The objective of this paper is to compare the effectiveness of the different methods of normalizing citations, in order to provide useful indic…
Authorship of scientific articles has profoundly changed from early science until now. If once upon a time a paper was authored by a handful of authors, scientific collaborations are much more prominent on average nowadays. As authorship (and citation) is essentially the primary reward mechanism according to the traditional research evaluation frameworks, it turned to be a rather hot-button topic from which a significant portion of academic disputes stems. However, the novel Open Science practices could be an opportunity to disrupt such dynamics and diversify the credit of the different scient…
Understanding how a scientist develops new scientific collaborations or how their papers receive new citations is a major challenge in scientometrics. The approach being proposed simultaneously examines the growth processes of the co-authorship and citation networks by analyzing the evolutions of the rich get richer and the fit get richer phenomena. In particular, the preferential attachment function and author fitnesses, which govern the two phenomena, are estimated non-parametrically in each network. The approach is applied to the co-authorship and citation networks of the flagship journal o…
In the past, several works have investigated ways for combining quantitative and qualitative methods in research assessment exercises. Indeed, the Italian National Scientific Qualification (NSQ), i.e. the national assessment exercise which aims at deciding whether a scholar can apply to professorial academic positions as Associate Professor and Full Professor, adopts a quantitative and qualitative evaluation process: it makes use of bibliometrics followed by a peer-review process of candidates' CVs. The NSQ divides academic disciplines into two categories, i.e. citation-based disciplines (CDs)…
This paper presents a citation analysis of Internext-Review of International Business to detect the impact caused by papers published for the period 2006-2013. The Publish or Perish (PoP) software is used, which retrieves articles and citations from Google Scholar database. As part of the applied indicators are: the distribution of authors by articles, citations per year, citation vs. self-citation, journal's citable vs. non citable documents, journal's cited vs. uncited documents, co-word analysis, and H Index. A total of 131 articles were obtained for 153 citations made until June, 2014. Mos…
Most Performance-based Research Funding Systems (PRFS) draw on peer review and bibliometric indicators, two different methodologies which are sometimes combined. A common argument against the use of indicators in such research evaluation exercises is their low correlation at the article level with peer review judgments. In this study, we analyse 191,000 papers from 154 higher education institutes which were peer reviewed in a national research evaluation exercise. We combine these data with 6.95 million citations to the original papers. We show that when citation-based indicators are applied a…
Using the InCites tool of Thomson Reuters, this study compares normalized citation impact values calculated for China, Japan, France, Germany, United States, and the UK throughout the time period from 1981 to 2010. The citation impact values are normalized to four subject areas: natural sciences; engineering and technology; medical and health sciences; and agricultural sciences. The results show an increasing trend in citation impact values for France, the UK and especially for Germany across the last thirty years in all subject areas. The citation impact of papers from China is still at a rel…
The field of digital libraries (DLs) coalesced in 1994: the first digital library conferences were held that year, awareness of the World Wide Web was accelerating, and the National Science Foundation awarded $24 Million (U.S.) for the Digital Library Initiative (DLI). In this paper we examine the state of the DL domain after a decade of activity by applying social network analysis to the co-authorship network of the past ACM, IEEE, and joint ACM/IEEE digital library conferences. We base our analysis on a common binary undirectional network model to represent the co-authorship network, and fro…
Collaborations are an integral part of scientific research and publishing. In the past, access to large-scale corpora has limited the ways in which questions about collaborations could be investigated. However, with improvements in data/metadata quality and access, it is possible to explore the idea of research collaboration in ways beyond the traditional definition of multiple authorship. In this paper, we examine scientific works through three different lenses of collaboration: across multiple authors, multiple institutions, and multiple departments. We believe this to be a first look at mul…
Authorship of scientific articles has profoundly changed from early science until now. While once upon a time a paper was authored by a handful of authors, scientific collaborations are much more prominent on average nowadays. As authorship (and citation) is essentially the primary reward mechanism according to the traditional research evaluation frameworks, it turned out to be a rather hot-button topic from which a significant portion of academic disputes stems. However, the novel Open Science practices could be an opportunity to disrupt such dynamics and diversify the credit of the different…
It is several years since national research evaluation systems around the globe started making use of quantitative indicators to measure the performance of researchers. Nevertheless, the effects on these systems on the behavior of the evaluated researchers are still largely unknown. We attempt to shed light on this topic by investigating how Italian researchers reacted to the introduction in 2011 of national regulations in which key passages of professional careers are governed by bibliometric indicators. A new inwardness measure, able to gauge the degree of scientific self-referentiality of a…
We investigate the problem of counting co-authorhip in order to quantify the impact and relevance of scientific research output through normalized \textit{h-index} and \textit{g-index}. We use the papers whose authors belong to a subset of full professors of the Italian Settore Scientifico Disciplinare (SSD) FIS01 - Experimental Physics. In this SSD two populations, characterized by the number of co-authors of each paper, are roughly present. The total number of citations for each individuals, as well as their h-index and g-index, strongly depends on the average number of co-authors. We show t…
Using co-authored publications between China and Korea in Web of Science (WoS) during the one-year period of 2014, we evaluate the government stimulation program for collaboration between China and Korea. In particular, we apply dual approaches, full integer vs. fractional counting, to collaborative publications in order to better examine both the patterns and contents of Sino-Korean collaboration networks in terms of individual countries and institutions. We first conduct a semi-automatic network analysis of Sino-Korean publications based on the full-integer counting method, and then compare …
With the passage of more time from the original date of publication, the measure of the impact of scientific works using subsequent citation counts becomes more accurate. However the measurement of individual and organizational research productivity should ideally refer to a period with closing date just prior to the evaluation exercise. Therefore it is necessary to compromise between accuracy and timeliness. This work attempts to provide an order of magnitude for the error in measurement that occurs with decreasing the time lapse between date of publication and citation count. The analysis is…
We study the national production of academic knowledge in all Iberoamerican countries (IAC) between 1973 and 2007. We show that the total number of mainstream scientific publications listed in SCI,SSCI and A&HCI follows an exponential growth, the same as the national productivity expressed in the number of publications per capita. We also explore the temporal evolution of the co-authorship patterns between a sample of 12 IAC responsible for 98% of the total regional publications, with a group of other 45 nations. We show that the scientific co-authorship among countries follows a power-law and…
Large, multi-institutional groups or collaborations of scientists are engaged in nuclear physics research projects, and the number of research facilities is dwindling. These collaborations have their own authorship rules, and they produce a large number of highly-cited papers. Multiple authorship of nuclear physics publications creates a problem with the assessment of an individual author's productivity relative to his/her colleagues and renders ineffective a performance metrics solely based on annual publication and citation counts. Many institutions are increasingly relying on the total numb…
In some research evaluation systems, credit awarded to an article depends on the number of co-authors on the article with total credit to the article increasing with the number of co-authors. There are many examples of such evaluation systems (e.g., the United States National Research Council evaluation of graduate programs gave full credit to each co-author). Such credit systems run the risk of encouraging ghost or honorary authorships. In a recent article, Antonio Osorio and Lutz Bornmann (2019) propose a scheme to discourage ghost authorships but increase the total credit to a paper when co…
We address the question how citation-based bibliometric indicators can best be normalized to ensure fair comparisons between publications from different scientific fields and different years. In a systematic large-scale empirical analysis, we compare a traditional normalization approach based on a field classification system with three source normalization approaches. We pay special attention to the selection of the publications included in the analysis. Publications in national scientific journals, popular scientific magazines, and trade magazines are not included. Unlike earlier studies, we …
Two methods for comparing impact factors and citation rates across fields of science are tested against each other using citations to the 3,705 journals in the Science Citation Index 2010 (CD-Rom version of SCI) and the 13 field categories used for the Science and Engineering Indicators of the US National Science Board. We compare (i) normalization by counting citations in proportion to the length of the reference list (1/N of references) with (ii) rescaling by dividing citation scores by the arithmetic mean of the citation rate of the cluster. Rescaling is analytical and therefore independent…
National bibliometric performance is commonly measured via relative impact indicators which appraise absolute national values through a global environment. Consequenty the resulting impact values mirror changes in the national performance as well as in its embedding. In order to assess the importance of the environment in this ratio, we analyse the increase in Chinese publications as an example for a structural change altering the whole database. Via a counterfactual comparison we quantify how Chinese publications benefit a large set of countries on their impact values, identify explanatory fa…
Policy makers are interested in the influence of geographic distance on knowledge flows, however these can be expected to vary across research fields. The effects of geographic distance on flows are analyzed by means of citations to scientific literature. The field of observation consists of the 2010-2012 Italian publications and relevant citations up to the close of 2017. The geographic proximity effect is analyzed at national, continental, and intercontinental level in 244 fields, and results as evident at national level and in some cases at continental level, but not at intercontinental lev…
The analysis of bibliometric networks, such as co-authorship, bibliographic coupling, and co-citation networks, has received a considerable amount of attention. Much less attention has been paid to the construction of these networks. We point out that different approaches can be taken to construct a bibliometric network. Normally the full counting approach is used, but we propose an alternative fractional counting approach. The basic idea of the fractional counting approach is that each action, such as co-authoring or citing a publication, should have equal weight, regardless of for instance t…
Government funding agencies and foundations tend to perceive novelty as necessary for scientific impact and hence prefer to fund novel instead of incremental projects. Evidence linking novelty and the eventual impact of a grant is surprisingly scarce, however. Here, we examine this link by analyzing 920,000 publications funded by 170,000 grants from the National Science Foundation (NSF) and the National Institutes of Health (NIH) between 2008 and 2016. We use machine learning to quantify grant novelty at the time of funding and relate that measure to the citation dynamics of these publications…
Nationally oriented scientific-scholarly journals are considered from a methodological-informetric viewpoint, analysing data extracted from Scimago Journal Rank based on Scopus. An operational definition is proposed of a journal's degree of national orientation based on the geographical distribution of its publishing or citing authors, and the role of international collaboration and a country's total publication output. A comprehensive analysis is presented of trends up until 2019 in national orientation and citation impact of national journals entering Scopus, extending outcomes in earlier st…
