Data Mining in Scientometrics: usage analysis for academic publications
Olesya Mryglod, Yurij Holovatch, Ralph Kenna
备注：
The paper will be presented during DSMP'2018 conference
发表信息：
IEEE Second International Conference on Data Stream Mining & Processing (Dsmp2018) IEEE Catalog Number: CFP18J13-POD, ISBN: 978-1-5386-2875-1 (2018) pp. 241-246
We perform a statistical analysis of scientific-publication data with a goal to provide quantitative analysis of scientific process. Such an investigation belongs to the newly established field of scientometrics: a branch of the general science of science that covers all quantitative methods to analyze science and research process. As a case study we consider download and citation statistics of the journal `Europhysics Letters' (EPL), as Europe's flagship letters journal of broad interest to the physics community. While citations are usually considered as an indicator of academic impact, downloads reflect rather the level of attractiveness or popularity of a publication. We discuss peculiarities of both processes and correlations between them.
In tandem with the rapid globalisation of science, spatial scientometrics has become an important research sub-field in scientometric studies. Recently, numerous spatial scientometric contributions have focused on the examination of cities' scientific output by using various scientometric indicators. In this paper, I analyse cities' scientific output worldwide in terms of the number of journal articles indexed by the Scopus database, in the period from 1986 to 2015. Furthermore, I examine which countries are the most important collaborators of cities. Finally, I identify the most productive di…
Scientometric analysis of 146 and 59 research articles published in Indian journal of Information Sources and Services (IJISS) and Pakistan Journal of Library and Information Science has been carried out. Seven Volumes of the IJISS containing 14 issues and Seven volumes of PJLIS containing 8 issues from 2011 - 2017 have been taken into consideration for the present study. The number of contributions, authorship pattern & author productivity, average citations, average length of articles, average keywords and collaborative papers has been analyzed. Out of 146 of IJISS contributions, only 39 are…
Scientometrics is the study of the quantitative aspects of the process of science as a communication system. It is centrally, but not only, concerned with the analysis of citations in the academic literature. In recent years it has come to play a major role in the measurement and evaluation of research performance. In this review we consider: the historical development of scientometrics, sources of citation data, citation metrics and the "laws" of scientometrics, normalisation, journal impact factors and other journal metrics, visualising and mapping science, evaluation and policy, and future …
Assessing the impact of scholarly books is a difficult research evaluation problem. Library Catalog Analysis facilitates the quantitative study, at different levels, of the impact and diffusion of academic books based on data about their availability in libraries. The WorldCat global catalog collates data on library holdings, offering a range of tools including the novel WorldCat Identities. This is based on author profiles and provides indicators relating to the availability of their books in library catalogs. Here, we investigate this new tool to identify its strengths and weaknesses based o…
In recent months the COVID-19 (also known as SARS-CoV-2 and Coronavirus) pandemic has spread throughout the world. In parallel, extensive scholarly research regarding various aspects of the pandemic has been published. In this work, we analyse the changes in biomedical publishing patterns due to the pandemic. We study the changes in the volume of publications in both peer reviewed journals and preprint servers, average time to acceptance of papers submitted to biomedical journals, international (co-)authorship of these papers (expressed by diversity and volume), and the possible association be…
This is the second part of a small-scale explorative study in an effort to assess reproducibility issues specific to scientometrics research. This effort is motivated by the desire to generate empirical data to inform debates about reproducibility in scientometrics. Rather than attempt to reproduce studies, we explore how we might assess "in principle" reproducibility based on a critical review of the content of published papers. While the first part of the study (Waltman et al. 2018) focuses on direct reproducibility - that is the ability to reproduce the specific evidence produced by an orig…
This is the first part of a small-scale explorative study in an effort to start assessing reproducibility issues specific to scientometrics research. This effort is motivated by the desire to generate empirical data to inform debates about reproducibility in scientometrics. Rather than attempt to reproduce studies, we explore how we might assess "in principle" reproducibility based on a critical review of the content of published papers. The first part of the study focuses on direct reproducibility - that is the ability to reproduce the specific evidence produced by an original study using the…
We have developed a (freeware) routine for "referenced publication years spectroscopy" (RPYS) and apply this method to the historiography of "iMetrics," that is, the junction of the journals Scientometrics, Informetrics, and the relevant subset of JASIST (approx. 20%) that shapes the intellectual space for the development of information metrics (bibliometrics, scientometrics, informetrics, and webometrics). The application to information metrics (our own field of research) provides us with the opportunity to validate this methodology, and to add a reflection about using citations for the histo…
Bibliographic records related to tribology research were extracted from SCOPUS and Web of Science databases for the period of 15 years from 1998 to 2012. Macro-level scientometric indicators such as growth rate, share of international collaborative papers, citations per paper, and share of non-cited papers were employed. Further, the Gini coefficient and Simpson Index of Diversity were used. Two new relative indicators : Relative International Collaboration Rate (RICR) and Relative Growth Index (RGI) are proposed in this study. The performance of top countries contributing more than 1000 paper…
This paper examines the use of scientometric overlay mapping as a tool of 'strategic intelligence' to aid the governance of emerging technologies. We develop an integrative synthesis of different overlay mapping techniques and associated perspectives on technological emergence across the geographical, social, and cognitive spaces. To do so, we longitudinally analyse (with publication and patent data) three case-studies of emerging technologies in the medical domain. These are: RNA interference (RNAi), Human Papilloma Virus (HPV) testing technologies for cervical cancer, and Thiopurine Methyltr…
When a research infrastructure is funded and implemented, new information and new publications are created. This new information is the measurable output of discovery process. In this paper, we describe the impact of infrastructure for physics experiments in terms of publications and citations. In particular, we consider the Large Hadron Collider (LHC) experiments (ATLAS, CMS, ALICE, LHCb) and compare them to the Large Electron Positron Collider (LEP) experiments (ALEPH, DELPHI, L3, OPAL) and the Tevatron experiments (CDF, D0). We provide an overview of the scientific output of these projects …
In this work we propose a metric to assess academic productivity based on publication outputs. We are interested in knowing how well a research group in an area of knowledge is doing relatively to a pre-selected set of reference groups, where each group is composed by academics or researchers. To assess academic productivity we propose a new metric, which we call P-score. Our metric P-score assigns weights to venues using only the publication patterns of selected reference groups. This implies that P-score does not depend on citation data and thus, that it is simpler to compute particularly in…
In course of the organization of Workshop III entitled "Cited References Analysis Using CRExplorer" at the International Conference of the International Society for Scientometrics and Informetrics (ISSI2021), we have prepared three reference publication year spectroscopy (RPYS) analyses: (i) papers published in Journal of Informetrics; (ii) papers regarding the topic altmetrics; and (iii) papers published by Ludo Waltman (we selected this researcher since he received the Derek de Solla Price Memorial Medal during the ISSI2021 conference). The first RPYS analysis has been presented live at the …
Several questions of scientometrics parameters organization are considered. Two new indices for scientific works citation analysis are proposed. They provide more detailed and reliable scientific significance assessment of individual authors and scientific groups basing on the publication activity
Literature-based discovery (LBD) aims to discover valuable latent relationships between disparate sets of literatures. This paper presents the first inclusive scientometric overview of LBD research. We utilize a comprehensive scientometric approach incorporating CiteSpace to systematically analyze the literature on LBD from the last four decades (1986-2020). After manual cleaning, we have retrieved a total of 409 documents from six bibliographic databases and two preprint servers. The 35 years' history of LBD could be partitioned into three phases according to the published papers per year: in…
Rather than "measuring" a scientist impact through the number of citations which his/her published work can have generated, isn't it more appropriate to consider his/her value through his/her scientific network performance illustrated by his/her co-author role, thus focussing on his/her joint publications, - and their impact through citations? Whence, on one hand, this paper very briefly examines bibliometric laws, like the $h$-index and subsequent debate about co-authorship effects, but on the other hand, proposes a measure of collaborative work through a new index. Based on data about the pu…
A theory of citations should not consider cited and/or citing agents as its sole subject of study. One is able to study also the dynamics in the networks of communications. While communicating agents (e.g., authors, laboratories, journals) can be made comparable in terms of their publication and citation counts, one would expect the communication networks not to be homogeneous. The latent structures of the network indicate different codifications that span a space of possible 'translations'. The various subdynamics can be hypothesized from an evolutionary perspective. Using the network of aggr…
Although there is a perception in the community that the state of optics research in Colombia has reached a mature state with international recognition, to date, there is no study that supports this view quantitatively. We aimed to assess the state of optics research in Colombia based on international journal publications. Therefore, we determined scientometric indicators using the research articles published by authors with Colombian affiliation in journals indexed in the Scopus database belonging to the Atomic and Molecular Physics and Optics subject category. The research output has increas…
Systematic scientometric reviews, empowered by scientometric and visual analytic techniques, offer opportunities to improve the timeliness, accessibility, and reproducibility of conventional systematic reviews. While increasingly accessible science mapping tools enable end users to visualize the structure and dynamics of a research field, a common bottleneck in the current practice is the construction of a collection of scholarly publications as the input of the subsequent scientometric analysis and visualization. End users often have to face a dilemma in the preparation process: the more they…
Scientometric predictors of research performance need to be validated by showing that they have a high correlation with the external criterion they are trying to predict. The UK Research Assessment Exercise (RAE), together with the growing movement toward making the full-texts of research articles freely available on the web -- offer a unique opportunity to test and validate a wealth of old and new scientometric predictors, through multiple regression analysis: Publications, journal impact factors, citations, co-citations, citation chronometrics (age, growth, latency to peak, decay rate), hub/…
Many academics use yearly publication numbers to quantify academic interest for their research topic. While such visualisations are ubiquitous in grant applications, manuscript introductions, and review articles, they fail to account for the rapid growth in scientific publications. As a result, any search term will likely show an increase in supposed "academic interest". One proposed solution is to normalise yearly publication rates by field size, but this is arduous and difficult. Here, we propose an simpler index that normalises keywords of interest by a ubiquitous and innocuous keyword, suc…
This is the first detailed study on the coverage of Microsoft Academic (MA). Based on the complete and verified publication list of a university, the coverage of MA was assessed and compared with two benchmark databases, Scopus and Web of Science (WoS), on the level of individual publications. Citation counts were analyzed, and issues related to data retrieval and data quality were examined. A Perl script was written to retrieve metadata from MA based on publication titles. The script is freely available on GitHub. We find that MA covers journal articles, working papers, and conference items t…
Microsoft Academic is a free citation index that allows large scale data collection. This combination makes it useful for scientometric research. Previous studies have found that its citation counts tend to be slightly larger than those of Scopus but smaller than Google Scholar, with disciplinary variations. This study reports the largest and most systematic analysis so far, of 172,752 articles in 29 large journals chosen from different specialisms. From Scopus citation counts, Microsoft Academic citation counts and Mendeley reader counts for articles published 2007-2017, Microsoft Academic fo…
The citation impact of a scientific publication is usually seen as a one-dimensional concept. We introduce a multi-dimensional framework for characterizing the citation impact of a publication. In addition to the level of citation impact, quantified by the number of citations received by a publication, we also conceptualize and operationalize the depth and breadth and the dependence and independence of the citation impact of a publication. The proposed framework distinguishes between publications that have a deep citation impact, typically in a relatively narrow research area, and publications…
Scholarly knowledge graphs are valuable sources of information in several research fields. Despite the number of existing datasets related to publications and researchers, resource quality, coverage and accessibility are still limited. This article presents the Enhanced Microsoft Academic Knowledge Graph, a large dataset of information about scientific publications and involved entities, and the methods developed to build it. Data includes geographical information, researchers' collaborative networks and movements between institutions, academic-related metrics, and linguistic features. The dat…
Following in the footsteps of the model of scientific communication, which has recently gone through a metamorphosis (from the Gutenberg galaxy to the Web galaxy), a change in the model and methods of scientific evaluation is also taking place. A set of new scientific tools are now providing a variety of indicators which measure all actions and interactions among scientists in the digital space, making new aspects of scientific communication emerge. In this work we present a method for capturing the structure of an entire scientific community (the Bibliometrics, Scientometrics, Informetrics, W…
What does the cost of academic publishing look like to the common researcher today? Our goal is to convey the current state of academic publishing, specifically in regards to the field of computer science and provide analysis and data to be used as a basis for future studies. We will focus on author and reader costs as they are the primary points of interaction within the publishing world. In this work, we restrict our focus to only computer science in order to make the data collection more feasible (the authors are computer scientists) and hope future work can analyze and collect data across …
We introduce a novel methodology for mapping academic institutions based on their journal publication profiles. We believe that journals in which researchers from academic institutions publish their works can be considered as useful identifiers for representing the relationships between these institutions and establishing comparisons. However, when academic journals are used for research output representation, distinctions must be introduced between them, based on their value as institution descriptors. This leads us to the use of journal weights attached to the institution identifiers. Since …
The application of mathematics and statistical methods to scholarly communication: scientometrics, has facilitated the systematic analysis of the modern digital tide of literature. This chapter reviews three of such applications: coauthorship, bibliographic coupling, and coword networks. It also presents an exploratory case of study for the knowledge circulation literature. It was found a diverse geographical production, mainly in the Global North and Asian institutions with significant intermediation of universities from USA, Colombia, and Japan. The research fronts identified were related to…
The numerical size of academic publications that are being published in recent years had grown rapidly. Accessing and searching massive academic publications that are distributed over several locations need large amount of computing resources to increase the system performance. Therefore, many grid-based search techniques were proposed to provide flexible methods for searching the distributed extensive data. This paper proposes search technique that is capable of searching the extensive publications by utilizing grid computing technology. The search technique is implemented as interconnected g…
Academic publications have been evaluated in terms of their impact on research communities based on many metrics, such as the number of citations. On the other hand, the impact of academic publications on industry has been rarely studied. This paper investigates how academic publications contribute to software development by analyzing publication citations in source code comments in open source software repositories. We propose an automated approach for detecting academic publications based on Named Entity Recognition, and achieve 0.90 in $F_1$ as detection accuracy. We conduct a large-scale s…
This study presents a large scale analysis of the distribution and presence of Mendeley readership scores over time and across disciplines. We study whether Mendeley readership scores (RS) can identify highly cited publications more effectively than journal citation scores (JCS). Web of Science (WoS) publications with DOIs published during the period 2004-2013 and across 5 major scientific fields have been analyzed. The main result of this study shows that readership scores are more effective (in terms of precision/recall values) than journal citation scores to identify highly cited publicatio…
The process of innovation follows non-linear patterns across the domains of science, technology, and the economy. Novel bibliometric mapping techniques can be used to investigate and represent distinctive, but complementary perspectives on the innovation process (e.g., "demand" and "supply") as well as the interactions among these perspectives. The perspectives can be represented as "continents" of data related to varying extents over time. For example, the different branches of Medical Subject Headings (MeSH) in the Medline database provide sources of such perspectives (e.g., "Diseases" versu…
We explore if and how Microsoft Academic (MA) could be used for bibliometric analyses. First, we examine the Academic Knowledge API (AK API), an interface to access MA data, and compare it to Google Scholar (GS). Second, we perform a comparative citation analysis of researchers by normalizing data from MA and Scopus. We find that MA offers structured and rich metadata, which facilitates data retrieval, handling and processing. In addition, the AK API allows retrieving frequency distributions of citations. We consider these features to be a major advantage of MA over GS. However, we identify fo…
The task of expert finding has been getting increasing attention in information retrieval literature. However, the current state-of-the-art is still lacking in principled approaches for combining different sources of evidence in an optimal way. This paper explores the usage of learning to rank methods as a principled approach for combining multiple estimators of expertise, derived from the textual contents, from the graph-structure with the citation patterns for the community of experts, and from profile information about the experts. Experiments made over a dataset of academic publications, f…
The main objective of this work is to analyse the contributions of Judit Bar-Ilan to the search engines studies. To do this, two complementary approaches have been carried out. First, a systematic literature review of 47 publications authored and co-authored by Judit and devoted to this topic. Second, an interdisciplinarity analysis based on the cited references (publications cited by Judit) and citing documents (publications that cite Judit's work) through Scopus. The systematic literature review unravels an immense amount of search engines studied (43) and indicators measured (especially tec…
This paper presents a method for classifying the varying degrees of interdependency between academic hospitals and universities in the context of the Leiden Ranking. A key question for ranking universities is whether or not to allocate the publication output of affiliated hospitals to universities. Hospital nomenclatures vary worldwide to denote some form of collaboration with a university: academic hospitals, teaching hospitals, university hospitals, and academic medical centres do not correspond to universally standard definitions. Thus, rather than seeking a normative definition of academic…
