Most currently available schemes for performance based ranking of Universities or Research organizations, such as, Quacarelli Symonds (QS), Times Higher Education (THE), Shanghai University based All Research of World Universities (ARWU) use a variety of criteria that include productivity, citations, awards, reputation, etc., while Leiden and Scimago use only bibliometric indicators. The research performance evaluation in the aforesaid cases is based on bibliometric data from Web of Science or Scopus, which are commercially available priced databases. The coverage includes peer reviewed journals and conference proceedings. Google Scholar (GS) on the other hand, provides a free and open alternative to obtaining citations of papers available on the net, (though it is not clear exactly which journals are covered.) Citations are collected automatically from the net and also added to self created individual author profiles under Google Scholar Citations (GSC). This data was used by Webometrics Lab, Spain to create a ranked list of 4000+ institutions in 2016, based on citations from only the top 10 individual GSC profiles in each organization. (GSC excludes the top paper for reasons explained in the text; the simple selection procedure makes the ranked list size-independent as claimed by the Cybermetrics Lab). Using this data (Transparent Ranking TR, 2016), we find the regional and country wise distribution of GS-TR Citations. The size independent ranked list is subdivided into deciles of 400 institutions each and the number of institutions and citations of each country obtained for each decile. We test for correlation between institutional ranks between GS TR and the other ranking schemes for the top 20 institutions.
Scholarly impact may be metricized using an author's total number of citations as a stand-in for real worth, but this measure varies in applicability between disciplines. The detail of the number of citations per publication is nowadays mapped in much more detail on the Web, exposing certain empirical patterns. This paper explores those patterns, using the citation data from Google Scholar for a number of authors.
As part of its scholarly data efforts, the Internet Archive (IA) releases a first version of a citation graph dataset, named refcat, derived from scholarly publications and additional data sources. It is composed of data gathered by the fatcat cataloging project (the catalog that underpins IA Scholar), related web-scale crawls targeting primary and secondary scholarly outputs, as well as metadata from the Open Library project and Wikipedia. This first version of the graph consists of over 1.3B citations. We release this dataset under a CC0 Public Domain Dedication, accessible through Internet …
This paper analyzes the new affiliation feature available in Google-Scholar Citations revealing that the affiliation-tool works well for most-institutions, it is unable to detect all existing institutions in database, and it is not always able to create unique-standardized entry for each-institution.
The launch of Google Scholar Citations and Google Scholar Metrics may provoke a revolution in the research evaluation field as it places within every researchers reach tools that allow bibliometric measuring. In order to alert the research community over how easily one can manipulate the data and bibliometric indicators offered by Google s products we present an experiment in which we manipulate the Google Citations profiles of a research group through the creation of false documents that cite their documents, and consequently, the journals in which they have published modifying their H index.…
Scholar Ranking 2023 is the second edition of U.S. Computer Science (CS) departments ranking based on faculty citation measures. Using Google Scholar, we gathered data about publication citations for 5,574 tenure-track faculty from 185 U.S. universities. For each faculty, we extracted their t10 index, defined as the number of citations received by their 10th highest cited paper. For each department, we calculated four quality metrics: median t10 (m10), the geometric mean of t10 (g10), and the number of well-cited faculty with t10 above 40% (c40) and 60% (c60) of the national average. We fitted…
Dissertations can be the single most important scholarly outputs of junior researchers. Whilst sets of journal articles are often evaluated with the help of citation counts from the Web of Science or Scopus, these do not index dissertations and so their impact is hard to assess. In response, this article introduces a new multistage method to extract Google Scholar citation counts for large collections of dissertations from repositories indexed by Google. The method was used to extract Google Scholar citation counts for 77,884 American doctoral dissertations from 2013-2017 via ProQuest, with a …
OpenCitations is an infrastructure organization for open scholarship dedicated to the publication of open citation data as Linked Open Data using Semantic Web technologies, thereby providing a disruptive alternative to traditional proprietary citation indexes. Open citation data are valuable for bibliometric analysis, increasing the reproducibility of large-scale analyses by enabling publication of the source data. Following brief introductions to the development and benefits of open scholarship and to Semantic Web technologies, this paper describes OpenCitations and its datasets, tools, servi…
New sources of citation data have recently become available, such as Microsoft Academic, Dimensions, and the OpenCitations Index of CrossRef open DOI-to-DOI citations (COCI). Although these have been compared to the Web of Science (WoS), Scopus, or Google Scholar, there is no systematic evidence of their differences across subject categories. In response, this paper investigates 3,073,351 citations found by these six data sources to 2,515 English-language highly-cited documents published in 2006 from 252 subject categories, expanding and updating the largest previous study. Google Scholar foun…
ResearchGate has emerged as a popular professional network for scientists and researchers in a very short span of time. Similar to Google Scholar, the ResearchGate indexing uses an automatic crawling algorithm that extracts bibliographic data, citations and other information about scholarly articles from various sources. However, it has been observed that the two platforms often show different publication and citation data for the same institutions, journals and authors. This paper, therefore, attempts to analyse and measure the differences in publication counts, citations and different metric…
The launch of Google Scholar Metrics as a tool for assessing scientific journals may be serious competition for Thomson Reuters Journal Citation Reports, and for Scopus powered Scimago Journal Rank. A review of these bibliometric journal evaluation products is performed. We compare their main characteristics from different approaches: coverage, indexing policies, search and visualization, bibliometric indicators, results analysis options, economic cost and differences in their ranking of journals. Despite its shortcomings, Google Scholar Metrics is a helpful tool for authors and editors in ide…
This report describes the feature introduced by Google to provide standardized access to institutional affiliations within Google Scholar Citations. First, this new tool is described, pointing out its main characteristics and functioning. Next, the coverage and precision of the tool are evaluated. Two special cases (Google Inc. and Spanish Universities) are briefly treated with the purpose of illustrating some aspects about the accuracy of the tool for the task of gathering authors within their appropriate institution. Finally, some inconsistencies, errors and malfunctioning are identified, ca…
Despite citation counts from Google Scholar (GS), Web of Science (WoS), and Scopus being widely consulted by researchers and sometimes used in research evaluations, there is no recent or systematic evidence about the differences between them. In response, this paper investigates 2,448,055 citations to 2,299 English-language highly-cited documents from 252 GS subject categories published in 2006, comparing GS, the WoS Core Collection, and Scopus. GS consistently found the largest percentage of citations across all areas (93%-96%), far ahead of Scopus (35%-77%) and WoS (27%-73%). GS found nearly…
This paper presents proof that Google Scholar (GS) can construct documentary sets relevant for evaluating researchers' works. Nobelists in economics were the researchers under analysis, and two types of tests of the GS cites to their works were performed: distributional and semantic. Distributional tests found that the GS cites to the laureates' works conformed to the power-law model with an asymptote or "tail" conterminous with their h-index demarcating their core oeuvre, validating both GS and the h-index. Semantic tests revealed that their works highest in GS cites were on topics for which …
Google Scholar has been well received by the research community. Its promises of free, universal and easy access to scientific literature as well as the perception that it covers better than other traditional multidisciplinary databases the areas of the Social Sciences and the Humanities have contributed to the quick expansion of Google Scholar Citations and Google Scholar Metrics: two new bibliometric products that offer citation data at the individual level and at journal level. In this paper we show the results of a experiment undertaken to analyze Google Scholar's capacity to detect citati…
Citing is an important aspect of scientific discourse and important for quantifying the scientific impact quantification of researchers. Previous works observed that citations are made not only based on the pure scholarly contributions but also based on non-scholarly attributes, such as the affiliation or gender of authors. In this way, citation bias is produced. Existing works, however, have not analyzed preprints with respect to citation bias, although they play an increasingly important role in modern scholarly communication. In this paper, we investigate whether preprints are affected by c…
A statistical analysis of full text downloads of articles in Elseviers ScienceDirect covering all disciplines reveals large differences in download frequencies, their skewness, and their correlation with Scopus-based citation counts, between disciplines, journals, and document types. Download counts tend to be two orders of magnitude higher and less skewedly distributed than citations. A mathematical model based on the sum of two exponentials does not adequately capture monthly download counts. The degree of correlation at the article level within a journal is similar to that at the journal le…
One of the critical issues in bibliometric research assessments is the time required to achieve maturity in citations. Citation counts can be considered a reliable proxy of the real impact of a work only if they are observed after sufficient time has passed from publication date. In the present work the authors investigate the effect of varying the time of citation observation on accuracy of productivity rankings for research institutions. Research productivity measures are calculated for all Italian universities active in the hard sciences in the 2001-2003 period, by individual field and disc…
A new methodology is proposed for comparing Google Scholar (GS) with other citation indexes. It focuses on the coverage and citation impact of sources, indexing speed, and data quality, including the effect of duplicate citation counts. The method compares GS with Elsevier's Scopus, and is applied to a limited set of articles published in 12 journals from six subject fields, so that its findings cannot be generalized to all journals or fields. The study is exploratory, and hypothesis generating rather than hypothesis-testing. It confirms findings on source coverage and citation impact obtained…
This article describes a procedure to generate a snapshot of the structure of a specific scientific community and their outputs based on the information available in Google Scholar Citations (GSC). We call this method MADAP (Multifaceted Analysis of Disciplines through Academic Profiles). The international community of researchers working in Bibliometrics, Scientometrics, Informetrics, Webometrics, and Altmetrics was selected as a case study. The records of the top 1,000 most cited documents by these authors according to GSC were manually processed to fill any missing information and deduplica…
Most of the scientometric indicators use only the total number of citations of an article and produce a single number for scientific assessment of scholars. Although this concept is very simple to compute, it fails to show the scientific productivity and impact of scholars during a time-span or in a year. To overcome this, several time series indicators have been proposed that consider the citations from the entire research career of a scholar. However, these indicators fail to give a comparative assessment of two scholars having same or very similar index value. To overcome this shortcoming, …
Unraveling the Ai2 Asta Scholarly Research Assistant Citation System
The iMetrics Lab. Department of Audiovisual Communication, Documentation and History of Art, Universitat Politècnica de València（瓦伦西亚理工大学）；Departament de Biblioteconomia, Documentació i Comunicació Audiovisual, Universitat de Barcelona（巴塞罗那大学）
The suggested methodic is the way of formatting the subject areas models and co-authors networks by sounding the content networks. The paper represents the notion networks which match tags and authors of Google Scholar Citations service. Models depicted in the work were built for the physical optics area, and it can be applied for other domains. The proposed ways of defining connections between science areas and authors depicts the collaborations opportunities and versatility of interdisciplinary.
Assessing the influence of a scholar's work is an important task for funding organizations, academic departments, and researchers. Common methods, such as measures of citation counts, can ignore much of the nuance and multidimensionality of scholarly influence. We present an approach for generating dynamic visualizations of scholars' careers. This approach uses an animated node-link diagram showing the citation network accumulated around the researcher over the course of the career in concert with key indicators, highlighting influence both within and across fields. We developed our design in …
Traditionally, scholarly impact and visibility have been measured by counting publications and citations in the scholarly literature. However, increasingly scholars are also visible on the Web, establishing presences in a growing variety of social ecosystems. But how wide and established is this presence, and how do measures of social Web impact relate to their more traditional counterparts? To answer this, we sampled 57 presenters from the 2010 Leiden STI Conference, gathering publication and citations counts as well as data from the presenters' Web "footprints." We found Web presence widespr…
The new web-based academic communication platforms do not only enable researchers to better advertise their academic outputs, making them more visible than ever before, but they also provide a wide supply of metrics to help authors better understand the impact their work is making. This study has three objectives: a) to analyse the uptake of some of the most popular platforms (Google Scholar Citations, ResearcherID, ResearchGate, Mendeley and Twitter) by a specific scientific community (bibliometrics, scientometrics, informetrics, webometrics, and altmetrics); b) to compare the metrics availab…
This paper investigates the impact of institutes and papers over time based on the heterogeneous institution-citation network. A new model, IPRank, is introduced to measure the impact of institution and paper simultaneously. This model utilises the heterogeneous structural measure method to unveil the impact of institution and paper, reflecting the effects of citation, institution, and structural measure. To evaluate the performance, the model first constructs a heterogeneous institution-citation network based on the American Physical Society (APS) dataset. Subsequently, PageRank is used to qu…
Citations measure the importance of a publication, and may serve as a proxy for its popularity and quality of its contents. Here we study the distributions of citations to publications from individual academic institutions for a single year. The average number of citations have large variations between different institutions across the world, but the probability distributions of citations for individual institutions can be rescaled to a common form by scaling the citations by the average number of citations for that institution. We find this feature seem to be universal for a broad selection o…
Following in the footsteps of the model of scientific communication, which has recently gone through a metamorphosis (from the Gutenberg galaxy to the Web galaxy), a change in the model and methods of scientific evaluation is also taking place. A set of new scientific tools are now providing a variety of indicators which measure all actions and interactions among scientists in the digital space, making new aspects of scientific communication emerge. In this work we present a method for capturing the structure of an entire scientific community (the Bibliometrics, Scientometrics, Informetrics, W…
Academic institutions, federal agencies, publishers, editors, authors, and librarians increasingly rely on citation analysis for making hiring, promotion, tenure, funding, and/or reviewer and journal evaluation and selection decisions. The Institute for Scientific Information's (ISI) citation databases have been used for decades as a starting point and often as the only tools for locating citations and/or conducting citation analyses. ISI databases (or Web of Science), however, may no longer be adequate as the only or even the main sources of citations because new databases and tools that allo…
In our chapter we address the statistical analysis of percentiles: How should the citation impact of institutions be compared? In educational and psychological testing, percentiles are already used widely as a standard to evaluate an individual's test scores - intelligence tests for example - by comparing them with the percentiles of a calibrated sample. Percentiles, or percentile rank classes, are also a very suitable method for bibliometrics to normalize citations of publications in terms of the subject category and the publication year and, unlike the mean-based indicators (the relative cit…
We measure the impact of "altmetrics" field by deploying altmetrics indicators using the data from Google Scholar, Twitter, Mendeley, Facebook, Google-plus, CiteULike, Blogs and Wiki during 2010- 2014. To capture the social impact of scientific publications, we propose an index called alt-index, analogues to h-index. Across the deployed indices, our results have shown high correlation among the indicators that capture social impact. While we observe medium Pearson's correlation (\r{ho}= .247) among the alt-index and h-index, a relatively high correlation is observed between social citations an…
In this article, we focus on the importance of open research information as the foundation for transparent and responsible research assessment and discovery of research outputs. We introduce work in which we support the open research information commons by enabling, in particular, independent and small Open Access journals to provide metadata to several open data hubs (Open Citations, Wikidata, Open Research Knowledge Graph). In this context, we present The OPTIMETA Way, a means to integrate metadata collection, enrichment, and distribution in an effective and quality-ensured way that enables …
The launch of Google Scholar (GS) marked the beginning of a revolution in the scientific information market. This search engine, unlike traditional databases, automatically indexes information from the academic web. Its ease of use, together with its wide coverage and fast indexing speed, have made it the first tool most scientists currently turn to when they need to carry out a literature search. Additionally, the fact that its search results were accompanied from the beginning by citation counts, as well as the later development of secondary products which leverage this citation data (such a…
The COVID-19 pandemic requires a fast response from researchers to help address biological, medical and public health issues to minimize its impact. In this rapidly evolving context, scholars, professionals and the public may need to quickly identify important new studies. In response, this paper assesses the coverage of scholarly databases and impact indicators during 21 March to 18 April 2020. The results confirm a rapid increase in the volume of research, which particularly accessible through Google Scholar and Dimensions, and less through Scopus, the Web of Science, PubMed. A few COVID-19 …
h-index retrieved by citation indexes (Scopus, Google scholar, and Web of Science) is used to measure the scientific performance and the research impact studies based on the number of publications and citations of a scientist. It also is easily available and may be used for performance measures of scientists, and for recruitment decisions. The aim of this study is to investigate the difference between the outputs and results from these three citation databases namely Scopus, Google Scholar, and Web of Science based upon the h-index of a group of highly cited researchers (Nobel Prize winner sci…
Science is a cumulative activity, which can manifest itself through the act of citing. Citations are also central to research evaluation, thus creating incentives for researchers to cite their own work. Using a dataset containing more than 63 million articles and 51 million disambiguated authors, this paper examines the relative importance of self-citations and self-references in the scholarly communication landscape, their relationship with the age and gender of authors, as well as their effects on various research evaluation indicators. Results show that self-citations and self-references ev…
The vast amount of research produced at institutions world-wide is extremely diverse, and coarse-grained quantitative measures of impact often obscure the individual contributions of these institutions to specific research fields and topics. We show that by applying an information retrieval model to index research articles which are faceted by institution and time, we can develop tools to rank institutions given a keyword query. We present an interactive atlas, Quoka, designed to enable a user to explore these rankings contextually by geography and over time. Through a set of use cases we demo…
This review summarizes papers which analyze impact of self-citation on research evaluation. We introduce a generalized definition of self-citation and its variants: author, institutional, country, journal, discipline, publisher self-citation. Formulae of the basic self-citation measures are given, namely self-citing and self-cited rates. World literature on author, institutional, country and journal self-citation is studied in more detail. Current views on the role and impact of self-citation are compiled and analyzed. It is found that there is a general consensus on some points: (a) pathologi…
Understanding the cognitive evolution of researchers as they progress in the academia is an important but complex problem, a problem belonging to a class of problems, which often require the development of models for gaining further understanding in the intricacies of the domain. The research question that we address in this paper is how to effectively model this temporal cognitive mental development of prolific researchers. Our proposed solution to this problem is based on noting that the academic progression and notability of a researcher are linked with a progressive increase in the citatio…
Unlike other academic bibliographic databases, Google Scholar intentionally operates in a way that does not maintain coverage stability: documents that stop being available to Google Scholar's crawlers are removed from the system. This can also affect Google Scholar's citation graph (citation counts can decrease). Furthermore, because Google Scholar is not transparent about its coverage, the only way to directly observe coverage loss is through regular monitorization of Google Scholar data. Because of this, few studies have empirically documented this phenomenon. This study analyses a large de…
There are two principal data sources for collaborative filtering recommenders in scholarly digital libraries: usage data obtained from harvesting a large, distributed collection of Open URL web logs and citation data obtained from the journal articles. This study explores the characteristics of recommendations generated by implementations of these two methods: the 'bX' system by ExLibris and an experimental citation-based recommender, Sarkanto. Recommendations from each system were compared according to their semantic similarity to the seed article that was used to generate them. Since the ful…
Citation numbers and other quantities derived from bibliographic databases are becoming standard tools for the assessment of productivity and impact of research activities. Though widely used, still their statistical properties have not been well established so far. This is especially true in the case of bibliometric indicators aimed at the evaluation of individual scholars, because large-scale data sets are typically difficult to be retrieved. Here, we take advantage of a recently introduced large bibliographic data set, Google Scholar Citations, which collects the entire publication record o…
The study of highly cited documents on Google Scholar (GS) has never been addressed to date in a comprehensive manner. The objective of this work is to identify the set of highly cited documents in Google Scholar and define their core characteristics: their languages, their file format, or how many of them can be accessed free of charge. We will also try to answer some additional questions that hopefully shed some light about the use of GS as a tool for assessing scientific impact through citations. The decalogue of research questions is shown below: 1. Which are the most cited documents in GS…
Google Scholar Metrics (GSM), which was recently launched in April 2012, features new bibliometric systems for gauging scientific journals by counting the number of citations obtained in Google Scholar. This way, it opens new possibilities for measuring journal impacts in the field of Humanities. The present article intends to evaluate the scope of this tool through analysing GSM searches, from the 5th through 6th of December 2012, of History journals published in Spain. In sum, 69 journals were identified, accounting for only 24% of the History journals published in Spain. The ranges of H ind…
With the vast majority of scientific papers now available online, this paper describes how the Web is allowing physicists and information providers to measure more accurately the impact of these papers and their authors. Provides a historical background of citation analysis, impact factor, new citation data sources (e.g., Google Scholar, Scopus, NASA's Astrophysics Data System Abstract Service, MathSciNet, ScienceDirect, SciFinder Scholar, Scitation/SPIN, and SPIRES-HEP), as well as h-index, g-index, and a-index.
The need for scholarly open data is ever increasing. While there are large repositories of open access articles and free publication indexes, there are still a few examples of free citation networks and their coverage is partial. One of the results is that most of the evaluation processes based on citation counts rely on commercial citation databases. Things are changing under the pressure of the Initiative for Open Citations (I4OC), whose goal is to campaign for scholarly publishers to make their citations as totally open. This paper investigates the growth of open citations with an experimen…
ChatGPT cites the most-cited articles and journals, relying solely on Google Scholar's citation counts. As a result, AI may amplify the Matthew Effect in environmental science
Charles University（查理大学）；OpenAI, Inc（OpenAI公司）
本研究通过让 ChatGPT 撰写环境科学子学科综述并分析其参考文献，发现其仅依赖 Google Scholar 引用次数而倾向引用高被引、较早且发表于顶级期刊的文献，从而可能加剧科学引用中的马太效应。
Hallucination Detector: A hybrid LLM and Semantic Scholar tool calling for detecting hallucination in scientific literature on AtomGPT.org
Hopkins Extreme Materials Institute, Johns Hopkins University（霍普金斯极端材料研究所，约翰斯·霍普金斯大学）；Department of Materials Science and Engineering, Johns Hopkins University（材料科学与工程系，约翰斯·霍普金斯大学）；Department of Physics and Astronomy, West Virginia University（物理与天文学系，西弗吉尼亚大学）；Department of Electrical and Computer Engineering, Johns Hopkins University（电气与计算机工程系，约翰斯·霍普金斯大学）
