Google Scholar as a data source for research assessment
Emilio Delgado López-Cózar, Enrique Orduna-Malea, Alberto Martín-Martín
备注：
42 pages. Forthcoming in: Springer Handbook of Science and Technology Indicators (Editors: Wolfgang Glaenzel, Henk Moed, Ulrich Schmoch, Michael Thelwall)
The launch of Google Scholar (GS) marked the beginning of a revolution in the scientific information market. This search engine, unlike traditional databases, automatically indexes information from the academic web. Its ease of use, together with its wide coverage and fast indexing speed, have made it the first tool most scientists currently turn to when they need to carry out a literature search. Additionally, the fact that its search results were accompanied from the beginning by citation counts, as well as the later development of secondary products which leverage this citation data (such as Google Scholar Metrics and Google Scholar Citations), made many scientists wonder about its potential as a source of data for bibliometric analyses. The goal of this chapter is to lay the foundations for the use of GS as a supplementary source (and in some disciplines, arguably the best alternative) for scientific evaluation. First, we present a general overview of how GS works. Second, we present empirical evidences about its main characteristics (size, coverage, and growth rate). Third, we carry out a systematic analysis of the main limitations this search engine presents as a tool for the evaluation of scientific performance. Lastly, we discuss the main differences between GS and other more traditional bibliographic databases in light of the correlations found between their citation data. We conclude that Google Scholar presents a broader view of the academic world because it has brought to light a great amount of sources that were not previously visible.
The launch of Google Scholar back in 2004 meant a revolution not only in the scientific information search market but also in research evaluation processes. Its dynamism, unparalleled coverage, and uncontrolled indexing make of Google Scholar an unusual product, especially when compared to traditional bibliographic databases. Conceived primarily as a discovery tool for academic information, it presents a number of limitations as a bibliometric tool. The main objective of this chapter is to show how Google Scholar operates and how its core database may be used for bibliometric purposes. To do t…
Unlike other academic bibliographic databases, Google Scholar intentionally operates in a way that does not maintain coverage stability: documents that stop being available to Google Scholar's crawlers are removed from the system. This can also affect Google Scholar's citation graph (citation counts can decrease). Furthermore, because Google Scholar is not transparent about its coverage, the only way to directly observe coverage loss is through regular monitorization of Google Scholar data. Because of this, few studies have empirically documented this phenomenon. This study analyses a large de…
Dissertations can be the single most important scholarly outputs of junior researchers. Whilst sets of journal articles are often evaluated with the help of citation counts from the Web of Science or Scopus, these do not index dissertations and so their impact is hard to assess. In response, this article introduces a new multistage method to extract Google Scholar citation counts for large collections of dissertations from repositories indexed by Google. The method was used to extract Google Scholar citation counts for 77,884 American doctoral dissertations from 2013-2017 via ProQuest, with a …
The objective of this report is to present a list of proceedings (conferences, workshops, symposia, meetings) in the areas of Computer Science, Electrical & Electronic Engineering, and Communications covered by Google Scholar Metrics and ranked according to their h-index. Google Scholar Metrics only displays publications that have published at least 100 papers and have received at least one citation in the last five years (2009-2013). The searches were conducted between the 15th and 22nd of December, 2014. A total of 1208 proceedings have been identified
The objective of this report is to present a list of proceedings (conferences, workshops, symposia, meetings) in the areas of Computer Science, Electrical & Electronic Engineering, and Communications covered by Google Scholar Metrics and ranked according to their h-index. Google Scholar Metrics only displays publications that have published at least 100 papers and have received at least one citation in the last five years (2010-2014). The searches were conducted between the 8th and 10th of December, 2015. A total of 1501 proceedings have been identified
Google Scholar and the h-index in biomedicine: the popularization of bibliometric asessment
本文的目的是审查的特点，优点和局限性的新的科学评价产品来自谷歌学术;谷歌学术期刊和谷歌学术引文，以及h指数，这是标准的文献计量指标，这些服务采用。它还概述了这个新数据库作为生物医学研究来源的潜力，并通过从Web of Science，Scopus和Google Scholar提取的数据，比较了重症监护医学领域最相关的期刊和研究人员获得的h指数。结果显示，虽然Google Scholar的平均h指数值比Web of Science高出近30%，比Scopus收集的高出约15%，但两种数据源产生的排名没有实质性变化。尽管存在一些技术问题，但可以得出结论，Google Scholar是健康科学研究人员的有效工具，无论是用于信息检索还是计算文献计量指标
The emergence of academic search engines (mainly Google Scholar and Microsoft Academic Search) that aspire to index the entirety of current academic knowledge has revived and increased interest in the size of the academic web. The main objective of this paper is to propose various methods to estimate the current size (number of indexed documents) of Google Scholar (May 2014) and to determine its validity, precision and reliability. To do this, we present, apply and discuss three empirical methods: an external estimate based on empirical studies of Google Scholar coverage, and two internal esti…
This paper analyzes the new affiliation feature available in Google-Scholar Citations revealing that the affiliation-tool works well for most-institutions, it is unable to detect all existing institutions in database, and it is not always able to create unique-standardized entry for each-institution.
Purpose: The purpose of this paper is to measure the coverage of Google Scholar for the Library and Information Science (LIS) journal literature as identified by a list of core LIS journals from a study by Schloegl and Petschnig (2005). Methods: We checked every article from 35 major LIS journals from the years 2004 to 2006 for availability in Google Scholar (GS). We also collected information on the type of availability-i.e., whether a certain article was available as a PDF for a fee, as a free PDF, or as a preprint. Results: We found that only some journals are completely indexed by Google S…
After giving a brief overview of Eugene Garfield contributions to the issue of identifying and studying the most cited scientific articles, manifested in the creation of his Citation Classics, the main characteristics and features of Google Scholar new service Classic Papers, as well as its main strengths and weaknesses, are addressed. This product currently displays the most cited English-language original research articles by fields and published in 2006
The main objective of this paper is to empirically test whether the identification of highly-cited documents through Google Scholar is feasible and reliable. To this end, we carried out a longitudinal analysis (1950 to 2013), running a generic query (filtered only by year of publication) to minimise the effects of academic search engine optimisation. This gave us a final sample of 64,000 documents (1,000 per year). The strong correlation between a document's citations and its position in the search results (r= -0.67) led us to conclude that Google Scholar is able to identify highly-cited paper…
H Index of History journals published in Spain according to Google Scholar Metrics (2007-2011)
Google Scholar（GSM）最近于2012年4月推出，具有新的文献计量系统，通过计算Google Scholar中获得的引用数量来衡量科学期刊。通过这种方式，它为衡量期刊在人文领域的影响开辟了新的可能性。本文旨在通过分析2012年12月5日至6日在西班牙出版的历史期刊的GSM搜索来评估该工具的范围。总之，确定了69种期刊，仅占西班牙出版的历史期刊的24%。这个领域的H指数值的范围是如此之小，排名不能再说是显示出一个歧视的潜力。有鉴于此，我们建议改变Google Scholar的设计方式，以便它也可以适应特定历史领域的生产和引用模式，并在更广泛的范围内，在人文领域也是如此。
This report describes the feature introduced by Google to provide standardized access to institutional affiliations within Google Scholar Citations. First, this new tool is described, pointing out its main characteristics and functioning. Next, the coverage and precision of the tool are evaluated. Two special cases (Google Inc. and Spanish Universities) are briefly treated with the purpose of illustrating some aspects about the accuracy of the tool for the task of gathering authors within their appropriate institution. Finally, some inconsistencies, errors and malfunctioning are identified, ca…
The paper aims to analyze and monitor the research publication productivity of the faculty members of Bulacan State University. This paper compiles all the scientific publications from Bulacan State University (BulSU) and its external campuses as an index in Google Scholar. This study was intended to track and monitor the scientific productivity of the faculty members of Bulacan State University and each college and campus.
The 2016 edition of Google Scholar Metrics was released on July 15th 2016. There haven't been any structural changes respect to previous versions, which means that most of its limitations still persist. The biggest changes are the addition of five new language rankings (Russian, Korean, Polish, Ukrainian, and Indonesian) and elimination of two other language rankings (Italian and Dutch). In addition, for reasons still unknown, this new edition doesn't include as many working paper and discussion paper series as previous editions.
The emergence of academic search engines (Google Scholar and Microsoft Academic Search essentially) has revived and increased the interest in the size of the academic web, since their aspiration is to index the entirety of current academic knowledge. The search engine functionality and human search patterns lead us to believe, sometimes, that what you see in the search engine's results page is all that really exists. And, even when this is not true, we wonder which information is missing and why. The main objective of this working paper is to calculate the size of Google Scholar at present (Ma…
Ranking journals: Could Google Scholar Metrics be an alternative to Journal Citation Reports and Scimago Journal Rank?
Google Scholar作为评估科学期刊的工具的推出可能会对汤森路透期刊引用报告和Scopus驱动的Scimago期刊排名构成严重竞争。这些文献计量学期刊评价产品进行审查。我们从不同的方法比较它们的主要特征：覆盖范围，索引政策，搜索和可视化，文献计量指标，结果分析选项，经济成本和它们的期刊排名的差异。尽管存在缺点，但Google Scholar搜索引擎是作者和编辑识别核心期刊的有用工具。作为一种越来越有用的科学期刊排名工具，它也可能挑战老牌期刊产品
Trade and investment between developing regions such as China and Latin America (LATAM) are growing prominently. However, insights on crucial factors such as innovation in business and management (iBM) about both regions have not been scrutinized. This study presents the research output, impact, and structure of iBM research published about China and LATAM in a comparative framework using Google Scholar, Dimensions, and Microsoft Academic. Findings showed i) that iBM topics of both regions were framed within research and development management, and technological development topics, ii) signifi…
This paper presents proof that Google Scholar (GS) can construct documentary sets relevant for evaluating researchers' works. Nobelists in economics were the researchers under analysis, and two types of tests of the GS cites to their works were performed: distributional and semantic. Distributional tests found that the GS cites to the laureates' works conformed to the power-law model with an asymptote or "tail" conterminous with their h-index demarcating their core oeuvre, validating both GS and the h-index. Semantic tests revealed that their works highest in GS cites were on topics for which …
Recently, a review concluded that Google Scholar (GS) is not a suitable source of information "for identifying recent conference papers or other gray literature publications". The goal of this letter is to demonstrate that GS can be an effective tool to search and find gray literature, as long as appropriate search strategies are used. To do this, we took as examples the same two case studies used by the original review, describing first how GS processes original's search strategies, then proposing alternative search strategies, and finally generalizing each case study to compose a general sea…
ResearchGate has emerged as a popular professional network for scientists and researchers in a very short span of time. Similar to Google Scholar, the ResearchGate indexing uses an automatic crawling algorithm that extracts bibliographic data, citations and other information about scholarly articles from various sources. However, it has been observed that the two platforms often show different publication and citation data for the same institutions, journals and authors. This paper, therefore, attempts to analyse and measure the differences in publication counts, citations and different metric…
Objective. To analyse quantitatively the articles published during 2008_2017 about the academic social networking site ResearchGate. Methods. A scoping bibliometric review of documents retrieved using Google Scholar was conducted, limited to publications that contained the word "ResearchGate" in their title and were published from 2008 to 2017. Results. The search yielded 159 documents, once a preliminary list of 386 documents retrieved from Google Scholar was filtered, which eliminated about 60% of the results that were bibliographic citations and not documents. Papers in journals were the mo…
We measure the impact of "altmetrics" field by deploying altmetrics indicators using the data from Google Scholar, Twitter, Mendeley, Facebook, Google-plus, CiteULike, Blogs and Wiki during 2010- 2014. To capture the social impact of scientific publications, we propose an index called alt-index, analogues to h-index. Across the deployed indices, our results have shown high correlation among the indicators that capture social impact. While we observe medium Pearson's correlation (\r{ho}= .247) among the alt-index and h-index, a relatively high correlation is observed between social citations an…
The main objective of this paper is to identify the set of highly-cited documents in Google Scholar and to define their core characteristics (document types, language, free availability, source providers, and number of versions), under the hypothesis that the wide coverage of this search engine may provide a different portrait about this document set respect to that offered by the traditional bibliographic databases. To do this, a query per year was carried out from 1950 to 2013 identifying the top 1,000 documents retrieved from Google Scholar and obtaining a final sample of 64,000 documents, …
New sources of citation data have recently become available, such as Microsoft Academic, Dimensions, and the OpenCitations Index of CrossRef open DOI-to-DOI citations (COCI). Although these have been compared to the Web of Science (WoS), Scopus, or Google Scholar, there is no systematic evidence of their differences across subject categories. In response, this paper investigates 3,073,351 citations found by these six data sources to 2,515 English-language highly-cited documents published in 2006 from 252 subject categories, expanding and updating the largest previous study. Google Scholar foun…
As the number of empirical studies increases unprecedentedly in line with expansions in higher education, theoretical developments in population studies suffer due to a discoverability crisis of related work. The systematic use of previous research is common in the medical sciences through various types of structured reviews. However, these are less common in the social sciences, despite their potential, especially in cross-disciplinary fields such as migration. We use Google Scholar to examine the niche of housing research within migration studies through a broad range of documents. The contr…
This article describes a procedure to generate a snapshot of the structure of a specific scientific community and their outputs based on the information available in Google Scholar Citations (GSC). We call this method MADAP (Multifaceted Analysis of Disciplines through Academic Profiles). The international community of researchers working in Bibliometrics, Scientometrics, Informetrics, Webometrics, and Altmetrics was selected as a case study. The records of the top 1,000 most cited documents by these authors according to GSC were manually processed to fill any missing information and deduplica…
The suggested methodic is the way of formatting the subject areas models and co-authors networks by sounding the content networks. The paper represents the notion networks which match tags and authors of Google Scholar Citations service. Models depicted in the work were built for the physical optics area, and it can be applied for other domains. The proposed ways of defining connections between science areas and authors depicts the collaborations opportunities and versatility of interdisciplinary.
This study explores the extent to which bibliometric indicators based on counts of highly-cited documents could be affected by the choice of data source. The initial hypothesis is that databases that rely on journal selection criteria for their document coverage may not necessarily provide an accurate representation of highly-cited documents across all subject areas, while inclusive databases, which give each document the chance to stand on its own merits, might be better suited to identify highly-cited documents. To test this hypothesis, an analysis of 2,515 highly-cited documents published i…
Following in the footsteps of the model of scientific communication, which has recently gone through a metamorphosis (from the Gutenberg galaxy to the Web galaxy), a change in the model and methods of scientific evaluation is also taking place. A set of new scientific tools are now providing a variety of indicators which measure all actions and interactions among scientists in the digital space, making new aspects of scientific communication emerge. In this work we present a method for capturing the structure of an entire scientific community (the Bibliometrics, Scientometrics, Informetrics, W…
Most currently available schemes for performance based ranking of Universities or Research organizations, such as, Quacarelli Symonds (QS), Times Higher Education (THE), Shanghai University based All Research of World Universities (ARWU) use a variety of criteria that include productivity, citations, awards, reputation, etc., while Leiden and Scimago use only bibliometric indicators. The research performance evaluation in the aforesaid cases is based on bibliometric data from Web of Science or Scopus, which are commercially available priced databases. The coverage includes peer reviewed journa…
Despite citation counts from Google Scholar (GS), Web of Science (WoS), and Scopus being widely consulted by researchers and sometimes used in research evaluations, there is no recent or systematic evidence about the differences between them. In response, this paper investigates 2,448,055 citations to 2,299 English-language highly-cited documents from 252 GS subject categories published in 2006, comparing GS, the WoS Core Collection, and Scopus. GS consistently found the largest percentage of citations across all areas (93%-96%), far ahead of Scopus (35%-77%) and WoS (27%-73%). GS found nearly…
This article uses Google Scholar (GS) as a source of data to analyse Open Access (OA) levels across all countries and fields of research. All articles and reviews with a DOI and published in 2009 or 2014 and covered by the three main citation indexes in the Web of Science (2,269,022 documents) were selected for study. The links to freely available versions of these documents displayed in GS were collected. To differentiate between more reliable (sustainable and legal) forms of access and less reliable ones, the data extracted from GS was combined with information available in DOAJ, CrossRef, O…
A new methodology is proposed for comparing Google Scholar (GS) with other citation indexes. It focuses on the coverage and citation impact of sources, indexing speed, and data quality, including the effect of duplicate citation counts. The method compares GS with Elsevier's Scopus, and is applied to a limited set of articles published in 12 journals from six subject fields, so that its findings cannot be generalized to all journals or fields. The study is exploratory, and hypothesis generating rather than hypothesis-testing. It confirms findings on source coverage and citation impact obtained…
The study of highly cited documents on Google Scholar (GS) has never been addressed to date in a comprehensive manner. The objective of this work is to identify the set of highly cited documents in Google Scholar and define their core characteristics: their languages, their file format, or how many of them can be accessed free of charge. We will also try to answer some additional questions that hopefully shed some light about the use of GS as a tool for assessing scientific impact through citations. The decalogue of research questions is shown below: 1. Which are the most cited documents in GS…
In this paper, the scientometric evaluation of faculty members of 50 Greek Science and Engineering University Departments is presented. 1978 academics were examined in total. The number of papers, citations, h-index and i10-index have been collected for each academic, department, school and university using Google Scholar and the citations analysis program Publish or Perish. Analysis of the collected data showed that departments of the same academic discipline are characterized by significant differences on the scientific outcome. In addition, in the majority of the evaluated departments a sig…
The new web-based academic communication platforms do not only enable researchers to better advertise their academic outputs, making them more visible than ever before, but they also provide a wide supply of metrics to help authors better understand the impact their work is making. This study has three objectives: a) to analyse the uptake of some of the most popular platforms (Google Scholar Citations, ResearcherID, ResearchGate, Mendeley and Twitter) by a specific scientific community (bibliometrics, scientometrics, informetrics, webometrics, and altmetrics); b) to compare the metrics availab…
With Google Scholar, scientists can maintain their publications on personal profile pages, while the citations to these works are automatically collected and counted. Maintenance of publications is done manually by the researcher herself, and involves deleting erroneous ones, merging ones that are the same but which were not recognized as the same, adding forgotten co-authors, and correcting titles of papers and venues. The publications are presented on pages with 20 or 100 papers in the web page interface from 2012--2014. The interface does not allow a scientist to merge two version of a pape…
Most of the scientometric indicators use only the total number of citations of an article and produce a single number for scientific assessment of scholars. Although this concept is very simple to compute, it fails to show the scientific productivity and impact of scholars during a time-span or in a year. To overcome this, several time series indicators have been proposed that consider the citations from the entire research career of a scholar. However, these indicators fail to give a comparative assessment of two scholars having same or very similar index value. To overcome this shortcoming, …
A study released by the Google Scholar team found an apparently increasing fraction of citations to old articles from studies published in the last 24 years (1990-2013). To demonstrate this finding we conducted a complementary study using a different data source (Journal Citation Reports), metric (aggregate cited half-life), time spam (2003-2013), and set of categories (53 Social Science subject categories and 167 Science subject categories). Although the results obtained confirm and reinforce the previous findings, the possible causes of this phenomenon keep unclear. We finally hypothesize th…
Does it Matter Which Citation Tool is Used to Compare the h-index of a Group of Highly Cited Researchers?
由引文索引（Scopus，Google Scholar和Web of Science）检索的h指数用于衡量科学家的科学表现和研究影响力研究，基于科学家的出版物和引文数量。它也很容易获得，可用于科学家的业绩衡量和招聘决定。本研究的目的是调查这三个引文数据库，即Scopus，Google Scholar和Web of Science基于一组高被引研究人员（诺贝尔奖获得者科学家）的h指数的输出和结果之间的差异。本研究采立意抽样方法收集资料。结果表明，Scopus、Google scholar和Web of Science 3个数据库的h指数存在显著性差异，其中Google scholar h指数大于其他2个数据库的h指数。研究还表明，基于Google Scholar和Scopus的h指数之间存在显著的正相关关系。Scopus、Google Scholar和Web of Science的引文索引对科学家h指数的评价有一定的参考价值，但也存在一定的局限性。
The overarching aim of this work is to provide a detailed description of the free version of Dimensions (new bibliographic database produced by Digital Science and launched in January 2018). To do this, the work is divided into two differentiated blocks. First, its characteristics, operation and features are described, focusing on its main strengths and weaknesses. Secondly, an analysis of its coverage is carried out (comparing it Scopus and Google Scholar) in order to determine whether the bibliometric indicators offered by Dimensions have an order of magnitude significant enough to be used. …
