Multivariate linear regression models suggest a trade-off in allocations of national R&D investments. Government funding, and spending in the higher education sector, seem to encourage publications, whereas other components such as industrial funding, and spending in the business sector, encourage patenting. Our results help explain why the US trails the EU in publications, because of its focus on industrial funding - some 70% of its total R&D investment. Conversely, it also helps explain why the EU trails the US in patenting. Government funding is indicated as a negative incentive to high-quality patenting. The models here can also be used to predict an output indicator for a country, once the appropriate input indicator is known. This usually is done within a dataset for a single year, but the process can be extended to predict outputs a few years into the future, if reasonable forecasts can be made of the input indicators. We provide new forecasts about the further relationships of the US, the EU-27, and the PRC in the case of publishing. Models for individual countries may be more successful, however, than regression models whose parameters are averaged over a set of countries.
Novelpy (v1.2) is an open-source Python package designed to compute bibliometrics indicators. The package aims to provide a tool to the scientometrics community that centralizes different measures of novelty and disruptiveness, enables their comparison and fosters reproducibility. This paper offers a comprehensive review of the different indicators available in Novelpy by formally describing these measures (both mathematically and graphically) and presenting their benefits and limitations. We then compare the different measures on a random sample of 1.5M articles drawn from Pubmed Knowledge Gr…
The use of outcome control modes of research evaluation exercises is ever more frequent. They are conceived as tools to stimulate increased levels of research productivity, and to guide choices in allocating components of government research budgets for publicly funded institutions. There are several contributions in the literature that compare the different methodological approaches that policy makers could adopt for these exercises, however the comparisons are limited to only a few disciplines. This work, examining the case of the whole of the "hard sciences" of the Italian academic system, …
One of the notable features of undone science debates is how formation of new interest groups becomes pivotal in mobilizing and championing emerging research on undone topics. Clearly money is one of the most important mediums through which different types of actors can support and steer scientists to work on undone topics. Yet which actors are more visible in their support for scientific research is something which has seldom been measured. This study delves into research funding in the context of rare diseases research, a topic which has evolved from the margins of medical research into a pr…
Thomson Reuters' Web of Science (WoS) began systematically collecting acknowledgment information in August 2008. Since then, bibliometric analysis of funding acknowledgment (FA) has been growing and has aroused intense interest and attention from both academia and policy makers. Examining the distribution of FA by citation index database, by language, and by acknowledgment type, we noted coverage limitations and potential biases in each analysis. We argue that in spite of its great value, bibliometric analysis of FA should be used with caution.
This paper deploys bibliometric indices and semantic techniques for understanding to what extent research grants are likely to impact publications, research direction, and co-authorship rate of principal investigators. The novelty of this paper lies within the fact that it includes semantic analysis in the research funding evaluation process in order to effectively study short-term and long-term funding impact in terms of publication outputs. Our dataset consists of researchers that receive research grants from the National ICT Research and Development funding program of Pakistan. We show a nu…
Matching Researchers to Funding Calls: A Reproducible Institution-Level Framework
INGENIO (CSIC-UPV)（INGENIO（CSIC-UPV））；EC3metrics Spin-Off, University of Granada（格拉纳达大学EC3metrics衍生公司）；Department of Information and Communication Sciences, University of Granada（格拉纳达大学信息与传播科学系）
On August 25, 2022, the White House Office of Science and Technology Policy (OSTP) released a memo regarding public access to scientific research. Signed by Director Alondra Nelson, this updated guidance eliminated the 12-month embargo period on publications arising from U.S. federal funding that had been allowed from a previous 2013 OSTP memo. While reactions to this updated federal guidance have been plentiful, to date there has not been a detailed analysis of the publications which would fall under this new framework. The OSTP released a companion report along with the memo, but it only pro…
Bi-National Academic Funding and Collaboration Dynamics: The Case of the German-Israeli Foundation
Bar-Ilan University（巴伊兰大学）；University of Haifa（海法大学）；Jönköping University（延雪平大学）；GESIS – Leibniz-Institute for the Social Sciences（GESIS – 莱布尼茨社会科学研究所）
The rising trend of coauthored academic works obscures the credit assignment that is the basis for decisions of funding and career advancements. In this paper, a simple model based on the assumption of an unvarying "author ability" is introduced. With this assumption, the weight of author contributions to a body of coauthored work can be statistically estimated. The method is tested on a set of some more than five-hundred authors in a coauthor network from the CiteSeerX database. The ranking obtained agrees fairly well with that given by total fractional citation counts for an author, but noti…
Governments and organizations design performance-based research funding systems (PBFRS) for strategic aims, such as to selectively allocate scarce resources and stimulate research efficiency. In this work we analyze the relative change in research productivity of Italian universities after the introduction of such a system, featuring financial and reputational incentives. Using a bibliometric approach, we compare the relative research performance of universities before and after introduction of PBFRS, at the overall, discipline and field levels. The findings show convergence in the universitie…
Research funding and reputation in the UK have, for over two decades, been increasingly dependent on a regular peer-review of all UK departments. This is to move to a system more based on bibliometrics. Assessment exercises of this kind influence the behavior of institutions, departments and individuals, and therefore bibliometrics will have effects beyond simple measurement. [arXiv:0910.3529]
The peer review system has been traditionally challenged due to its many limitations especially for allocating funding. Bibliometric indicators may well present themselves as a complement. Objective: We analyze the relationship between peers' ratings and bibliometric indicators for Spanish researchers in the 2007 National R&D Plan for 23 research fields. We analyze peers' ratings for 2333 applications. We also gathered principal investigators' research output and impact and studied the differences between accepted and rejected applications. We used the Web of Science database and focused on th…
Nationally oriented scientific-scholarly journals are considered from a methodological-informetric viewpoint, analysing data extracted from Scimago Journal Rank based on Scopus. An operational definition is proposed of a journal's degree of national orientation based on the geographical distribution of its publishing or citing authors, and the role of international collaboration and a country's total publication output. A comprehensive analysis is presented of trends up until 2019 in national orientation and citation impact of national journals entering Scopus, extending outcomes in earlier st…
A crucial goal of funding research and development has always been to advance economic development. On this basis, a consider-able body of research undertaken with the purpose of determining what exactly constitutes economic impact and how to accurately measure that impact has been published. Numerous indicators have been used to measure economic impact, although no single indicator has been widely adapted. Based on patent data collected from Altmetric we predict patent citations through various social media features using several classification models. Patents citing a research paper implies …
Green technology is viewed as a means of creating a sustainable society and a catalyst for sustainable development by the global community. It is responsible for both the potential reduction of production waste and the reduction of carbon footprint and CO2 emissions. However, alongside with the growing popularity of green technologies, there is an emerging skepticism about their contribution to solving environmental challenges. This article focuses on three areas of eco-innovation in green technology: renewable energy, hydrogen power, and decarbonization. Our main goal is to analyze the relati…
The starting point of this paper is a desktop research assessment model that does not take properly into account the complexities of research assessment, but rather bases itself on a series of highly simplifying, questionable assumptions related to the availability, validity and evaluative significance of research performance indicators, and to funding policy criteria. The paper presents a critique of this model, and proposes alternative assessment approaches, based on an explicit evaluative framework, focusing on preconditions to performance or communication effectiveness rather than on perfo…
There is an evident and rapid trend towards the adoption of evaluation exercises for national research systems for purposes, among others, of improving allocative efficiency in public funding of individual institutions. However the desired macroeconomic aims could be compromised if internal redistribution of government resources within each research institution does not follow a consistent logic: the intended effects of national evaluation systems can result only if a "funds for quality" rule is followed at all levels of decision-making. The objective of this study is to propose a bibliometric…
Bibliometric indicators are increasingly used in support of decisions for recruitment, career advancement, rewarding and selective funding for scientists. Given the importance of the applications, bibliometricians are obligated to carry out empirical testing of the robustness of the indicators, in simulations of real contexts. In this work we compare the results of national-scale research assessments at the individual level, based on three different indexes: the h-index, g-index and "fractional scientific strength", or FSS, an indicator previously proposed by the authors. For each index, we co…
Comparing Retrieval Strategies to Capture Interdisciplinary Scientific Research: A Bibliometric Evaluation of the Integration of Neuroscience and Computer Science
We are witnessing a rapid trend towards the adoption of exercises for evaluation of national research systems, generally based on the peer review approach. They respond to two main needs: stimulating higher efficiency in research activities by public laboratories, and realizing better allocative efficiency in government funding of such institutions. However the peer review approach is typified by several limitations that raise doubts for the achievement of the ultimate objectives. In particular, subjectivity of judgment, which occurs during the step of selecting research outputs to be submitte…
How to quantify the impact of a researcher's or an institution's body of work is a matter of increasing importance to scientists, funding agencies, and hiring committees. The use of bibliometric indicators, such as the h-index or the Journal Impact Factor, have become widespread despite their known limitations. We argue that most existing bibliometric indicators are inconsistent, biased, and, worst of all, susceptible to manipulation. Here, we pursue a principled approach to the development of an indicator to quantify the scientific impact of both individual researchers and research institutio…
We discuss, at the macro-level of nations, the contribution of research funding and rate of international collaboration to research performance, with important implications for the science of science policy. In particular, we cross-correlate suitable measures of these quantities with a scientometric-based assessment of scientific success, studying both the average performance of nations and their temporal dynamics in the space defined by these variables during the last decade. We find significant differences among nations in terms of efficiency in turning (financial) input into bibliometricall…
