The objective of the OSCOSS research project on "Opening Scholarly Communication in the Social Sciences" is to build a coherent collaboration environment that facilitates scholarly communication workflows of social scientists in the roles of authors, reviewers, editors and readers. This paper presents the implementation of the core of this environment: the integration of the Fidus Writer academic word processor with the Open Journal Systems (OJS) submission and review management system.
In this work, we try to answer the question of which method, peer review vs bibliometrics, better predicts the future overall scholarly impact of scientific publications. We measure the agreement between peer review evaluations of Web of Science indexed publications submitted to the first Italian research assessment exercise and long-term citations of the same publications. We do the same for an early citation-based indicator. We find that the latter shows stronger predictive power, i.e., it more reliably predicts late citations in all the disciplinary areas examined, and for any citation time…
Our system will initially provide readers, authors and reviewers with an alternative, thus having the potential to gain wider acceptance and gradually replace the old, incoherent publication process of our journals and of others in related fields. It will make journals more "open" (in terms of reusability) that are open access already, and it has the potential to serve as an incentive for turning "closed" journals into open access ones. In this poster paper we will present the framework of the OSCOSS system and highlight the reviewer use case.
This paper aims to gain a better understanding of the perspectives of contributors to Spanish academic journals regarding open access, open peer review, and altmetrics. It also explores how age, gender, professional experience, career history, and perception and use of social media influence authors opinions toward these developments in scholarly publishing. A sample of contributors (n-1254) to Spanish academic journals was invited to participate in a survey about the aforementioned topics. The response rate was 24 per cent (n-295). Contributors to Spanish scholarly journals hold a favourable …
This study examines whether there is any evidence of bias in two areas of common critique of open, non-anonymous peer review - and used in the post-publication, peer review system operated by the open-access scholarly publishing platform F1000Research. First, is there evidence of bias where a reviewer based in a specific country assesses the work of an author also based in the same country? Second, are reviewers influenced by being able to see the comments and know the origins of previous reviewer? Methods: Scrutinising the open peer review comments published on F1000Research, we assess the ex…
Review articles are a means to structure state-of-the-art literature and to organize the growing number of scholarly publications. However, review articles are suffering from numerous limitations, weakening the impact the articles could potentially have. A key limitation is the inability of machines to access and process knowledge presented within review articles. In this work, we present SmartReviews, a review authoring and publishing tool, specifically addressing the limitations of review articles. The tool enables community-based authoring of living articles, leveraging a scholarly knowledg…
In order to better understand the effect of social media in the dissemination of scholarly articles, employing the daily updated referral data of 110 PeerJ articles collected over a period of 345 days, we analyze the relationship between social media attention and article visitors directed by social media. Our results show that social media presence of PeerJ articles is high. About 68.18% of the papers receive at least one tweet from Twitter accounts other than @PeerJ, the official account of the journal. Social media attention increases the dissemination of scholarly articles. Altmetrics coul…
Having always been at the forefront of information management and open access, High-Energy Physics (HEP) proves to be an ideal test-bed for innovations in scholarly communication including new information and communication technologies. Three selected topics of scholarly communication in High-Energy Physics are presented here: A new open access business model, SCOAP3, a world-wide sponsoring consortium for peer-reviewed HEP literature; the design, development and deployment of an e-infrastructure for information management; and the emerging debate on long-term preservation, re-use and (open) a…
Scholarly communication is today immersed in publish or perish culture that propels noncooperative behaviour in the sense of strategic games played by researchers. Here we introduce and describe a blockchain based platform for decentralized scholarly communication. The design of the platform rests on community driven publishing reviewing processes and implements incentives that promote cooperative user behaviour. Key to achieve cooperation in blockchain based scholarly communication is to transform a static research paper into a modifiable research paper under continuous peer review process. W…
The OSCOSS project (Opening Scholarly Communication in Social Sciences), which will be outlined, aims at providing integrated support for all steps of the scholarly communication process. Incl. collaborative writing of a scientific paper, collecting data related to existing publications, interpreting and including data in a paper, submitting the paper for peer review, reviewing the paper, publishing an article, and, finally, facilitating its consumption by readers. The OSCOSS project will support this process considering in particular the perspective of three main actors detailed in the use ca…
Scholarly article impact reflects the significance of academic output recognised by academic peers, and it often plays a crucial role in assessing the scientific achievements of researchers, teams, institutions and countries. It is also used for addressing various needs in the academic and scientific arena, such as recruitment decisions, promotions, and funding allocations. This article provides a comprehensive review of recent progresses related to article impact assessment and prediction. The~review starts by sharing some insight into the article impact research and outlines current research…
Social media metrics - commonly coined as "altmetrics" - have been heralded as great democratizers of science, providing broader and timelier indicators of impact than citations. These metrics come from a range of sources, including Twitter, blogs, social reference managers, post-publication peer review, and other social media platforms. Social media metrics have begun to be used as indicators of scientific impact, yet the theoretical foundation, empirical validity, and extent of use of platforms underlying these metrics lack thorough treatment in the literature. This editorial provides an ove…
Peer review is a process designed to produce a fair assessment of research quality before the publication of scholarly work in a journal. Demographics, nepotism, and seniority have been all shown to affect reviewer behavior suggesting the most common, single-blind review method (or the less common open review method) might be biased. A survey of current research indicates that double-blind review offers a solution to many biases stemming from author's gender, seniority, or location without imposing any significant downsides.
Science depends on a communication system, and today that is largely provided by digital technologies such as the internet and web. Despite that digital technologies provide the infrastructure for that communication system, peer-reviewed journals continue to mimic workflows and processes from the print era. This paper focuses on one artifact from the print era, the journal issue, and describes how this artifact has been detrimental to the communication of science and therefore to science itself. To replace the journal issue, this paper argues that scholarly publishing and journals could more f…
Automated scholarly paper review: Concepts, technologies, and challenges
Xiamen University（厦门大学）；Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism（文化和旅游部闽台非物质文化遗产数字化保护与智能处理重点实验室）
The general pupose of the scholarly communication process is to support the creation and dissemination of ideas within the scientific community. At a finer granularity, there exists multiple stages which, when confronted by a member of the community, have different requirements and therefore different solutions. In order to take a researcher's idea from an initial inspiration to a community resource, the scholarly communication infrastructure may be required to 1) provide a scientist initial seed ideas; 2) form a team of well suited collaborators; 3) located the most appropriate venue to publi…
We present the problem of finding comparable researchers for any given researcher. This problem has many motivations. Firstly, know thyself. The answers of where we stand among research community and who we are most alike may not be easily found by existing evaluations of ones' research mainly based on citation counts. Secondly, there are many situations where one needs to find comparable researchers e.g., for reviewing peers, constructing programming committees or compiling teams for grants. It is often done through an ad hoc and informal basis. Utilizing the large scale scholarly data access…
At this time, developers of research information systems are experimenting with new tools for research outputs usage that can expand the open access to research. These tools allow researchers to record research as annotations, nanopublications or other micro research outputs and link them by scientific relationships. If these micro outputs and relationships are shared by their creators publicly, these actions can initiate direct scholarly communication between the creators and the authors of the used research outputs. Such direct communication takes place while researchers are manipulating and…
The purpose of scholarly peer review is to evaluate the quality of scientific manuscripts. However, study after study demonstrates that peer review neither effectively nor reliably assesses research quality. Empirical standards attempt to address this problem by modelling a scientific community's expectations for each kind of empirical study conducted in that community. This should enhance not only the quality of research but also the reliability and predictability of peer review, as scientists adopt the standards in both their researcher and reviewer roles. However, these improvements depend …
Stakeholders in research and scientific publishing are gradually joining the Open-Access (OA) movement, which is gaining momentum to become nowadays at the heart of scientific policies in high-income countries. The rise of OA generates profound changes in the chain of production and dissemination of knowledge. Free access to peer-reviewed research methods and results has contributed to the dynamics of science observed in recent years. The modes of publication and access have also evolved; the classic model, based on journal subscriptions is gradually giving way to new economic models that have…
Citation measures, and newer altmetric measures such as downloads are now commonly used to inform personnel decisions. How well do or can these measures measure or predict the past, current of future scholarly performance of an individual? Using data from the Smithsonian/NASA Astrophysics Data System we analyze the publication, citation, download, and distinction histories of a cohort of 922 individuals who received a U.S. PhD in astronomy in the period 1972-1976. By examining the same and different measures at the same and different times for the same individuals we are able to show the capab…
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）
Keeping up with the research literature plays an important role in the workflow of scientists - allowing them to understand a field, formulate the problems they focus on, and develop the solutions that they contribute, which in turn shape the nature of the discipline. In this paper, we examine the literature review practices of data scientists. Data science represents a field seeing an exponential rise in papers, and increasingly drawing on and being applied in numerous diverse disciplines. Recent efforts have seen the development of several tools intended to help data scientists cope with a d…
MOPRD: A multidisciplinary open peer review dataset
School of Informatics, Xiamen University（厦门大学信息学院）；Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism（文化和旅游部闽台非物质文化遗产数字化保护与智能处理重点实验室）；College of Foreign Languages and Cultures, Xiamen University（厦门大学外文学院）
Large language models for automated scholarly paper review: A survey
School of Computer Science and Engineering, Guangzhou Institute of Science and Technology（计算机科学与工程学院，科学与技术研究院）；School of Economics and Management, Guizhou Normal University（经济学与管理学院，贵州师范大学）；School of Artificial Intelligence, Guangzhou Institute of Science and Technology（人工智能学院，科学与技术研究院）
FMMD: A multimodal multidisciplinary dataset of open peer reviews from F1000Research
School of Computer Science and Engineering, Guangzhou Institute of Science and Technology（计算机科学与工程学院，广州科学与技术研究所）；College of Foreign Languages and Cultures, Xiamen University（外语学院，厦门大学）；Science and Education Evaluation Lab, Guangzhou Institute of Science and Technology（科学与教育评估实验室，广州科学与技术研究所）
Pre-review to Peer review: Pitfalls of Automating Reviews using Large Language Models
AllSci Corp（AllSci公司）；Sunwater Capital（Sunwater资本）；Kellogg School of Management, Northwestern University（西北大学凯洛格管理学院）；Dept. of Computer Science, Northern Illinois University（北伊利诺伊大学计算机科学系）；Institute of Foundation Models, MBZUAI（基础模型研究所，MBZUAI）
Preprint is a version of a scientific paper that is publicly distributed preceding formal peer review. Since the launch of arXiv in 1991, preprints have been increasingly distributed over the Internet as opposed to paper copies. It allows open online access to disseminate the original research within a few days, often at a very low operating cost. This work overviews how preprint has been evolving and impacting the research community over the past thirty years alongside the growth of the Web. In this work, we first report that the number of preprints has exponentially increased 63 times in 30 …
Today, it is not clear how the impact of research on other areas of society than science should be measured. While peer review and bibliometrics have become standard methods for measuring the impact of research in science, there is not yet an accepted framework within which to measure societal impact. Alternative metrics (called altmetrics to distinguish them from bibliometrics) are considered an interesting option for assessing the societal impact of research, as they offer new ways to measure (public) engagement with research output. Altmetrics is a term to describe web-based metrics for the…
Purpose: Science is subject to a normative structure that includes how the contributions and interactions between scientists are rewarded. Authorship and citations have been the key elements within the reward system of science, whereas acknowledgements, despite being a well-established element in scholarly communication, have not received the same attention. This paper aims to put forward the bearing of acknowledgements in the humanities to bring to the foreground contributions and interactions that, otherwise, would remain invisible through traditional indicators of research performance. Desi…
