The emergence of large language models (LLMs), such as Generative Pre-trained Transformer 4 (GPT-4) used by ChatGPT, has profoundly impacted the academic and broader community. While these models offer numerous advantages in terms of revolutionizing work and study methods, they have also garnered significant attention due to their potential negative consequences. One example is generating academic reports or papers with little to no human contribution. Consequently, researchers have focused on developing detectors to address the misuse of LLMs. However, most existing methods prioritize achieving higher accuracy on restricted datasets, neglecting the crucial aspect of generalizability. This limitation hinders their practical application in real-life scenarios where reliability is paramount. In this paper, we present a comprehensive analysis of the impact of prompts on the text generated by LLMs and highlight the potential lack of robustness in one of the current state-of-the-art GPT detectors. To mitigate these issues concerning the misuse of LLMs in academic writing, we propose a reference-based Siamese detector named Synthetic-Siamese which takes a pair of texts, one as the inquiry and the other as the reference. Our method effectively addresses the lack of robustness of previous detectors (OpenAI detector and DetectGPT) and significantly improves the baseline performances in realistic academic writing scenarios by approximately 67% to 95%.
LLMs can Perform Multi-Dimensional Analytic Writing Assessments: A Case Study of L2 Graduate-Level Academic English Writing
Department of Linguistics & Institute for Advanced Computational Science, Stony Brook University（语言学系与高级计算科学研究所，石溪大学）；Department of Linguistics, University of Saskatchewan（语言学系，萨斯喀彻温大学）
Meow: End-to-End Outline Writing for Automatic Academic Survey
Beijing Wenge Technology Co.,Ltd（北京文格科技有限公司）；Peking University（北京大学）；Duke University（杜克大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
OverleafCopilot: Empowering Academic Writing in Overleaf with Large Language Models
School of Computer and Information Technology, Beijing Jiaotong University（北京交通大学计算机与信息技术学院）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
On the Generalization and Adaptation Ability of Machine-Generated Text Detectors in Academic Writing
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Southern University of Science and Technology（南方科技大学）；National University of Singapore (Chongqing Research Institute)（新加坡国立大学（重庆研究机构））；Fudan University（复旦大学）；Qilu University of Technology（齐鲁大学）；CISPA Helmholtz Center for Information Security（CISPA 欧洲信息安全研究所）
CoachGPT: A Scaffolding-based Academic Writing Assistant
Institute for Financial Services Analytics（金融服务分析研究所）；University of Delaware（德克萨斯大学）；School of Education（教育学院）；Department of Electrical and Computer Engineering（电气与计算机工程系）
Exposía: Teaching and Assessment of Academic Writing Skills for Research Project Proposals and Peer Feedback
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science and Hessian Center for AI (hessian.AI), Technical University of Darmstadt（普遍知识处理实验室（UKP实验室）、计算机科学系和海斯曼人工智能中心（hessian.AI）、达姆施塔特技术大学）；Department of Computer Science at Queens College, City University of New York (CUNY)（纽约城市大学皇后学院计算机科学系）
Measuring Human Involvement in AI-Generated Text: A Case Study on Academic Writing
Dept. of Information & Communication Engineering (ICE), The University of Tokyo (Japan)（信息与通信工程系（ICE），东京大学（日本））；National Institute of Informatics (Japan)（日本信息处理学会）
Large Language Models Penetration in Scholarly Writing and Peer Review
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Electronic Science and Technology of China（电子科技大学）；Wuhan University of Technology（武汉理工大学）；University of Copenhagen（哥本哈根大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）
EduResearchBench: A Hierarchical Atomic Task Decomposition Benchmark for Full-Lifecycle Educational Research
Shanghai Institute of AI for Education, East China Normal University（上海人工智能教育研究院，东华大学）；Shanghai Innovation Institute（上海创新研究院）；School of Computer Science and Technology, East China Normal University（东华大学计算机科学与技术学院）
University of Portsmouth（朴茨茅斯大学）；Varna University of Management（瓦尔纳管理大学）；Zangador Research Institute（赞加多尔研究所）；Bournemouth University（伯恩茅斯大学）
The paper looks at the role of large language models in academic knowledge creation based on a scoping review (2018 to January 2023) of how researchers have previously used the language model GPT to assist in the performance of academic knowledge creation tasks beyond data analysis. These tasks include writing, editing, reviewing, dataset creation and curation, which have been difficult to perform using earlier ML tools. Based on a synthesis of these papers, this study identifies pathways for a future academic research landscape that incorporates wider usage of large language models based on t…
Inclusivity in Large Language Models: Personality Traits and Gender Bias in Scientific Abstracts
Information Sciences Institute, University of Southern California（南加州大学信息科学学院）；Iovine and Young Academy, University of Southern California（南加州大学约万与杨学院）；In Vivo Group
From Intention to Text: AI-Supported Goal Setting in Academic Writing
Department of Media Technology and Interaction Design（媒体技术与交互设计系）；KTH Royal Institute of Technology（皇家理工学院）；Department of Digital Learning（数字学习系）；Digital Futures（数字未来）
Universität Hamburg（汉堡大学）；International Institute of Information Technology, Bangalore（班加罗尔国际信息技术学院）；City University of Hong Kong（香港城市大学）
We present the first approach to automatically building resources for academic writing. The aim is to build a writing aid system that automatically edits a text so that it better adheres to the academic style of writing. On top of existing academic resources, such as the Corpus of Contemporary American English (COCA) academic Word List, the New Academic Word List, and the Academic Collocation List, we also explore how to dynamically build such resources that would be used to automatically identify informal or non-academic words or phrases. The resources are compiled using different generic app…
Delving into LLM-assisted writing in biomedical publications through excess vocabulary
Hertie Institute for AI in Brain Health, University of Tübingen, Germany（人工智能与脑健康赫尔蒂研究所、图宾根大学，德国）；Northwestern University, Evanston, Illinois, USA（西北大学，埃文斯顿，伊利诺伊州，美国）
Tohoku University（东北大学）；Langsmith Inc.（Langsmith公司）；Yahoo Japan Corporation（雅虎日本公司）；RIKEN（理化学研究所）；Octanove Labs LLC（Octanove实验室有限责任公司）
The writing process consists of several stages such as drafting, revising, editing, and proofreading. Studies on writing assistance, such as grammatical error correction (GEC), have mainly focused on sentence editing and proofreading, where surface-level issues such as typographical, spelling, or grammatical errors should be corrected. We broaden this focus to include the earlier revising stage, where sentences require adjustment to the information included or major rewriting and propose Sentence-level Revision (SentRev) as a new writing assistance task. Well-performing systems in this task ca…
Smaller, Younger, and More Impactful: How AI-Assisted Writing Transforms Research Teams
School of Information, University of Texas at Austin（德克萨斯大学奥斯汀分校信息学院）；National Science Library, Chinese Academy of Sciences（中国科学院国家科学图书馆）；Department of Information Resources Management, School of Economics and Management, University of Chinese Academy of Sciences（中国科学院大学经济管理学院信息资源管理系）；Department of Information Management, Peking University（北京大学信息管理系）；Institute of Big Data, Fudan University（复旦大学大数据研究院）；Institute for Global Public Policy, Fudan University（复旦大学全球公共政策研究院）；Faculty of Finance, City University of Macau（澳门城市大学金融学院）
The strength with which a statement is made can have a significant impact on the audience. For example, international relations can be strained by how the media in one country describes an event in another; and papers can be rejected because they overstate or understate their findings. It is thus important to understand the effects of statement strength. A first step is to be able to distinguish between strong and weak statements. However, even this problem is understudied, partly due to a lack of data. Since strength is inherently relative, revisions of texts that make claims are a natural so…
