Recent neural language models have taken a significant step forward in producing remarkably controllable, fluent, and grammatical text. Although studies have found that AI-generated text is not distinguishable from human-written text for crowd-sourcing workers, there still exist errors in AI-generated text which are even subtler and harder to spot. We primarily focus on the scenario in which scientific AI writing assistant is deeply involved. First, we construct a feature description framework to distinguish between AI-generated text and human-written text from syntax, semantics, and pragmatics based on the human evaluation. Then we utilize the features, i.e., writing style, coherence, consistency, and argument logistics, from the proposed framework to analyze two types of content. Finally, we adopt several publicly available methods to investigate the gap of between AI-generated scientific text and human-written scientific text by AI-generated scientific text detection models. The results suggest that while AI has the potential to generate scientific content that is as accurate as human-written content, there is still a gap in terms of depth and overall quality. The AI-generated scientific content is more likely to contain errors in factual issues. We find that there exists a "writing style" gap between AI-generated scientific text and human-written scientific text. Based on the analysis result, we summarize a series of model-agnostic and distribution-agnostic features for detection tasks in other domains. Findings in this paper contribute to guiding the optimization of AI models to produce high-quality content and addressing related ethical and security concerns.
Semantic Differentiation in Speech Emotion Recognition: Insights from Descriptive and Expressive Speech Roles
School of Electrical Engineering and Computer Science, University of Ottawa(电气工程与计算机科学学院,渥太华大学);School of Psychology, University of Ottawa(心理学学院,渥太华大学);National Research Council Canada, Ottawa, Canada(加拿大国家研究理事会,渥太华,加拿大)
Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation
University of Technology Nuremberg (UTN)(图恩大学(UTN));University of Tübingen, Tübingen AI Center(图宾根大学,图宾根人工智能中心);TIB Leibniz Information Centre for Science and Technology(莱比锡信息科学与技术研究中心);Austrian Research Institute for Artificial Intelligence(奥地利人工智能研究所);IT:U Interdisciplinary Transformation University Austria(奥地利 interdisciplinary transformation 大学);University of Hamburg(汉堡大学);University of Manchester(曼彻斯特大学);University of Sheffield(谢菲尔德大学);University of Aberdeen(阿伯丁大学);University of Manitoba(曼尼托巴大学)
SciSummPip: An Unsupervised Scientific Paper Summarization Pipeline
Monash University(莫纳什大学);Deakin University(迪肯大学)
学术文献处理(SDP)研讨会是为了鼓励更多的努力对科学任务的自然语言的理解。它包含三个共享任务,我们参与LongSumm共享任务。在本文中,我们描述了我们的文本摘要系统,SciSummPip,灵感来自SummPip(Zhao等人,2020),这是一个无监督的文本摘要系统的多文档在新闻领域。我们的SciSummPip包括基于转换器的语言模型SciBERT(Beltagy等人,2019)用于上下文句子表示,使用PageRank进行内容选择(Page等人,1999),具有深度和语言信息的句子图构造,句子图聚类和图内摘要生成。我们的工作不同于以前的方法,在内容选择和摘要长度约束适用于科学领域。在训练数据集和盲测试数据集上的实验结果表明了我们方法的有效性,并且我们用BERTScore(Zhang et al.,2019年a)。
Project Alexandria: Towards Freeing Scientific Knowledge from Copyright Burdens via LLMs
LAION;Open-Ψ (Open-Sci) Collective(Open-Ψ(Open-Sci)集体);L3S Research Center, Leibniz University Hannover(汉诺威莱布尼茨大学L3S研究中心);Stanford University(斯坦福大学);TIB—Leibniz Information Centre for Science and Technology(莱布尼茨科技信息中心);Juelich Supercomputing Center(于利希超级计算中心);Technical University of Munich(慕尼黑工业大学)
Fine-Tuning Large Language Models for Scientific Text Classification: A Comparative Study
Doctoral School of Applied Informatics(应用信息学博士学院);Applied Mathematics Óbuda University Budapest, Hungary(应用数学 奥布达大学 布达佩斯 匈牙利);John von Neumann Faculty of Informatics(约翰·冯·诺依曼信息学学院)
SciXGen: A Scientific Paper Dataset for Context-Aware Text Generation
The University of Tokyo(东京大学);Tokyo Institute of Technology(东京工业大学);National Institute of Advanced Industrial Science Technology(日本产业技术综合研究所)
在科学论文中生成文本不仅需要捕获给定输入中包含的内容,还需要经常获取称为\textit{context}的外部信息。我们提出了一个新的任务,即科学领域的\textbf{context-aware text generation},旨在开发语境在生成文本中的贡献,从而推动了科学文本的生成。为此,我们为Conte \textbf{X} t-Aware Text \textbf{Gen} eration(SciXGen)提出了一个新的具有挑战性的大规模\textbf{Sci}有效论文数据集,由注释良好的205,304篇论文组成,其中充分引用了广泛使用的对象(例如,表、图、算法)。我们使用最先进的技术,对我们新构建的SciXGen数据集在生成描述和段落方面的功效进行了全面的基准测试。我们的数据集和基准将公开提供,希望能促进科学文本生成研究。