Causality is fundamental in human cognition and has drawn attention in diverse research fields. With growing volumes of textual data, discerning causalities within text data is crucial, and causal text mining plays a pivotal role in extracting meaningful patterns. This study conducts comprehensive evaluations of ChatGPT's causal text mining capabilities. Firstly, we introduce a benchmark that extends beyond general English datasets, including domain-specific and non-English datasets. We also provide an evaluation framework to ensure fair comparisons between ChatGPT and previous approaches. Finally, our analysis outlines the limitations and future challenges in employing ChatGPT for causal text mining. Specifically, our analysis reveals that ChatGPT serves as a good starting point for various datasets. However, when equipped with a sufficient amount of training data, previous models still surpass ChatGPT's performance. Additionally, ChatGPT suffers from the tendency to falsely recognize non-causal sequences as causal sequences. These issues become even more pronounced with advanced versions of the model, such as GPT-4. In addition, we highlight the constraints of ChatGPT in handling complex causality types, including both intra/inter-sentential and implicit causality. The model also faces challenges with effectively leveraging in-context learning and domain adaptation. We release our code to support further research and development in this field.
ChatGPT Chemistry Assistant for Text Mining and Prediction of MOF Synthesis
University of California, Berkeley（加州大学伯克利分校）；Kavli Energy Nanoscience Institute, University of California, Berkeley（加州大学伯克利分校卡夫利能源纳米科学研究所）；King Abdulaziz City for Science and Technology（阿卜杜勒阿齐兹国王科技城）
Detection of ChatGPT Fake Science with the xFakeSci Learning Algorithm
State University of New York at Binghamton（纽约州立大学宾汉姆顿分校）；Sano Centre for Computational Medicine（萨诺计算医学中心）；Hefei University of Technology（合肥工业大学）；China Electronics Data Industry Group（中国电子数据产业集团）
Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification
Mass General Brigham, Harvard Medical School（麻省总医院布莱根，哈佛医学院）；Brigham and Women’s Hospital/Dana-Farber Cancer Institute（布莱根妇女医院/丹娜-法伯癌症研究所）；Technische Universität Darmstadt（达姆施塔特工业大学）；Maastricht University（马斯特里赫特大学）
Efficient argument classification with compact language models and ChatGPT-4 refinements
Institute of Electronics（电子学研究所）；AGH University of Krakow（克拉科夫AGH大学）；Center for Collective Intelligence（集体智能中心）；Massachusetts Institute of Technology（麻省理工学院）；Faculty of Humanities（人文学院）
Empirical evidence of Large Language Model's influence on human spoken communication
Center for Humans and Machines（人类与机器中心）；Max-Planck Institute for Human Development（人类发展马克斯·普朗克研究所）；Center for Adaptive Rationality（适应性理性中心）；TUD Dresden University of Technology（德累斯顿技术大学）；Department of Business Analytics and Decision Science（商业分析与决策科学系）；Vienna University of Economics and Business（维也纳经济与商业大学）
Prompt4Vis: Prompting Large Language Models with Example Mining and Schema Filtering for Tabular Data Visualization
The Hong Kong Polytechnic University（香港理工大学）；University of Chinese Academy of Sciences（中国科学院大学）；The Hong Kong University of Science and Technology（香港科技大学）；WeBank Co., Ltd（微众银行）；University of Liverpool（利物浦大学）
针对自然语言转数据可视化查询性能不足的问题，提出基于大语言模型与上下文学习的 Prompt4Vis 框架，通过多目标示例挖掘和模式过滤模块显著提升了生成性能，在 NVBench 数据集上大幅超越现有 SOTA 方法。
NepaliGPT: A Generative Language Model for the Nepali Language
Technological University Dublin（都柏林技术大学）；Institute of Engineering（工程学院）；Kathmandu Engineering College（加德满都工程学院）；Fanshawe College（Fanshawe学院）；Lambton College（Lambton学院）
BioPars: A Pretrained Biomedical Large Language Model for Persian Biomedical Text Mining
Department of Computer Science, Faculty of Education, University of Kufa（计算机科学系，教育学院，科菲大学）；Department of Computer Science, Florida State University（计算机科学系，佛罗里达州立大学）；Department of Computer Engineering, Velayat University（计算机工程系，瓦莱yat大学）；Department of Mechanical Engineering, Amirkabir University of Technology（机械工程系，阿米卡比尔技术大学）；Department of Computer Engineering, Ferdowsi University of Mashhad（计算机工程系，法尔德大学）
The Causal News Corpus: Annotating Causal Relations in Event Sentences from News
National University of Singapore（新加坡国立大学）；Koc University（科奇大学）；Rijksuniversiteit Groningen（格罗宁根大学）；Radboud University（拉德堡德大学）；National Institute of Japanese Literature（日本文学研究所）；Birmingham City University（伯明翰城市大学）；Instituto Politécnico Nacional（国立理工学院）；Mersin University（梅尔辛大学）；University of East Anglia（东英吉利大学）；ETH Zürich（苏黎世联邦理工学院）
Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机技术研究所）；Computer Science Department, University of California, Los Angeles（加州大学洛杉矶分校计算机科学系）；The MOE Key Laboratory of Computational Linguistics, Peking University（北京大学计算语言学教育部重点实验室）
CausalChaos! Dataset for Comprehensive Causal Action Question Answering Over Longer Causal Chains Grounded in Dynamic Visual Scenes
Institute of High-Performance Computing, Agency for Science, Technology and Research, Singapore（高性能计算研究所，科技研究局，新加坡）；Nanyang Technological University（南洋理工大学）；National University of Singapore（新加坡国立大学）；Singapore Polytechnic（新加坡理工学院）
