Over the past decade, extensive research efforts have been dedicated to the extraction of information from textual process descriptions. Despite the remarkable progress witnessed in natural language processing (NLP), information extraction within the Business Process Management domain remains predominantly reliant on rule-based systems and machine learning methodologies. Data scarcity has so far prevented the successful application of deep learning techniques. However, the rapid progress in generative large language models (LLMs) makes it possible to solve many NLP tasks with very high quality without the need for extensive data. Therefore, we systematically investigate the potential of LLMs for extracting information from textual process descriptions, targeting the detection of process elements such as activities and actors, and relations between them. Using a heuristic algorithm, we demonstrate the suitability of the extracted information for process model generation. Based on a novel prompting strategy, we show that LLMs are able to outperform state-of-the-art machine learning approaches with absolute performance improvements of up to 8\% $F_1$ score across three different datasets. We evaluate our prompting strategy on eight different LLMs, showing it is universally applicable, while also analyzing the impact of certain prompt parts on extraction quality. The number of example texts, the specificity of definitions, and the rigour of format instructions are identified as key for improving the accuracy of extracted information. Our code, prompts, and data are publicly available.
Hierarchical Prompting Taxonomy: A Universal Evaluation Framework for Large Language Models Aligned with Human Cognitive Principles
Indian Institute of Technology Kharagpur（印度理工学院Khargapur分校）；Rochester Institute of Technology（罗切斯特理工学院）；Researcher, Fatima Fellowship（研究员，Fatima fellowship）；AI Institute, University of South Carolina（人工智能研究所，南卡罗来纳大学）；Amazon GenAI（亚马逊生成人工智能）；Stanford University（斯坦福大学）；James Silberrad Brown Center for AI, San Diego State University（詹姆斯·西伯拉德·布朗人工智能中心，圣地亚哥州立大学）
Enhancing Document-level Argument Extraction with Definition-augmented Heuristic-driven Prompting for LLMs
School of Engineering and Informatics, University of Sussex（萨塞克斯大学工程与信息学院）；International Business School Suzhou, Xi’an Jiaotong-Liverpool University（西交利物浦大学苏州国际商学院）；Management School, University of Liverpool（利物浦大学管理学院）
Leveraging Large Language Models for Automated Causal Loop Diagram Generation: Enhancing System Dynamics Modeling through Curated Prompting Techniques
MGH Institute for Technology Assessment, Harvard Medical School（哈佛医学院MGH技术评估研究所）；Melbourne Business School, University of Melbourne（墨尔本大学墨尔本商学院）；Sloan School of Management, Massachusetts Institute of Technology（麻省理工学院斯隆管理学院）
Prompting Science Report 1: Prompt Engineering is Complicated and Contingent
Generative AI Labs, The Wharton School of Business, University of Pennsylvania（宾夕法尼亚大学沃顿商学院生成式人工智能实验室）；WHU–Otto Beisheim School of Management（WHU-奥托·贝希姆管理学院）；Glowforge（Glowforge公司）
Decomposed Prompting: Probing Multilingual Linguistic Structure Knowledge in Large Language Models
Center for Information and Language Processing (CIS)（信息与语言处理中心）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；ScaDS.AI and TU Dresden（ScaDS.AI 和 梅克伦堡-前波美拉尼亚技术大学）；Department of Statistics, LMU Munich（统计学系，慕尼黑大学）；University of Maryland, College Park（马里兰大学 College Park 分校）
No One Fits All: From Fixed Prompting to Learned Routing in Multilingual LLMs
Department of Computer Science and Information Engineering, National Taiwan University, Taiwan（国立台湾大学计算机科学与资讯工程学系）；Institute of Information Science, Academia Sinica, Taiwan（台湾“中央研究院”信息科学研究所）；AI Research Center (AINTU), National Taiwan University, Taiwan（国立台湾大学人工智能研究中心）
Extracting Research Instruments from Educational Literature Using LLMs
College of Education, University of Maryland, College Park, USA（美国马里兰大学教育学院）；Annenberg Institute, Brown University, USA（布朗大学安纳伯格研究所）；College of Information, University of Maryland, College Park, USA（美国马里兰大学信息学院）
Prompting is not Enough: Exploring Knowledge Integration and Controllable Generation
University of Science and Technology of China（中国科学技术大学）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；Chinese Academy of Sciences（中国科学院）；BOSS Zhipin Career Science Lab（BOSS智联招聘科研实验室）
