We introduce RealKIE, a benchmark of five challenging datasets aimed at advancing key information extraction methods, with an emphasis on enterprise applications. The datasets include a diverse range of documents including SEC S1 Filings, US Non-disclosure Agreements, UK Charity Reports, FCC Invoices, and Resource Contracts. Each presents unique challenges: poor text serialization, sparse annotations in long documents, and complex tabular layouts. These datasets provide a realistic testing ground for key information extraction tasks like investment analysis and contract analysis. In addition to presenting these datasets, we offer an in-depth description of the annotation process, document processing techniques, and baseline modeling approaches. This contribution facilitates the development of NLP models capable of handling practical challenges and supports further research into information extraction technologies applicable to industry-specific problems. The annotated data, OCR outputs, and code to reproduce baselines are available to download at https://indicodatasolutions.github.io/RealKIE/.
CEntRE: A paragraph-level Chinese dataset for Relation Extraction among Enterprises
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Henan University（河南大学）
ICDAR 2023 Competition on Structured Text Extraction from Visually-Rich Document Images
Huazhong University of Science and Technology（华中科技大学）；Baidu Inc.（百度公司）；Tencent YouTu Lab（腾讯优图实验室）；Peking University（北京大学）；Harbin Institute of Technology（哈尔滨工业大学）；CAS（中国科学院）
KH-FUNSD: A Hierarchical and Fine-Grained Layout Analysis Dataset for Low-Resource Khmer Business Document
National Engineering Research Center of Speech and Language Information Processing (NERC-SLIP) University of Science and Technology of China（国家语信息处理语音工程研究中心（NERC-SLIP）大学科学技术大学）
Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）
Information Extraction from Electricity Invoices with General-Purpose Large Language Models
Centro de Tecnologías de la Imagen (CTIM)（图像技术中心）；Instituto Universitario de Cibernética, Empresas y Sociedad (IUCES)（大学网络研究所、企业与社会）；University of Las Palmas de Gran Canaria（大加那利岛大学）
