LongHisDoc: A Comprehensive Benchmark for Chinese Long Historical Document Understanding
School of Electronic and Information Engineering, South China University of Technology(华南理工大学电子与信息工程学院);College of Literature and Journalism, Sichuan University(四川大学文学与新闻学院)
Chinese historical documents are invaluable cultural treasures, but their complexity poses significant comprehension challenges for non-specialists. Despite the success of Large Language Models (LLMs) and Large Vision-Language Models (LVLMs) in modern document tasks, their capabilities in Long (full-book) Historical Document understanding remain largely unexplored. To bridge this gap, we introduce LongHisDoc, the first benchmark tailored for long historical document understanding. LongHisDoc comprises 1,012 expert-annotated QA pairs derived from 101 PDF documents, with an average of 85.1 pages and 42,776.4 tokens per document. It spans 10 categories and features four QA types: single-page, cross-page, multi-hop, and unanswerable. Answering these questions requires integrating multimodal evidence from text, layout, and visual elements. Our evaluation reveals that the top-performing LVLM and LLM achieve only 62.9% and 69.9% accuracy, respectively, highlighting a substantial performance gap. Furthermore, we assess various modal retrievers and the end-to-end RAG performance to provide references for Retrieval-Augmented Generation (RAG) systems. We believe that LongHisDoc serves as a comprehensive benchmark to catalyze future research in full-book historical document understanding. The benchmark and evaluation code are available at https://huggingface.co/datasets/qweq12433454/LongHisDoc and https://github.com/SCUT-DLVCLab/LongHisDoc.
A GAN and LLM-Driven Data Augmentation Framework for Dynamic Linguistic Pattern Modeling in Chinese Sarcasm Detection
School of Cyber Science and Engineering, Sichuan University(四川大学网络空间安全学院);State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University(浙江大学流体动力与机电系统国家重点实验室);Law School, Sichuan University(四川大学法学院)
CCiV: A Benchmark for Structure, Rhythm and Quality in LLM-Generated Chinese \textit{Ci} Poetry
School of Computer Science and Technology, East China Normal University, China(东华师范大学计算机科学与技术学院);Shanghai Institute of Artificial Intelligence for Education, East China Normal University, China(东华师范大学教育人工智能研究所);Department of Chinese Language and Literature, East China Normal University, China(东华师范大学中文语言文学系)
CulMind: Benchmarking Multimodal Understanding and Reasoning in Chinese Cultural Heritage
University of International Relations(国际关系学院);Kedge Business School(凯致商学院);Peking University(北京大学);Tsinghua University(清华大学);Beijing University of Posts and Telecommunications(北京邮电大学);Capital Normal University(首都师范大学)
Evaluation of Google Translate for Mandarin Chinese translation using sentiment and semantic analysis
School of Mathematics and Statistics, UNSW Sydney(新南威尔士大学悉尼分校数学与统计学院);Pingala Institute(平加拉研究所);Centre for Artificial Intelligence and Innovation(人工智能与创新中心)
Ground Truth Generation for Multilingual Historical NLP using LLMs
ARTFL Project, Romance Languages and Literatures, University of Chicago(ARTFL项目, Romance语言与文学系,芝加哥大学);Department of History, University of Chicago(历史系,芝加哥大学);Libraries and School of Information Studies, Purdue University(图书馆与信息科学学院,普渡大学)
Enhancing Multimodal Large Language Models for Ancient Chinese Character Evolution Analysis via Glyph-Driven Fine-Tuning
College of Computer Science and Technology, Jilin University, China(吉林大学计算机科学与技术学院);Key Laboratory of Ancient Chinese Script, Culture Relics and Artificial Intelligence, Jilin University, China(吉林大学古文字、文物与人工智能重点实验室);School of Artificial Intelligence, Jilin University, China(吉林大学人工智能学院);School of Archaeology, Jilin University, China(吉林大学考古学院)
School of Systems Science, Beijing Normal University(北京师范大学系统科学学院);New England Complex Systems Institute(新英格兰复杂系统研究所);Brandeis University(布兰迪斯大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Business College, Shanghai Normal University(上海师范大学商学院)
A Phonemically Comprehensive, ASCII-Only Romanization Scheme for Thai and Lao: Systematic Cross-Lingual Correspondence and Chinese-User-Friendly Design
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
Neo-Classic: A Benchmark for Evaluating Linguistic-Aesthetic Reasoning in Classical Chinese Poetry
Shanghai Jiao Tong University(上海交通大学);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)