Evaluation frameworks for text summarization have evolved in terms of both domain coverage and metrics. However, existing benchmarks still lack domain-specific assessment criteria, remain predominantly English-centric, and face challenges with human annotation due to the complexity of reasoning. To address these, we introduce MSumBench, which provides a multi-dimensional, multi-domain evaluation of summarization in English and Chinese. It also incorporates specialized assessment criteria for each domain and leverages a multi-agent debate system to enhance annotation quality. By evaluating eight modern summarization models, we discover distinct performance patterns across domains and languages. We further examine large language models as summary evaluators, analyzing the correlation between their evaluation and summarization capabilities, and uncovering systematic bias in their assessment of self-generated summaries. Our benchmark dataset is publicly available at https://github.com/DISL-Lab/MSumBench.
Think Carefully and Check Again! Meta-Generation Unlocking LLMs for Low-Resource Cross-Lingual Summarization
University of California, San Diego（加利福尼亚大学圣迭戈分校）；Nanyang Technological University（南洋理工大学）；National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）
Evaluating the Evaluator: Summarization Metrics and LLM-Judges beyond English
HiTZ Center, University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU）HiTZ中心）；University of Alicante (UA)（阿利坎特大学（UA））；GOI Institute, Basque Summer University (UEU)（巴斯克夏季大学（UEU）GOI研究所）
NTS-CoT: Mitigating Hallucinations in LLM-based News Timeline Summarization with Chain-of-Thought Reasoning
Central South University（中南大学）；Tsinghua University（清华大学）；Nanjing University（南京大学）；Suzhou Aerospace Information Research Institute（苏州空天信息研究院）；McGill University（麦吉尔大学）
Reasoning or Not? A Comprehensive Evaluation of Reasoning LLMs for Dialogue Summarization
Faculty of Applied Sciences, Macao Polytechnic University（澳门理工大学应用科学学院）；Computer Science and Communication Research Center, School of Technology and Management, Polytechnic of Leiria（莱里亚理工学院技术与管理学院计算机科学与通信研究中心）；Institute of International Language Services Studies, Macau Millennium College（澳门 millennium 学院国际语言服务研究学院）；CISUC, Department of Informatics Engineering, University of Coimbra（科英布拉大学信息工程系 CISUC）
A Guide To Effectively Leveraging LLMs for Low-Resource Text Summarization: Data Augmentation and Semi-supervised Approaches
Cheriton School of Computer Science, University of Waterloo（滑铁卢大学切尔顿计算机科学学院）；ServiceNow Research（ServiceNow研究院）；University of British Columbia（不列颠哥伦比亚大学）
FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs
Vectara, Inc.（Vectara公司）；Iowa State University（爱荷华州立大学）；University of Southern California（南加利福尼亚大学）；Entropy Technologies（熵技术公司）；University of Waterloo（滑铁卢大学）；University of Wisconsin, Madison（威斯康星大学麦迪逊分校）
Don't Believe Everything You Read: Enhancing Summarization Interpretability through Automatic Identification of Hallucinations in Large Language Models
Saarland University（萨尔大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；University of Cambridge（剑桥大学）；University of Edinburgh（爱丁堡大学）；Zhejiang University（浙江大学）；Tencent YouTu Lab（腾讯优图实验室）
LLM aided semi-supervision for Extractive Dialog Summarization
Amsterdam UMC（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；Amsterdam Public Health（阿姆斯特丹公共卫生研究所）；University of Waterloo（滑铁卢大学）；ServiceNow Research（ServiceNow研究院）；University of British Columbia（不列颠哥伦比亚大学）
A Training-Free Mixture-of-Agents Framework for Multi-Document Summarization using LLMs and Knowledge Graphs
Faculty of Artificial Intelligence and Data Science, Phenikaa University（人工智能与数据科学学院，泛尼克大学）；VNPT AI, VNPT Group（VNPT AI，VNPT集团）；MobiFone Research and Development Center, MobiFone Corporation（MobiFone研发与开发中心，MobiFone公司）；Business AI Lab, Faculty of Data Science and Artificial Intelligence, National Economics University, College of Technology（商业人工智能实验室，数据科学与人工智能学院，国家经济大学，技术学院）
SCURank: Ranking Multiple Candidate Summaries with Summary Content Units for Enhanced Summarization
Department of Computer Science and Information Engineering, National Cheng Kung University（国立成功大学计算机科学与资讯工程系）；Artificial Intelligence Research Center, Chang Gung University（长庚大学人工智能研究中心）；Department of Artificial Intelligence, Chang Gung University（长庚大学人工智能系）；Department of Computer Science, National Tsing Hua University（国立清华大学计算机科学系）
