M3TQA:大规模多语言多任务表格问答
M3TQA: Massively Multilingual Multitask Table Question Answering
AI总结:
本文提出M3TQA基准,覆盖97种语言的多任务表格问答,通过LLM翻译流程构建并验证合成数据可显著提升低资源语言的跨语言表格推理性能。
摘要:
表格数据是现实世界信息系统的基础组成部分,然而大多数表格理解研究仍局限于英语,使得多语言理解能力在很大程度上未得到充分探索。现有的多语言表格基准存在地理语言分布不均衡的问题——过度代表某些语言,并且缺乏足够规模以支持严格的跨语言分析。为解决这些局限性,我们引入了一个面向大规模多语言多任务表格问答的综合框架,其核心是m3TQA-Instruct,这是一个跨越97种语言、覆盖不同语系(包括代表性不足和低资源语言)的大规模基准。我们通过整理50个中英文真实世界表格来构建m3TQA,然后应用由DeepSeek和GPT-4o驱动的稳健六步基于大语言模型的翻译流程,实现了高翻译保真度,经回译验证的中位BLEU分数为60.19。该基准包含2,916个经过专业标注的问答对,涵盖四项任务,旨在评估精细的表格推理能力。在最先进的大语言模型上的实验揭示了对跨语言泛化的关键见解,表明合成生成的未标注问答数据可以显著提升性能,尤其是对于低资源语言。M3T-Bench为多语言表格理解建立了新标准,既提供了具有挑战性的评估平台,也为未来研究提供了可扩展的方法论。
英文摘要:
Tabular data is a fundamental component of real-world information systems, yet most research in table understanding remains confined to English, leaving multilingual comprehension significantly underexplored. Existing multilingual table benchmarks suffer from geolinguistic imbalance - overrepresenting certain languages and lacking sufficient scale for rigorous cross-lingual analysis. To address these limitations, we introduce a comprehensive framework for massively multilingual multitask table question answering, featuring m3TQA-Instruct, a large-scale benchmark spanning 97 languages across diverse language families, including underrepresented and low-resource languages. We construct m3TQA by curating 50 real-world tables in Chinese and English, then applying a robust six-step LLM-based translation pipeline powered by DeepSeek and GPT-4o, achieving high translation fidelity with a median BLEU score of 60.19 as validated through back-translation. The benchmark includes 2,916 professionally annotated question-answering pairs across four tasks designed to evaluate nuanced table reasoning capabilities. Experiments on state-of-the-art LLMs reveal critical insights into cross-lingual generalization, demonstrating that synthetically generated, unannotated QA data can significantly boost performance, particularly for low-resource languages. M3T-Bench establishes a new standard for multilingual table understanding, providing both a challenging evaluation platform and a scalable methodology for future research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。