TQA-Bench：评估大语言模型在多表问答中的表现
TQA-Bench: Evaluating LLMs for Multi-Table Question Answering
AI总结：
提出TQA-Bench基准，通过长上下文多表问答任务评估LLM，揭示其在复杂数据驱动环境中的挑战与机遇。
摘要：
大语言模型（LLMs）的进步为复杂的多模态数据管理任务带来了巨大机遇，尤其是在涉及复杂多表关系数据的问答（QA）中。尽管取得了显著进展，但由于分析关系数据结构模态的固有复杂性以及序列化表格数据可能的大规模性，系统评估LLMs在多表QA上的表现仍然是一个关键挑战。现有基准主要关注单表QA，未能捕捉金融、医疗和电子商务等真实世界领域中多个关系表之间连接的复杂性。我们提出了TQA-Bench，一个基于真实世界公共数据集的长上下文分析型多表QA基准，具有灵活的采样机制，可变化上下文长度（8K--64K tokens）和符号扩展，以评估超越检索和模式匹配的推理能力。我们系统评估了一系列参数规模从20亿到6710亿的LLMs。大量实验揭示了LLMs在多表QA中的关键性能洞察，突出了推进其在复杂数据驱动环境中应用的挑战和机遇。
英文摘要：
The advance of large language models (LLMs) has unlocked great opportunities in complex multi-modal data management tasks, particularly in question answering (QA) over complicated multi-table relational data. Despite significant progress, systematically evaluating LLMs on multi-table QA remains a critical challenge due to the inherent complexity of analyzing the modality of relational data structures and the potentially large scale of serialized tabular data. Existing benchmarks primarily focus on single-table QA, failing to capture the intricacies of connections across multiple relational tables, as required in real-world domains such as finance, healthcare, and e-commerce. We present TQA-Bench, a long-context analytical multi-table QA benchmark derived from real-world public datasets, with a flexible sampling mechanism that varies context length (8K--64K tokens) and symbolic extensions for assessing reasoning beyond retrieval and pattern matching. We systematically evaluate a set of LLMs spanning model scales from 2 billion to 671 billion parameters. Our extensive experiments reveal critical insights into the performance of LLMs in multi-table QA, highlighting both challenges and opportunities for advancing their application in complex, data-driven environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
