TARGET：面向生成式任务的表格检索基准测试
TARGET: Benchmarking Table Retrieval for Generative Tasks
AI总结：
针对生成式任务中合适表格检索的核心问题，提出表格检索基准TARGET，评估不同检索器性能及对下游任务影响，发现稠密嵌入检索器优于BM25，且检索性能受元数据、数据集和任务影响差异显著。
摘要：
数据领域中结构化数据十分丰富，通常对组织具有高价值，推动了数据分析与机器学习领域的重要应用。近年来，针对这类数据的表示学习与生成式模型取得进展，催生了结构化数据自然语言接口的开发，其中包括利用text-to-SQL（文本转SQL）的接口。通过检索增强生成技术，借助对话接口或智能体组件为结构化数据交互提供上下文，能够在答案的时效性、准确性和全面性方面带来显著收益。核心问题在于：我们如何为当前的分析查询或任务检索到合适的表格？为此，我们提出TARGET：一个用于评估面向生成式任务的表格检索（TAble Retrieval for GEnerative Tasks）的基准测试。借助TARGET，我们分析了不同检索器单独使用时的检索性能，以及它们对下游任务的影响。我们发现，基于稠密嵌入的检索器性能远优于BM25基线，而BM25在结构化数据检索上的效果不如其在非结构化文本检索中的表现。我们还揭示了检索器在各类元数据（如缺失表格标题）下的敏感性，并证明检索性能在不同数据集和任务间存在显著差异。TARGET可在https://target-benchmark.github.io获取。
英文摘要：
The data landscape is rich with structured data, often of high value to organizations, driving important applications in data analysis and machine learning. Recent progress in representation learning and generative models for such data has led to the development of natural language interfaces to structured data, including those leveraging text-to-SQL. Contextualizing interactions, either through conversational interfaces or agentic components, in structured data through retrieval-augmented generation can provide substantial benefits in the form of freshness, accuracy, and comprehensiveness of answers. The key question is: how do we retrieve the right table(s) for the analytical query or task at hand? To this end, we introduce TARGET: a benchmark for evaluating TAble Retrieval for GEnerative Tasks. With TARGET we analyze the retrieval performance of different retrievers in isolation, as well as their impact on downstream tasks. We find that dense embedding-based retrievers far outperform a BM25 baseline which is less effective than it is for retrieval over unstructured text. We also surface the sensitivity of retrievers across various metadata (e.g., missing table titles), and demonstrate a stark variation of retrieval performance across datasets and tasks. TARGET is available at https://target-benchmark.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
