DeepTable：用于层级表格理解的结构注意力偏置与树路径编码
DeepTable: Structural Attention Biases and Tree Path Encoding for Hierarchical Table Understanding
浏览论文内容
中文总结 AI 辅助
针对LLM表格理解中层级结构信息丢失的问题，提出DeepTable，通过结构注意力偏置和树路径编码显式建模行列层级关系，在三个问答基准上显著提升TableLoRA性能。
中文摘要 AI 辅助
大型语言模型（LLM）在表格理解方面展现了强大的性能。然而，它们通常将表格内容和表头作为线性化的词元序列进行处理。这种表示削弱了由多级行和列表头编码的二维及层级结构关系。现有的参数高效微调方法融入了基本的行和列信息，但并未显式捕获层级表头所引发的丰富结构依赖。我们提出了DeepTable，一种面向LLM表格理解的结构感知方法。DeepTable包含两个互补的组件。结构注意力偏置（SAB）在注意力对数中引入可学习的偏置，以显式表示表格词元对是否共享同一行或列。树路径编码（TPE）使用其行和列表头的祖先路径来表示每个表格词元，保留其在多级表格结构中的位置。我们将DeepTable与TableLoRA（He等人，2025）集成，以将结构信息注入参数高效的适配中。在三个LLM骨干网络上，DeepTable在三个表格问答基准上持续提升了相应的TableLoRA基线，在HiTab上平均提升7.42个百分点，在WikiTQ上提升3.23个百分点，在FeTaQA上提升2.01个BLEU分数。这些结果证明了所提出的结构偏置在不同LLM骨干网络上的有效性。
英文摘要
Large language models (LLMs) have demonstrated strong performance in table understanding. However, they typically process table content and headers as linearized token sequences. This representation weakens the two-dimensional and hierarchical structural relationships encoded by multi-level row and column headers. Existing parameter-efficient fine-tuning methods incorporate basic row and column information but do not explicitly capture the rich structural dependencies induced by hierarchical table headers. We propose DeepTable, a structure-aware approach for table understanding with LLMs. DeepTable comprises two complementary components. Structural Attention Bias (SAB) introduces learnable biases into the attention logits to explicitly represent whether pairs of table tokens share the same row or column. Tree Path Encoding (TPE) represents each table token using the ancestor paths of its row and column headers, preserving its position within the multi-level table structure. We integrate DeepTable with TableLoRA (He et al., 2025) to inject structural information into parameter-efficient adaptation. Across three LLM backbones, DeepTable consistently improves the corresponding TableLoRA baselines on three table question answering benchmarks, achieving average gains of 7.42 points on HiTab, 3.23 points on WikiTQ, and 2.01 BLEU points on FeTaQA. These results demonstrate the effectiveness of the proposed structural biases across different LLM backbones.
发表机构
- National Yang Ming Chiao Tung University（国立阳明交通大学）
机构由 AI 辅助整理，请以论文原文为准。
