面向表格或文本的参数高效抽象式问答
Parameter-Efficient Abstractive Question Answering over Tables or Text
AI总结：
本文研究在表格和文本问答中使用参数高效适配器，仅用0.7%-1.5%的额外参数即可在多个数据集上达到或超越微调模型的性能。
摘要：
信息寻求型问答系统的一个长期目标是能够对多模态上下文进行推理，并针对用户查询生成自然答案。目前，内存密集型的预训练语言模型通过在特定模态（如非结构化文本或结构化表格）的问答数据上进行微调，来适应问答等下游任务。为了避免训练这种内存消耗巨大的模型，同时又能为每种模态采用统一的架构，参数高效适配器在Transformer层之间添加并训练小型的任务特定瓶颈层。在这项工作中，我们研究了编码器-解码器模型中面向结构化表格数据和非结构化文本数据的参数高效抽象式问答，每种模态仅使用1.5%的额外参数。我们还对编码器和解码器模块中的适配器层进行了消融实验，以研究效率与性能之间的权衡，并证明将额外可训练参数降低到0.7%-1.0%仍能取得具有可比性的结果。我们的模型在Tablesum和FeTaQA等表格问答数据集上超越了当前最先进的模型，并在NarrativeQA等文本问答数据集上以远少于微调的可训练参数量取得了具有可比性的性能。
英文摘要：
A long-term ambition of information seeking QA systems is to reason over multi-modal contexts and generate natural answers to user queries. Today, memory intensive pre-trained language models are adapted to downstream tasks such as QA by fine-tuning the model on QA data in a specific modality like unstructured text or structured tables. To avoid training such memory-hungry models while utilizing a uniform architecture for each modality, parameter-efficient adapters add and train small task-specific bottle-neck layers between transformer layers. In this work, we study parameter-efficient abstractive QA in encoder-decoder models over structured tabular data and unstructured textual data using only 1.5% additional parameters for each modality. We also ablate over adapter layers in both encoder and decoder modules to study the efficiency-performance trade-off and demonstrate that reducing additional trainable parameters down to 0.7%-1.0% leads to comparable results. Our models out-perform current state-of-the-art models on tabular QA datasets such as Tablesum and FeTaQA, and achieve comparable performance on a textual QA dataset such as NarrativeQA using significantly less trainable parameters than fine-tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
