联邦文档视觉问答：一项初步研究
Federated Document Visual Question Answering: A Pilot Study
AI总结：
本研究提出联邦学习框架下的文档VQA训练方法，结合自预训练与集中式自适应优化，在异构私有文档数据上提升模型性能，并验证超参数调优的重要性。
摘要：
文档分析研究的一个重要障碍是文档往往受版权保护或包含私人信息，这禁止了它们的公开出版以及集中式大规模文档数据集的创建。相反，文档分散在私有数据孤岛中，使得在异构数据上进行大规模训练成为一项繁琐的任务。在这项工作中，我们探索使用联邦学习（FL）方案作为在去中心化的私有文档数据上训练共享模型的一种方式。我们聚焦于文档VQA问题，这是一个特别适合该方法的任务，因为在不同领域中，模型所需的推理能力类型可能大不相同。因此，在异构文档数据集上实现训练可以极大地丰富DocVQA模型。我们汇集了来自不同领域的现有DocVQA数据集，以反映真实世界应用中的数据异构性。我们在这种多模态设置中探索自预训练技术，其中相同的数据既用于预训练又用于微调，使其与隐私保护相关。我们进一步提出将自预训练与使用集中式自适应优化的联邦DocVQA训练方法相结合，该方法优于FedAvg基线。通过大量实验，我们还对使用FL训练DocVQA模型进行了多方面的分析，为未来关于该任务的研究提供了见解。我们表明，我们的预训练策略可以在联邦训练下使用多样化的DocVQA数据集有效地学习和扩展，并且调整超参数对于联邦下的实际文档任务至关重要。
英文摘要：
An important handicap of document analysis research is that documents tend to be copyrighted or contain private information, which prohibits their open publication and the creation of centralised, large-scale document datasets. Instead, documents are scattered in private data silos, making extensive training over heterogeneous data a tedious task. In this work, we explore the use of a federated learning (FL) scheme as a way to train a shared model on decentralised private document data. We focus on the problem of Document VQA, a task particularly suited to this approach, as the type of reasoning capabilities required from the model can be quite different in diverse domains. Enabling training over heterogeneous document datasets can thus substantially enrich DocVQA models. We assemble existing DocVQA datasets from diverse domains to reflect the data heterogeneity in real-world applications. We explore the self-pretraining technique in this multi-modal setting, where the same data is used for both pretraining and finetuning, making it relevant for privacy preservation. We further propose combining self-pretraining with a Federated DocVQA training method using centralized adaptive optimization that outperforms the FedAvg baseline. With extensive experiments, we also present a multi-faceted analysis on training DocVQA models with FL, which provides insights for future research on this task. We show that our pretraining strategies can effectively learn and scale up under federated training with diverse DocVQA datasets and tuning hyperparameters is essential for practical document tasks under federation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
