SwinDocSegmenter:一种用于文档实例分割的端到端统一域自适应Transformer
SwinDocSegmenter: An End-to-End Unified Domain Adaptive Transformer for Document Instance Segmentation
AI总结:
本文提出一种用于文档图像复杂布局端到端实例分割的统一Transformer架构,采用对比训练与混合查询选择进行锚点初始化,并在四大基准上以十亿参数超越现有最先进方法。
摘要:
文档的实例级分割包括为图像的每个像素分配类别感知和实例感知的标签。这是文档解析以理解其内容的关键步骤。本文提出了一种统一的Transformer编码器-解码器架构,用于文档图像中复杂布局的端到端实例分割。该方法采用对比训练与混合查询选择来进行解码器中的锚点初始化。随后,在获得的查询嵌入和(来自编码器的)像素嵌入图之间执行点积,以进行语义推理。在PubLayNet、PRIMA、Historical Japanese (HJ)和TableBank等竞争性基准上的大量实验表明,具有SwinL骨干网络的模型在十亿参数下实现了更好的分割性能,优于现有的最先进方法,平均精度分别为93.72、54.39、84.65和98.04。代码已在以下地址公开提供:github.com/ayanban011/SwinDocSegmenter
英文摘要:
Instance-level segmentation of documents consists in assigning a class-aware and instance-aware label to each pixel of the image. It is a key step in document parsing for their understanding. In this paper, we present a unified transformer encoder-decoder architecture for en-to-end instance segmentation of complex layouts in document images. The method adapts a contrastive training with a mixed query selection for anchor initialization in the decoder. Later on, it performs a dot product between the obtained query embeddings and the pixel embedding map (coming from the encoder) for semantic reasoning. Extensive experimentation on competitive benchmarks like PubLayNet, PRIMA, Historical Japanese (HJ), and TableBank demonstrate that our model with SwinL backbone achieves better segmentation performance than the existing state-of-the-art approaches with the average precision of \textbf{93.72}, \textbf{54.39}, \textbf{84.65} and \textbf{98.04} respectively under one billion parameters. The code is made publicly available at: \href{https://github.com/ayanban011/SwinDocSegmenter}{github.com/ayanban011/SwinDocSegmenter}
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。