arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.15765cs.CVcs.AIcs.IR
收藏

迈向类人的机器理解:视觉富文本文档中的少样本关系学习

Towards Human-Like Machine Comprehension: Few-Shot Relational Learning in Visually-Rich Documents

School of Computer Engineering and Science, Shanghai University(上海大学计算机工程与科学学院);Graduate School of Informatics, Kyoto University(京都大学信息学研究科);Department of Computer Science and Engineering, Shanghai Jiao Tong University(上海交通大学计算机科学与工程系)

Hao Wang, Tang Li, Chenhui Chu, Nengjun Zhu, Rui Wang, Pinpin Zhu

AI总结:

针对视觉富文本文档中少样本键值关系提取,提出结合2D空间先验与原型校正的变分方法,生成类人感知的关系表示,并在新建基准上优于现有方法。

摘要:

键值关系在视觉富文本文档(VRDs)中普遍存在,通常表现为在特定空间区域中呈现,并伴有特定的颜色和字体样式。这些非文本线索作为重要指示器,极大地增强了人类对这类关系三元组的理解和获取。然而,当前的文档AI方法往往未能考虑与视觉和空间特征相关的宝贵先验信息,导致性能次优,尤其是在处理有限样本时。为解决这一局限,我们的研究聚焦于少样本关系学习,专门针对VRDs中键值关系三元组的提取。鉴于该任务缺乏合适的数据集,我们基于现有的监督基准数据集引入了两个新的少样本基准。此外,我们提出了一种变分方法,结合了关系2D空间先验和原型校正技术。该方法旨在以类似人类感知的方式,生成对空间上下文和未见关系更具感知能力的关系表示。实验结果通过展示其优于现有方法的能力,证明了我们提出方法的有效性。本研究也为实际应用开辟了新的可能性。

英文摘要:

Key-value relations are prevalent in Visually-Rich Documents (VRDs), often depicted in distinct spatial regions accompanied by specific color and font styles. These non-textual cues serve as important indicators that greatly enhance human comprehension and acquisition of such relation triplets. However, current document AI approaches often fail to consider this valuable prior information related to visual and spatial features, resulting in suboptimal performance, particularly when dealing with limited examples. To address this limitation, our research focuses on few-shot relational learning, specifically targeting the extraction of key-value relation triplets in VRDs. Given the absence of a suitable dataset for this task, we introduce two new few-shot benchmarks built upon existing supervised benchmark datasets. Furthermore, we propose a variational approach that incorporates relational 2D-spatial priors and prototypical rectification techniques. This approach aims to generate relation representations that are more aware of the spatial context and unseen relation in a manner similar to human perception. Experimental results demonstrate the effectiveness of our proposed method by showcasing its ability to outperform existing methods. This study also opens up new possibilities for practical applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑