arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.09987cs.CV
收藏

OneChart:通过一个辅助Token纯化图表结构提取

OneChart: Purify the Chart Structural Extraction via One Auxiliary Token

MEGVII Technology(旷视科技)

Jinyue Chen, Lingyu Kong, Haoran Wei, Chenglong Liu, Zheng Ge, Liang Zhao, Jianjian Sun, Chunrui Han, Xiangyu Zhang

AI总结:

提出OneChart,一个专为图表结构提取设计的可靠智能体,通过引入辅助token和额外解码器增强数值特征提取与自评估机制,在仅0.2B参数下显著超越SOTA模型,并提升下游LVLM在ChartQA上的准确率10%以上。

摘要:

由于样式、数值、文本等的多样性,图表解析提出了重大挑战。即使是拥有数十亿参数的先进大型视觉语言模型(LVLMs)也难以令人满意地处理此类任务。为了解决这个问题,我们提出了OneChart:一个专门为图表信息结构提取设计的可靠智能体。与流行的LVLMs类似,OneChart包含一个自回归主体。独特的是,为了提高输出数值部分的可靠性,我们引入了一个放置在总tokens开头的辅助token以及一个额外的解码器。经过数值优化的(辅助)token允许后续用于图表解析的tokens通过因果注意力捕获增强的数值特征。此外,借助辅助token,我们设计了一种自我评估机制,使模型能够通过为生成内容提供置信度分数来衡量其图表解析结果的可靠性。与当前最先进的(SOTA)图表解析模型(例如DePlot、ChartVLM、ChartAst)相比,尽管仅拥有0.2B参数,OneChart在多个公共基准测试中的图表结构提取平均精度(AP)上显著优于它们。此外,作为图表解析智能体,它还在下游ChartQA基准测试中为流行的LVLM(LLaVA-1.6)带来了10%以上的准确率提升。

英文摘要:

Chart parsing poses a significant challenge due to the diversity of styles, values, texts, and so forth. Even advanced large vision-language models (LVLMs) with billions of parameters struggle to handle such tasks satisfactorily. To address this, we propose OneChart: a reliable agent specifically devised for the structural extraction of chart information. Similar to popular LVLMs, OneChart incorporates an autoregressive main body. Uniquely, to enhance the reliability of the numerical parts of the output, we introduce an auxiliary token placed at the beginning of the total tokens along with an additional decoder. The numerically optimized (auxiliary) token allows subsequent tokens for chart parsing to capture enhanced numerical features through causal attention. Furthermore, with the aid of the auxiliary token, we have devised a self-evaluation mechanism that enables the model to gauge the reliability of its chart parsing results by providing confidence scores for the generated content. Compared to current state-of-the-art (SOTA) chart parsing models, e.g., DePlot, ChartVLM, ChartAst, OneChart significantly outperforms in Average Precision (AP) for chart structural extraction across multiple public benchmarks, despite enjoying only 0.2 billion parameters. Moreover, as a chart parsing agent, it also brings 10%+ accuracy gains for the popular LVLM (LLaVA-1.6) in the downstream ChartQA benchmark.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑