arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.03643cs.CVcs.CL
收藏

Image Over Text:以字符检测匹配革新公式识别评估

Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matching

Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学)

Bin Wang, Fan Wu, Linke Ouyang, Zhuangcheng Gu, Rui Zhang, Renqiu Xia, Bo Zhang, Conghui He

AI总结:

针对现有公式识别文本类评估指标存在的局限性,本研究提出基于图像级字符检测匹配的CDM评估指标,通过渲染公式为图像并结合空间信息做字符匹配,实验证明其更贴合人类评估标准,评估更公平。

摘要:

由于数学表达式结构复杂、符号多样,公式识别面临着巨大挑战。尽管公式识别模型在不断发展,但这类模型使用的BLEU、编辑距离等评估指标仍存在明显局限性。它们忽略了同一公式存在多种表示形式的事实,且对训练数据分布高度敏感,进而导致公式识别评估存在不公平性。为此,我们提出了字符检测匹配(Character Detection Matching,CDM)指标,通过设计图像层面而非LaTeX层面的指标分数来确保评估的客观性。具体而言,CDM将模型预测的LaTeX公式和真实标注的LaTeX公式都渲染为图像格式的公式,随后利用视觉特征提取与定位技术实现精确的字符级匹配,并融入空间位置信息。这种具备空间感知能力的字符匹配方法,相较于此前仅依赖文本字符匹配的BLEU和编辑距离指标,能提供更准确、更公平的评估。实验中,我们使用CDM、BLEU和ExpRate指标评估了多种公式识别模型。结果表明,CDM更符合人类评估标准,并且通过消除不同公式表示形式造成的差异,能在不同模型间实现更公平的比较。相关代码可在https://github.com/opendatalab/UniMERNet/tree/main/cdm获取。

英文摘要:

Formula recognition presents significant challenges due to the complicated structure and varied notation of mathematical expressions. Despite continuous advancements in formula recognition models, the evaluation metrics employed by these models, such as BLEU and Edit Distance, still exhibit notable limitations. They overlook the fact that the same formula has diverse representations and is highly sensitive to the distribution of training data, thereby causing unfairness in formula recognition evaluation. To this end, we propose a Character Detection Matching (CDM) metric, ensuring the evaluation objectivity by designing an image-level rather than a LaTeX-level metric score. Specifically, CDM renders both the model-predicted LaTeX and the ground-truth LaTeX formulas into image-formatted formulas, then employs visual feature extraction and localization techniques for precise character-level matching, incorporating spatial position information. Such a spatially-aware and character-matching method offers a more accurate and equitable evaluation compared with previous BLEU and Edit Distance metrics that rely solely on text-based character matching. Experimentally, we evaluated various formula recognition models using CDM, BLEU, and ExpRate metrics. Their results demonstrate that the CDM aligns more closely with human evaluation standards and provides a fairer comparison across different models by eliminating discrepancies caused by diverse formula representations. Code is available at https://github.com/opendatalab/UniMERNet/tree/main/cdm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑