重新思考手写文本生成评估：桥接生成与识别
Rethinking HTG Evaluation: Bridging Generation and Recognition
AI总结：
本文针对手写文本生成评估提出三种新指标，结合手写识别与书写者识别模型，从风格、内容和多样性角度弥补 FID 等传统指标的不足，并在 IAM 数据库上验证其有效性。
摘要：
针对自然图像任务的生成模型评估已被广泛研究。类似的协议和指标也被用于具有独特特殊性的场景中，例如手写文本生成（HTG），即使它们可能并不完全适用。在本工作中，我们引入了三种专为 HTG 评估量身定制的度量指标：$ \ ext{HTG}_{\ ext{HTR}} $、$ \ ext{HTG}_{\ ext{style}} $ 和 $ \ ext{HTG}_{\ ext{OOV}} $，并论证这些指标更适合评估生成的手写图像质量。这些指标依赖于手写文本识别（Handwriting Text Recognition）和书写者识别（Writer Identification）模型的识别错误率/准确率，并强调书写风格、文本内容和多样性是符合手写图像内容的主要方面。我们在 IAM 手写数据库上进行了全面实验，结果表明广泛使用的指标（如 FID）无法恰当地量化生成手写样本的多样性和实际效用。我们的发现表明，所提出的指标信息更丰富，并强调了在 HTG 中建立标准化评估协议的必要性。所提出的指标为评估 HTG 质量提供了一种更稳健、更具信息量的协议，有助于提升手写文本识别（HTR）的性能。评估协议的代码可在以下地址获取：https://github.com/koninik/HTG_evaluation。
英文摘要：
The evaluation of generative models for natural image tasks has been extensively studied. Similar protocols and metrics are used in cases with unique particularities, such as Handwriting Generation, even if they might not be completely appropriate. In this work, we introduce three measures tailored for HTG evaluation, $ \text{HTG}_{\text{HTR}} $, $ \text{HTG}_{\text{style}} $, and $ \text{HTG}_{\text{OOV}} $, and argue that they are more expedient to evaluate the quality of generated handwritten images. The metrics rely on the recognition error/accuracy of Handwriting Text Recognition and Writer Identification models and emphasize writing style, textual content, and diversity as the main aspects that adhere to the content of handwritten images. We conduct comprehensive experiments on the IAM handwriting database, showcasing that widely used metrics such as FID fail to properly quantify the diversity and the practical utility of generated handwriting samples. Our findings show that our metrics are richer in information and underscore the necessity of standardized evaluation protocols in HTG. The proposed metrics provide a more robust and informative protocol for assessing HTG quality, contributing to improved performance in HTR. Code for the evaluation protocol is available at: https://github.com/koninik/HTG_evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
