arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1106.0718cs.DBcs.DLcs.IR
收藏

使用RDBMS对OCR数据进行概率管理

Probabilistic Management of OCR Data using an RDBMS

Arun Kumar, Christopher Ré

摘要:

扫描表格和文档的数字化正在改变企业管理的数据源。为了将这些新的数据源与企业数据相结合,目前最先进的方法是使用光学字符识别(OCR)软件将图像转换为ASCII文本,然后将得到的ASCII文本存储在关系数据库中。OCR问题具有挑战性,因此OCR的输出通常包含错误。反过来,对OCR输出的查询可能无法检索到相关答案。最先进的OCR程序,例如,为Google Books提供动力的OCR使用概率模型,该模型在OCR过程中捕获许多替代方案。只有当OCR结果存储在数据库中时,这些方法才能消除不确定性。在这项工作中,我们建议保留概率模型OCR过程中产生的关系数据库管理系统。一个关键的技术挑战是OCR软件产生的概率数据非常大(一本书从400 kB的ASCII扩展到2GB)。因此,将这些模型与RDBMS集成的基线解决方案比单表选择项目查询的标准文本处理慢1000倍以上。然而,许多应用程序可能具有介于ASCII和OCR软件输出的完整模型这两个极端之间的质量-性能需求。因此,我们提出了一种新的近似方案称为断奏,允许用户交易召回查询性能。此外,我们提供了一个正式的分析我们的计划的属性,并描述了我们如何将我们的计划与标准的RDBMS文本索引。

英文摘要:

The digitization of scanned forms and documents is changing the data sources that enterprises manage. To integrate these new data sources with enterprise data, the current state-of-the-art approach is to convert the images to ASCII text using optical character recognition (OCR) software and then to store the resulting ASCII text in a relational database. The OCR problem is challenging, and so the output of OCR often contains errors. In turn, queries on the output of OCR may fail to retrieve relevant answers. State-of-the-art OCR programs, e.g., the OCR powering Google Books, use a probabilistic model that captures many alternatives during the OCR process. Only when the results of OCR are stored in the database, do these approaches discard the uncertainty. In this work, we propose to retain the probabilistic models produced by OCR process in a relational database management system. A key technical challenge is that the probabilistic data produced by OCR software is very large (a single book blows up to 2GB from 400kB as ASCII). As a result, a baseline solution that integrates these models with an RDBMS is over 1000x slower versus standard text processing for single table select-project queries. However, many applications may have quality-performance needs that are in between these two extremes of ASCII and the complete model output by the OCR software. Thus, we propose a novel approximation scheme called Staccato that allows a user to trade recall for query performance. Additionally, we provide a formal analysis of our scheme's properties, and describe how we integrate our scheme with standard-RDBMS text indexing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑