In recent years, machine learning and deep learning approaches such as artificial neural networks have gained in popularity for the resolution of automatic puzzle resolution problems. Indeed, these methods are able to extract high-level representations from images, and then can be trained to separate matching image pieces from non-matching ones. These applications have many similarities to the problem of ancient document reconstruction from partially recovered fragments. In this work we present a solution based on a Graph Neural Network, using pairwise patch information to assign labels to edges representing the spatial relationships between pairs. This network classifies the relationship between a source and a target patch as being one of Up, Down, Left, Right or None. By doing so for all edges, our model outputs a new graph representing a reconstruction proposal. Finally, we show that our model is not only able to provide correct classifications at the edge-level, but also to generate partial or full reconstruction graphs from a set of patches.
University of Sfax（斯法克斯大学）；Université de La Rochelle（拉罗谢尔大学）
The analysis of historical documents is still a topical issue given the importance of information that can be extracted and also the importance given by the institutions to preserve their heritage. The main idea in order to characterize the content of the images of ancient documents after attempting to clean the image is segmented blocks texts from the same image and tries to find similar blocks in either the same image or the entire image database. Most approaches of offline handwriting recognition proceed by segmenting words into smaller pieces (usually characters) which are recognized separ…
University of Sfax（斯法克斯大学）；University of La Rochelle（拉罗谢尔大学）
This paper deals with the recognition and matching of text in both cartographic maps and ancient documents. The purpose of this work is to find similar text regions based on statistical and global features. A phase of normalization is done first, in object to well categorize the same quantity of information. A phase of wordspotting is done next by combining local and global features. We make different experiments by combining the different techniques of extracting features in order to obtain better results in recognition phase. We applied fontspotting on both ancient documents and cartographic…
National Technical University of Athens（雅典国家技术大学）；Technical Institute of Chalkida（哈尔基达技术学院）；Ionian University（爱奥尼亚大学）；Furman University（弗曼大学）
In this paper, a novel approach is introduced for classifying curves into proper families, according to their similarity. First, a mathematical quantity we call plane curvature is introduced and a number of propositions are stated and proved. Proper similarity measures of two curves are introduced and a subsequent statistical analysis is applied. First, the efficiency of the curve fitting process has been tested on 2 shapes datasets of reference. Next, the methodology has been applied to the very important problem of classifying 23 Byzantine codices and 46 Ancient inscriptions to their writers…
Puzzle Pieces Picker: Deciphering Ancient Chinese Characters with Radical Reconstruction
Huazhong University of Science and Technology（华中科技大学）；The University of Adelaide（阿德莱德大学）；Anyang Normal University（安阳师范学院）；South China University of Technology（华南理工大学）
Pontifical Catholic University of Parana (PUCPR)（巴拉那天主教大学）；State University of Ponta Grossa (UEPG)（蓬塔格罗萨州立大学）；École de Technologie Supérieure (ÉTS), Université du Québec（魁北克大学高等技术学院）；Normandie Univ（诺曼底大学）；UNIROUEN（鲁昂大学）；UNIHAVRE（勒阿弗尔大学）；INSA Rouen（鲁昂国立应用科学学院）；Federal University of Paraná（巴拉那联邦大学）
This paper describes two approaches for content-based image retrieval and pattern spotting in document images using deep learning. The first approach uses a pre-trained CNN model to cope with the lack of training data, which is fine-tuned to achieve a compact yet discriminant representation of queries and image candidates. The second approach uses a Siamese Convolution Neural Network trained on a previously prepared subset of image pairs from the ImageNet dataset to provide the similarity-based feature maps. In both methods, the learned representation scheme considers feature maps of different…
Can Artificial Intelligence Reconstruct Ancient Mosaics?
Universidad Antonio de Nebrija（安东尼奥·德·内夫里哈大学）；Universidad de Valladolid（巴利亚多利德大学）；Universidad Politécnica de Madrid（马德里理工大学）；Northeastern University（东北大学）
University of Sfax（斯法克斯大学）；Université de La Rochelle（拉罗谢尔大学）
Characterizing noisy or ancient documents is a challenging problem up to now. Many techniques have been done in order to effectuate feature extraction and image indexation for such documents. Global approaches are in general less robust and exact than local approaches. That's why, we propose in this paper, a hybrid system based on global approach(fractal dimension), and a local one based on SIFT descriptor. The Scale Invariant Feature Transform seems to do well with our application since it's rotation invariant and relatively robust to changing illumination.In the first step the calculation of…
Three-stage binarization of color document images based on discrete wavelet transform and generative adversarial networks
Graduate Institute of Networking and Multimedia, National Taiwan University（国立台湾大学网络与多媒体研究所）；Department of Electrical and Computer Engineering, Tamkang University（潭府大学电子工程系）；Department of Electrical and Computer Engineering, Rice University（里德大学电子工程系）
Instituto Federal do Espírito Santo (IFES)（埃斯皮里图桑托联邦学院）；Universidade Federal do Espírito Santo (UFES)（埃斯皮里图桑托联邦大学）；École de Technologie Supérieure (ÉTS)（高等技术学院）；Université du Québec（魁北克大学）
The reconstruction of shredded documents consists of coherently arranging fragments of paper (shreds) to recover the original document(s). A great challenge in computational reconstruction is to properly evaluate the compatibility between the shreds. While traditional pixel-based approaches are not robust to real shredding, more sophisticated solutions compromise significantly time performance. The solution presented in this work extends our previous deep learning method for single-page reconstruction to a more realistic/complex scenario: the reconstruction of several mixed shredded documents …
University of the Philippines Los Baños（菲律宾大学洛斯巴尼奥斯分校）
A shredded document $D$ is a document whose pages have been cut into strips for the purpose of destroying private, confidential, or sensitive information $I$ contained in $D$. Shredding has become a standard means of government organizations, businesses, and private individuals to destroy archival records that have been officially classified for disposal. It can also be used to destroy documentary evidence of wrongdoings by entities who are trying to hide $I$. In this paper, we present an optimal $O((n\times m)^2)$ algorithm $A$ that reconstructs an $n$-page $D$, where each page $p$ is shredde…
University of Science and Technology of China（中国科学技术大学）；iFLYTEK Research（科大讯飞研究院）
The problem of document structure reconstruction refers to converting digital or scanned documents into corresponding semantic structures. Most existing works mainly focus on splitting the boundary of each element in a single document page, neglecting the reconstruction of semantic structure in multi-page documents. This paper introduces hierarchical reconstruction of document structures as a novel task suitable for NLP and CV fields. To better evaluate the system performance on the new task, we built a large-scale dataset named HRDoc, which consists of 2,500 multi-page documents with nearly 2…
Npix2Cpix: A GAN-Based Image-to-Image Translation Network With Retrieval- Classification Integration for Watermark Retrieval From Historical Document Images
BRAC University（BRAC大学）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）
A line of a bilingual document page may contain text words in regional language and numerals in English. For Optical Character Recognition (OCR) of such a document page, it is necessary to identify different script forms before running an individual OCR system. In this paper, we have identified a tool of morphological opening by reconstruction of an image in different directions and regional descriptors for script identification at word level, based on the observation that every text has a distinct visual appearance. The proposed system is developed for three Indian major bilingual documents, …
Indian Institute of Technology, Patna（印度理工学院巴特那分校）；Indian Statistical Institute, Kolkata（印度统计研究所加尔各答分所）
Image restoration is very crucial computer vision task. This paper describes two novel methods for the restoration of old degraded handwritten documents using deep neural network. In addition to that, a small-scale dataset of 26 heritage letters images is introduced. The ground truth data to train the desired network is generated semi automatically involving a pragmatic combination of color transformation, Gaussian mixture model based segmentation and shape correction by using mathematical morphological operators. In the first approach, a deep neural network has been used for text extraction f…
A patch-based non-local restoration and reconstruction method for preprocessing degraded document images is introduced. The method collects relative data from the whole input image, while the image data are first represented by a content-level descriptor based on patches. This patch-equivalent representation of the input image is then corrected based on similar patches identified using a modified genetic algorithm (GA) resulting in a low computational load. The corrected patch-equivalent is then converted to the output restored image. The fact that the method uses the patches at the content le…
Federal Institute of Espírito Santo (IFES)（圣埃斯皮里图联邦学院（IFES））；Federal University of Espírito Santo (UFES)（圣埃斯皮里图联邦大学（UFES））；École de Technologie Supérieure (ÉTS)（高等技术学院（ÉTS））
The reconstruction of shredded documents consists in arranging the pieces of paper (shreds) in order to reassemble the original aspect of such documents. This task is particularly relevant for supporting forensic investigation as documents may contain criminal evidence. As an alternative to the laborious and time-consuming manual process, several researchers have been investigating ways to perform automatic digital reconstruction. A central problem in automatic reconstruction of shredded documents is the pairwise compatibility evaluation of the shreds, notably for binary text documents. In thi…
MESA: A Training-Free Multi-Exemplar Deep Framework for Restoring Ancient Inscription Textures
Department of Computer Science and Engineering（计算机科学与工程系）；University of Ioannina（伊奥安纳大学）；Inria Paris-Rocquencourt（巴黎-罗克琴堡研究所）；Rajiv Gandhi University（拉朱·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）
University of California, San Diego（加州大学圣迭戈分校）；University of Maryland（马里兰大学）
We present a self-supervised pre-training approach for learning rich visual language representations for both handwritten and printed historical document transcription. After supervised fine-tuning of our pre-trained encoder representations for low-resource document transcription on two languages, (1) a heterogeneous set of handwritten Islamicate manuscript images and (2) early modern English printed documents, we show a meaningful improvement in recognition accuracy over the same supervised model trained from scratch with as few as 30 line image transcriptions for training. Our masked languag…
