Result Details

TG2: text-guided transformer GAN for restoring document readability and perceived quality

KODYM, O.; HRADIŠ, M. TG2: text-guided transformer GAN for restoring document readability and perceived quality. International Journal on Document Analysis and Recognition, 2021, vol. 2021, no. 1, p. 1-14. ISSN: 1433-2825.
Type
journal article
Language
English
Authors
Kodym Oldřich, Ing., Ph.D.
Hradiš Michal, Ing., Ph.D., UAMT (FEEC), DCGM (FIT)
Abstract

Most image enhancement methods focused on restoration of digitized textual documents are limited to cases where the text information is still preserved in the input image, which may often not be the case. In this work, we propose a novel generative document restoration method which allows conditioning the restoration on a guiding signal in form of target text transcription and which does not need paired high- and low-quality images for training. We introduce a neural network architecture with an implicit text-to-image alignment module.We demonstrate good results on inpainting, debinarization and deblurring tasks, and we show that the trained models can be used to manually alter text in document images.A user study shows that that human observers confuse the outputs of the proposed enhancement method with reference high-quality images in as many as 30% of cases.

Keywords

Generative adversarial networks, Attention neural networks, Textual document restoration, Text inpainting

URL
Published
2021
Pages
1–14
Journal
International Journal on Document Analysis and Recognition, vol. 2021, no. 1, ISSN 1433-2825
Book
International Journal on Document Analysis and Recognition
Publisher
Springer Verlag
DOI
UT WoS
000698372200001
EID Scopus
BibTeX
@article{BUT175769,
  author="Oldřich {Kodym} and Michal {Hradiš}",
  title="TG2: text-guided transformer GAN for restoring document readability and perceived quality",
  journal="International Journal on Document Analysis and Recognition",
  year="2021",
  volume="2021",
  number="1",
  pages="1--14",
  doi="10.1007/s10032-021-00387-z",
  issn="1433-2833",
  url="https://link.springer.com/article/10.1007/s10032-021-00387-z"
}
Projects
Advanced content extraction and recognition for printed and handwritten documents for better accessibility and usability, MK, Program na podporu aplikovaného výzkumu a experimentálního vývoje národní a kulturní identity na léta 2016 až 2022 (NAKI II), DG18P02OVV055, start: 2018-03-01, end: 2022-12-31, completed
IT4Innovations excellence in science, MŠMT, Národní program udržitelnosti II, LQ1602, start: 2016-01-01, end: 2020-12-31, completed
Moderní metody zpracování, analýzy a zobrazování multimediálních a 3D dat, BUT, Vnitřní projekty VUT, FIT-S-20-6460, start: 2020-03-01, end: 2023-02-28, completed
Research groups
Departments
Back to top