ITrans : generative image inpainting with transformers
Miao, W., Wang, L., Lu, H., Huang, K., Shi, X., & Liu, B. (2024). ITrans : generative image inpainting with transformers. Multimedia Systems, 30(1), Article 21. https://doi.org/10.1007/s00530-023-01211-w
Julkaistu sarjassa
Multimedia SystemsPäivämäärä
2024Tekijänoikeudet
© The Author(s) 2024
2024:62 | 2025:5
Despite significant improvements, convolutional neural network (CNN) based methods are struggling with handling long-range global image dependencies due to their limited receptive fields, leading to an unsatisfactory inpainting performance under complicated scenarios. To address this issue, we propose the Inpainting Transformer (ITrans) network, which combines the power of both self-attention and convolution operations. The ITrans network augments convolutional encoder–decoder structure with two novel designs, i.e., the global and local transformers. The global transformer aggregates high-level image context from the encoder in a global perspective, and propagates the encoded global representation to the decoder in a multi-scale manner. Meanwhile, the local transformer is intended to extract low-level image details inside the local neighborhood at a reduced computational overhead. By incorporating the above two transformers, ITrans is capable of both global relationship modeling and local details encoding, which is essential for hallucinating perceptually realistic images. Extensive experiments demonstrate that the proposed ITrans network outperforms favorably against state-of-the-art inpainting methods both quantitatively and qualitatively.
...
Julkaisija
SpringerISSN Hae Julkaisufoorumista
0942-4962Asiasanat
Julkaisu tutkimustietojärjestelmässä
https://converis.jyu.fi/converis/portal/detail/Publication/197929931
Metadata
Näytä kaikki kuvailutiedotKokoelmat
Lisätietoja rahoituksesta
Open Access funding provided by University of Jyväskylä (JYU).Lisenssi
Samankaltainen aineisto
Näytetään aineistoja, joilla on samankaltainen nimeke tai asiasanat.
-
Causality-Aware Convolutional Neural Networks for Advanced Image Classification and Generation
Terziyan, Vagan; Vitko, Oleksandra (Elsevier, 2023)Smart manufacturing uses emerging deep learning models, and particularly Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), for different industrial diagnostics tasks, e.g., classification, ... -
Hyper-flexible Convolutional Neural Networks based on Generalized Lehmer and Power Means
Terziyan, Vagan; Malyk, Diana; Golovianko, Mariia; Branytskyi, Vladyslav (Elsevier, 2022)Convolutional Neural Network is one of the famous members of the deep learning family of neural network architectures, which is used for many purposes, including image classification. In spite of the wide adoption, such ... -
Differentiating Malignant from Benign Pigmented or Non-Pigmented Skin Tumours : A Pilot Study on 3D Hyperspectral Imaging of Complex Skin Surfaces and Convolutional Neural Networks
Lindholm, Vivian; Raita-Hakola, Anna-Maria; Annala, Leevi; Salmivuori, Mari; Jeskanen, Leila; Saari, Heikki; Koskenmies, Sari; Pitkänen, Sari; Pölönen, Ilkka; Isoherranen, Kirsi; Ranki, Annamari (MDPI AG, 2022)Several optical imaging techniques have been developed to ease the burden of skin cancer disease on our health care system. Hyperspectral images can be used to identify biological tissues by their diffuse reflected spectra. ... -
Tree Species Identification Using 3D Spectral Data and 3D Convolutional Neural Network
Pölönen, Ilkka; Annala, Leevi; Rahkonen, Samuli; Nevalainen, Olli; Honkavaara, Eija; Tuominen, Sakari; Viljanen, Niko; Hakala, Teemu (IEEE, 2019)In this study we apply 3D convolutional neural network (CNN) for tree species identification. Study includes the three most common Finnish tree species. Study uses a relatively large high-resolution spectral data set, ... -
One Dimensional Convolutional Neural Networks for Seizure Onset Detection Using Long-term Scalp and Intracranial EEG
Wang, Xiaoshuang; Wang, Xiulin; Liu, Wenya; Chang, Zheng; Kärkkäinen, Tommi; Cong, Fengyu (Elsevier, 2021)Epileptic seizure detection using scalp electroencephalogram (sEEG) and intracranial electroencephalogram (iEEG) has attracted widespread attention in recent two decades. The accurate and rapid detection of seizures not ...
Ellei toisin mainittu, julkisesti saatavilla olevia JYX-metatietoja (poislukien tiivistelmät) saa vapaasti uudelleenkäyttää CC0-lisenssillä.