ITrans : generative image inpainting with transformers
Miao, W., Wang, L., Lu, H., Huang, K., Shi, X., & Liu, B. (2024). ITrans : generative image inpainting with transformers. Multimedia Systems, 30(1), Article 21. https://doi.org/10.1007/s00530-023-01211-w
Julkaistu sarjassa
Multimedia SystemsPäivämäärä
2024Tekijänoikeudet
© The Author(s) 2024
Despite significant improvements, convolutional neural network (CNN) based methods are struggling with handling long-range global image dependencies due to their limited receptive fields, leading to an unsatisfactory inpainting performance under complicated scenarios. To address this issue, we propose the Inpainting Transformer (ITrans) network, which combines the power of both self-attention and convolution operations. The ITrans network augments convolutional encoder–decoder structure with two novel designs, i.e., the global and local transformers. The global transformer aggregates high-level image context from the encoder in a global perspective, and propagates the encoded global representation to the decoder in a multi-scale manner. Meanwhile, the local transformer is intended to extract low-level image details inside the local neighborhood at a reduced computational overhead. By incorporating the above two transformers, ITrans is capable of both global relationship modeling and local details encoding, which is essential for hallucinating perceptually realistic images. Extensive experiments demonstrate that the proposed ITrans network outperforms favorably against state-of-the-art inpainting methods both quantitatively and qualitatively.
...
Julkaisija
SpringerISSN Hae Julkaisufoorumista
0942-4962Asiasanat
Julkaisu tutkimustietojärjestelmässä
https://converis.jyu.fi/converis/portal/detail/Publication/197929931
Metadata
Näytä kaikki kuvailutiedotKokoelmat
Lisätietoja rahoituksesta
Open Access funding provided by University of Jyväskylä (JYU).Lisenssi
Samankaltainen aineisto
Näytetään aineistoja, joilla on samankaltainen nimeke tai asiasanat.
-
Causality-Aware Convolutional Neural Networks for Advanced Image Classification and Generation
Terziyan, Vagan; Vitko, Oleksandra (Elsevier, 2023)Smart manufacturing uses emerging deep learning models, and particularly Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), for different industrial diagnostics tasks, e.g., classification, ... -
Hyper-flexible Convolutional Neural Networks based on Generalized Lehmer and Power Means
Terziyan, Vagan; Malyk, Diana; Golovianko, Mariia; Branytskyi, Vladyslav (Elsevier, 2022)Convolutional Neural Network is one of the famous members of the deep learning family of neural network architectures, which is used for many purposes, including image classification. In spite of the wide adoption, such ... -
Generating Hyperspectral Skin Cancer Imagery using Generative Adversarial Neural Network
Annala, Leevi; Neittaanmäki, Noora; Paoli, John; Zaar, Oscar; Pölönen, Ilkka (IEEE, 2020)In this study we develop a proof of concept of using generative adversarial neural networks in hyperspectral skin cancer imagery production. Generative adversarial neural network is a neural network, where two neural ... -
Tree Species Identification Using 3D Spectral Data and 3D Convolutional Neural Network
Pölönen, Ilkka; Annala, Leevi; Rahkonen, Samuli; Nevalainen, Olli; Honkavaara, Eija; Tuominen, Sakari; Viljanen, Niko; Hakala, Teemu (IEEE, 2019)In this study we apply 3D convolutional neural network (CNN) for tree species identification. Study includes the three most common Finnish tree species. Study uses a relatively large high-resolution spectral data set, ... -
One and Two Dimensional Convolutional Neural Networks for Seizure Detection Using EEG Signals
Wang, Xiaoshuang; Ristaniemi, Tapani; Cong, Fengyu (IEEE, 2020)Deep learning for the automated detection of epileptic seizures has received much attention during recent years. In this work, one dimensional convolutional neural network (1D-CNN) and two dimensional convolutional neural ...
Ellei toisin mainittu, julkisesti saatavilla olevia JYX-metatietoja (poislukien tiivistelmät) saa vapaasti uudelleenkäyttää CC0-lisenssillä.