Hyper-flexible Convolutional Neural Networks based on Generalized Lehmer and Power Means
Terziyan, V., Malyk, D., Golovianko, M., & Branytskyi, V. (2022). Hyper-flexible Convolutional Neural Networks based on Generalized Lehmer and Power Means. Neural Networks, 155, 177-203. https://doi.org/10.1016/j.neunet.2022.08.017
Published in
Neural NetworksDate
2022Copyright
© 2022 The Author(s). Published by Elsevier Ltd.
Convolutional Neural Network is one of the famous members of the deep learning family of neural network architectures, which is used for many purposes, including image classification. In spite of the wide adoption, such networks are known to be highly tuned to the training data (samples representing a particular problem), and they are poorly reusable to address new problems. One way to change this would be, in addition to trainable weights, to apply trainable parameters of the mathematical functions, which simulate various neural computations within such networks. In this way, we may distinguish between the narrowly focused task-specific parameters (weights) and more generic capability-specific parameters. In this paper, we suggest a couple of flexible mathematical functions (Generalized Lehmer Mean and Generalized Power Mean) with trainable parameters to replace some fixed operations (such as ordinary arithmetic mean or simple weighted aggregation), which are traditionally used within various components of a convolutional neural network architecture. We named the overall architecture with such an update as a hyper-flexible convolutional neural network. We provide mathematical justification of various components of such architecture and experimentally show that it performs better than the traditional one, including better robustness regarding the adversarial perturbations of testing data.
...
Publisher
ElsevierISSN Search the Publication Forum
0893-6080Keywords
Publication in research information system
https://converis.jyu.fi/converis/portal/detail/Publication/151802598
Metadata
Show full item recordCollections
License
Related items
Showing items with similar title or keywords.
-
Causality-Aware Convolutional Neural Networks for Advanced Image Classification and Generation
Terziyan, Vagan; Vitko, Oleksandra (Elsevier, 2023)Smart manufacturing uses emerging deep learning models, and particularly Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), for different industrial diagnostics tasks, e.g., classification, ... -
Domain‐specific neural networks improve automated bird sound recognition already with small amount of local data
Lauha, Patrik; Somervuo, Panu; Lehikoinen, Petteri; Geres, Lisa; Richter, Tobias; Seibold, Sebastian; Ovaskainen, Otso (Wiley-Blackwell, 2022)An automatic bird sound recognition system is a useful tool for collecting data of different bird species for ecological analysis. Together with autonomous recording units (ARUs), such a system provides a possibility to ... -
Convolutional Neural Network Based Sleep Stage Classification with Class Imbalance
Xu, Qi; Zhou, Dongdong; Wang, Jian; Shen, Jiangrong; Kettunen, Lauri; Cong, Fengyu (IEEE, 2022)Accurate sleep stage classification is vital to assess sleep quality and diagnose sleep disorders. Numerous deep learning based models have been designed for accomplishing this labor automatically. However, the class ... -
One and Two Dimensional Convolutional Neural Networks for Seizure Detection Using EEG Signals
Wang, Xiaoshuang; Ristaniemi, Tapani; Cong, Fengyu (IEEE, 2020)Deep learning for the automated detection of epileptic seizures has received much attention during recent years. In this work, one dimensional convolutional neural network (1D-CNN) and two dimensional convolutional neural ... -
Classification of Heart Sounds Using Convolutional Neural Network
Li, Fan; Tang, Hong; Shang, Shang; Mathiak, Klaus; Cong, Fengyu (MDPI, 2020)Heart sounds play an important role in the diagnosis of cardiac conditions. Due to the low signal-to-noise ratio (SNR), it is problematic and time-consuming for experts to discriminate different kinds of heart sounds. Thus, ...