Multilayer perceptron training with multiobjective memetic optimization

Abstract

Machine learning tasks usually come with several mutually conflicting objectives. One example is the simplicity of the learning device contrasted with the accuracy of its performance after learning. Another common example is the trade-off that must often be made between the rate of false positive and false negative predictions in diagnostic applications. For computer programs that learn from data, these objectives are formulated as mathematical functions, each of which describes one facet of the desired learning outcome. Even functions that intend to optimize the same facet may behave in a subtly different and mutually conflicting way, depending on the task and the dataset being examined. Multiobjective optimization methods developed for simultaneous optimization of such multiple objectives found their way to machine learning a few decades ago. This dissertation discusses the past and current uses of multiobjective optimization in supervised learning, especially in training a multilayer perceptron (MLP) artificial neural network for object classification. A literature overview of multiobjective MLP training is presented, supported by a semi-automatic survey using a software tool created partly by the author. Based on the literature, key goals and algorithmic elements are identified and applied to create a new framework for training MLPs consistent with an implementation used earlier for industrial projects using single-objective methods. Simulated datasets are used to illustrate the functionality of the created training algorithm, and how memetic Pareto-based multiobjective learning can be used for MLP classifier training. Emphasis is put on formulating useful representations and objective functions for the task.