An interactive simple indicator-based evolutionary algorithm (I-SIBEA) for multiobjective optimization problems

. This paper presents a new preference based interactive evolutionary algorithm (I-SIBEA) for solving multiobjective optimization problems using weighted hypervolume. Here the decision maker iteratively provides her/his preference information in the form of identifying preferred and/or non-preferred solutions from a set of nondominated solutions. This preference information provided by the decision maker is used to assign weights of the weighted hypervolume calculation to solutions in subsequent generations. In any generation, the weighted hypervolume is calculated and solutions are selected to the next generation based on their contribution to the weighted hypervolume. The algorithm is compared with a recently developed interactive evolutionary algorithm, W-Hype on some benchmark multiobjective optimization problems. The results show signiﬁcant promise in the use of the I-SIBEA algorithm. In addition, the performance of the algorithm is demonstrated using a human decision maker to show its ﬂexibility towards changes in the preference information. The I-SIBEA algorithm is found to ﬂexibly exploit the preference information from the decision maker and generate solutions in the regions preferable to her/him.


Introduction
Industrial optimization problems often involve multiple conflicting objectives, which usually have multiple Pareto optimal solutions with different trade-offs. Different methods have been proposed in the literature (see e.g. [13]) and evolutionary multiobjective optimization (EMO) algorithms [5,6] have often been applied to solve multiobjective optimization problems and find an approximation of the Pareto front consisting of all the Pareto optimal solutions. However, finding an approximation of the Pareto front is not easy, especially when objective and constraint functions are computationally expensive.
When EMO algorithms are used to find an approximation of the Pareto front, a human decision maker (DM) who is an expert in the domain of the problem is supposed to choose one among several nondominated solutions for implementation or further evaluation. Such an approach is often termed as a posteriori approach in multiobjective optimization [13]. Since finding a good approximation of the Pareto front is often difficult, especially when more than two computationally expensive objectives are involved, it is practical to approximate a region of the Pareto front that is of interest to the DM. At least two different approaches involving preference information have been considered in the literature: 1. a priori approaches where the DM initially expresses her/his preference information, which is subsequently used to find a set of solutions reflecting her/his preferences [7,9], and 2. interactive methods where the DM iteratively provides her/his preference information and drives the algorithm towards her/his preferred region(s) of the Pareto front [8,12,15].
We can easily incorporate DM's preference information in indicator based evolutionary algorithms. These algorithms have been proposed in the literature [3,4] to handle a large number of objectives and in this article we focus our attention on these algorithms. In them, a hypervolume of the dominated region of the objective space is used as the indicator of the quality of the approximation of the Pareto front due to the Pareto compliance of the indicator [16]. However, as the number of objectives increases, the calculation of the hypervolume gets extremely time consuming. Recently, a Monte-Carlo simulation based approach to calculate hypervolume has been proposed to speed up the calculation [2].
In this paper, we propose a new interactive preference based EMO algorithm called interactive simple indicator-based evolutionary algorithm (I-SIBEA) where different weights are associated with different regions of the Pareto front such that the importance given by the DM for different regions of the Pareto front can be altered. In the proposed algorithm, we extend the simple indicatorbased evolutionary algorithm (SIBEA) [16] to take into account the preference information of the DM iteratively and direct the search towards the preferred regions of the DM. Specifically, the DM is iteratively shown a set of nondominated solutions and asked to provide her/his preferences by classifying this set into preferred and/or non-preferred solutions. The weights of the preferred solutions in the weighted hypervolume calculation are subsequently altered such that their selection pressure is increased. Using preference information for both the preferred and non-preferred solutions simultaneously is a novel approach in preference based EMO algorithms and provides more flexibility to the DM in guiding the search.
The rest of the paper is organized as follows. In Section 2, we introduce the main concepts and discuss how the preference information is incorporated into the method. Then I-SIBEA algorithm is presented in Section 3 with detailed description. In Section 4, we present preliminary numerical experiments used to test the method. Finally, the conclusions are drawn in Section 5.
2 Main concepts and principles of utilizing preference information from the decision maker in I-SIBEA

Concepts and notations
We consider multiobjective optimization problems of the form [13]: For the simplicity of presentation, we assume that all the objective functions are to be minimized. If some objective function f i is to be maximized, it is equivalent to minimize −f i . The (nonempty) feasible region (set) S is a subset of the decision variable region n and consists of decision variable vectors x = (x 1 , . . . , x n ) T that satisfy all the constraints. The image of the feasible region S in the objective region k is called the feasible objective region (set) denoted by Z. The elements of Z are called feasible objective vectors denoted by f (x) or z = (z 1 , . . . , z k ) T , where z i = f i (x), i = 1, . . . , k, are the objective function values. An ideal objective vector z * ∈ k is determined by minimizing each objective function individually, that is z * i = minimize x∈S f i (x). We say that a vector z 1 ∈ k is said to weakly dominate a vector z 2 ∈ k and denoted by z 1 z 2 if and only if for all 1 ≤ i ≤ k: In this paper, we consider an interactive preference based EMO algorithm, wherein a DM iteratively provides her/his preference information as a set of preferred and/or non-preferred solutions. To emphasize solutions in the preferred region, the weighted hypervolume, I w H (A) is used, where A is the set of nondominated solutions in the objective space. The weighted hypervolume is defined as the integral over the product of the weight distribution function w(z) and the attainment function α(z) [16], that is, and A z represent that at least one element of A weakly dominates z ∈ Z.

Incorporating preference information into the algorithm
There are different ways to obtain preference information from the DM. In the preference based EMO algorithms literature where the hypervolume based selection criterion is used [3,4,16], as far as we know, only the preferred solutions are considered as the preference information from the DM. In the proposed I-SIBEA algorithm, we provide the flexibility to the DM to give her/his preferences by selecting preferred and/or non-preferred solutions among a set of nondominated solutions shown to her/him. For example, if the DM selects only preferred solutions, the rest of the solutions can be regarded as either non-preferred solutions or solutions with no preference information. However, in this study we consider them as non-preferred solutions. On the other hand, if the DM selects both preferred and non-preferred solutions, the rest of the solutions are regarded as solutions with no preference information.
It is often assumed that the DM has prior information about the preferred solutions before starting the solution process [3,16]. In the I-SIBEA algorithm, it is not assumed that the DM has some prior information about preferred and/or non-preferred solutions and that the DM is consistent during interaction. The DM iteratively gives her/his preference information, which is used by the I-SIBEA algorithm to focus its search towards solutions that lie in the preferred region. In what follows, we discuss how DM's preferences are incorporated in the I-SIBEA algorithm.
As mentioned in the introduction, the proposed algorithm extends SIBEA to consider the preference information of the DM. After a fixed number of generations of SIBEA, in the first interaction with the DM, a set A ⊂ Z of nondominated solutions (in the objective space) is shown to the DM. The number of solutions shown is a parameter that the DM can set. Next, we suppose that the DM selects preferred and non-preferred solutions from the set A. Therefore, the obtained preference information creates a partition of A into three non-overlapping subsets: After the partitioning of A into the three subsets, Z is partitioned into three regions based on the preferences from the DM. The regions are called dominated (Do), preferred (P r) and no preference information (In) and an example illustrating them is shown in Fig. 1 for a biobjective optimization problem. The shaded region in the Fig. 1 represents the infeasible region.
In what follows, three regions, Do, P r and In are defined based on preference information from the DM. The weight distribution function is then derived using hypervolume of these three regions which is incorporated into the algorithm to calculate the weighted hypervolume.
The region Do is the part of Z which is weakly dominated by at least one element of RA: The hypervolume µ(Do) and the weighted hypervolume w(Do) for region Do are calculated as The region P r is the part of Z which weakly dominates at least one element of AA: P r = {z ∈ Z | there exists z ∈ AA, z z} .
The hypervolume µ(P r) and the weighted hypervolume w(P r) for region P r are calculated as The region In is the remaining part of Z ( Fig. 1): The reason to partition Z into these three regions is to emphasize the solutions that lie in the preferred region (P r). There can exist several ways to implement this principle. In the literature [3,12,16], several weight distribution functions (e.g. stressing objectives with exponential weights, guiding single solutions with dirac-type weights etc.) are used to incorporate the DM's preference information in the solution process. We present here a uniform weight distribution as one of the possibilities. As the DM wants to avoid the non-preferred solutions, w(z) remains zero for the region Do. Therefore, we define the weight distribution function as: for all z ∈ P r (so that Z w(z)dz = 1). This weight distribution function is then used to calculate the weighted hypervolume in the subsequent generations and the solutions are selected based on their contribution to the weighted hypervolume. In this way, the preference information from the DM is incorporated into the algorithm. In what follows, the I-SIBEA algorithm is presented with detailed description.

Interactive simple indicator-based evolutionary algorithm (I-SIBEA)
The main motivation of the proposed algorithm is to direct its search process towards solutions that lie in the preferred region defined by the DM's preferences.
To do this, solutions having a large contribution to the weighted hypervolume are selected and solutions having the smallest contribution to the weighted hypervolume are removed from the population after every generation. This criterion of selecting solutions is common among hypervolume based search algorithms [16,17]. In the proposed I-SIBEA, in addition to hypervolume based selection criterion, different preference information from the DM is incorporated into the algorithm. In this algorithm, the DM gives her/his preference information by selecting preferred and/or non-preferred solutions. This preference information guides the algorithm to focus its search direction for solutions that lie in the preferred region. The algorithm is presented in the I-SIBEA algorithm and we discuss now the step by step procedure of the algorithm. Initially, a population P of individuals of size N P is created randomly in step 1. Next in step 2, crossover and mutation operators are used to create an offspring population Q of the same size (N P ). The parent and the offspring populations are combined P := P + Q and then environmental selection is used to select individuals as mentioned in step 3 of the I-SIBEA algorithm. Nondominated sorting [14] is used to rank the individuals of the combined population and different fronts F i , i = 1, 2, . . . are identified. These fronts are added to an empty set P 1 as long as the size of the population of P 1 becomes equal to or exceeds N P . If the size of P 1 is N P , the population for next generation is set as P := P 1. Otherwise, the set of individuals in the worst rank front in P 1 is identified and denoted by P . To remove solutions from the worst rank front so that the population size of P 1 does not exceed N P , the usual hypervolume based selection is used. For each solution z ∈ P , the loss in the hypervolume d(z) = I(P ) − I(P \ z) is determined, where I is the hypervolume indicator and represented as the hypervolume or the weighted hypervolume for a given set. The solution for each solution z ∈ P belonging to the regions Do and In.
d. Remove the |P 1| − N P solutions from P with the smallest loss d(z) (ties are broken randomly) and include the remaining solutions of P into P 1. Set P := P 1.
Step 4: If m ≥ N G or m ≥ N then go to step 5. Otherwise set m := m + 1 and go to step 2. with the smallest loss is removed until the size of the population does no longer exceed N P and the population is set as P := P 1 for the next generation. After a fixed number of generations, N I in step 4, the DM interacts with the algorithm. In step 5, a fixed number of nondominated solutions DA ⊆ A (input from the DM) is identified and then shown to her/him in step 6, where A is the set of nondominated solutions. There exist different ways to select the fixed number of solutions from A and we use k-means clustering [11] in this study. Here, a solution is selected randomly from each cluster and shown to the DM. In step 7, if the DM wants to quit, s(he) selects the most preferred solution f * from DA. The final solution is obtained by projecting f * to the actual Pareto front by optimizing an achievement scalarizing function (ASF) [13], that is by solving the problem where ρ > 0 is the augmentation coefficient which takes a small positive value e.g. 10 −6 . The weight vector w i = , respectively. One of the advantages for using an ASF is that the optimal solution of an ASF is always Pareto optimal [13]. Therefore, optimizing an ASF ensures that final solution is locally Pareto optimal. Since we assume that less is preferred to more for the DM, the projected solution is at least as preferred to the DM as the solution s(he) selected. We can utilize an equivalent differentiable formulation of ASF when all the objective functions are differentiable by adding extra real valued variable, δ and k new constraints [13] In addition to termination by the DM, the solution process is ended if the maximum number of generations (NG) is reached. In that case, solutions DA ⊆ A are shown to the DM and (s)he is asked to select the the most preferred solution (f * ). This solution is then projected to the Pareto front and the final solution is obtained by solving problem (2) or (3) with a single objective optimization method appropriate to the characteristics of the problem in question. A local search method can be used since the evolutionary algorithm is supposed to take care of the global search. If the termination criterion is not met, the DM is then asked in step 8 to select preferred (AA) and non-preferred solutions (RA) from DA to get the three non-overlapping subsets AA, RA and IA. The three regions Do, P r and In are then derived using this preference information to get the weight distribution function. This completes one interaction with the DM. The weight distribution function is then used to calculate the weighted hypervolume as the selection criterion in the subsequent generations.
In the next generation (after the first interaction), the offspring are created again in step 2 and other steps are then followed. If the population size exceeds N P in P 1, solutions z ∈ P belonging to region P r are identified and denoted by P P r . The set P 1 is then updated as P 1 := P 1 \ P + P P r . If the size of the population of P 1 exceeds N P , the usual hypervolume based selection is used to remove the solutions from P belonging to region P r. Otherwise, the solutions z ∈ P belonging to regions Do and In are added to P 1. If the population size exceeds N P , the weighted hypervolume based selection is used to remove solutions z ∈ P belonging to regions Do and In. This principle of selecting individuals emphasizes solutions in the preferred region. The regions Do, P r and In are updated after every interaction after the DM has classified DA into AA and RA. In this way, the DM gives her/his preference and the weights of the solutions in the weighted hypervolume calculation are altered in such a way that solutions in P r are emphasized and solutions in Do are avoided.
In the proposed algorithm, the DM has the freedom to choose the number of times (s)he wishes to interact with the algorithm. From this input, the maximum number of generations (N G) is uniformly divided by H to get the number of generations before each interaction. For example, the first interaction will take place after N I = N G/H generations and the second interaction will take place after N = N I + (N G − N I)/(H − 1) generations. Even though, the maximum number of interactions H is given by the DM in the beginning, the DM is free to change it during any interaction. However, this is not presented in the algorithm but if the DM gives her/his updated number of interactions, the remaining generations can be uniformly divided accordingly. In the next section, the algorithm is tested using some benchmark problems.

Numerical experiments
The I-SIBEA algorithm was tested on standard benchmark problems [5] with 2-3 objectives and 7-11 decision variables. Firstly, we compare I-SIBEA against a recently proposed interactive weighted hypervolume based algorithm called W-Hype [4]. One of the main differences between W-Hype and I-SIBEA is that W-Hype considers information for only preferred solutions while I-SIBEA considers information for both preferred and non-preferred solutions. To enable easy comparison, we use the same set of test problems i.e. DTLZ2, ZDT4 and DTLZ1 and the same criterion of [12] to get the number of generations before each interaction as used in W-Hype. The parameter values used in I-SIBEA are provided in Table 1. In all three problems, polynomial mutation (distribution index is 20 and probability of mutation is 1/number of decision variables) and simulated binary crossover (distribution index is 20 and probability of crossover is 0.9) were used. While testing the algorithm for these test problems, we replaced the DM by a weighted Chebyshev function max i=1,...,k i )] at each interaction step with z * i as the ideal objective vector. The weight vector (w 1 , . . . , w k ) T is assigned to each objective function and used to describe the DM's preferences. After each interaction step, the solution that minimized the weighted Chebyshev function was considered as the preferred solution (AA) and the rest of the solutions were considered as non-preferred solutions (RA) i.e. it was assumed that there were no solutions with no preference information (IA). This setting was used to be able to compare I-SIBEA with W-Hype (where this setting had been used). To measure the performance of the proposed algorithm, mean, standard deviation, absolute deviation and optimal Chebyshev function value were calculated after a maximum number of function evaluations. In the three tables reporting the results of I-SIBEA and W-Hype, the values of performance criteria are written in bold face if the difference was greater than 0.001. The algorithm was also tested by varying the maximum number of interactions. The comparison of the present algorithm with W-Hype for DTLZ2 is shown in Table 2. The results show that W-Hype performed better than I-SIBEA for H = 2. Otherwise, equivalent results were obtained for H = 4, 6, 8 and I-SIBEA needed fewer function evaluations when compared with W-Hype. The total number of function evaluations used for this problem by W-Hype and I-SIBEA were 25, 000 and 20, 000, respectively. In addition, better results were obtained by both algorithms with increase in H. We also observed that, after a certain number of generations, the mean of the weighted Chebyshev function did not change for H = 4, 6 and 8 which indicates the convergence of the algorithm. Moreover, there was no considerable difference in the results for H = 6 and H = 8 in I-SIBEA and, therefore, we restricted ourselves to H = 4 and 6 when solving the following two problems.
In case of the ZDT4 problem, the weight vector w = (0.5, 0.5) T was used in the weighted Chebyshev function to identify the preferred solution (AA). The comparison of I-SIBEA with W-Hype is shown in Table 3. I-SIBEA performed better than W-Hype both in terms of results obtained and the number of function evaluations used. I-SIBEA used 40,000 function evaluations and converged in 50% fewer function evaluations as comparison with W-Hype. Next, we tested I-SIBEA on a 3-objective DTLZ1 problem. The weight vector w = (0.7, 0.2, 0.1) T was used in weighted Chebyshev function to identify the preferred solution (AA). Table 4 shows the results of this study. For this problem as well, equivalent results were obtained in 62.5% fewer function evaluations. I-SIBEA used 120,000 function evaluations in contrast to W-Hype which used 320,000 function evaluations. In all three problems, better or equivalent results were obtained but I-SIBEA always consumed fewer function evaluations. The reason for fewer function evaluations using I-SIBEA can be attributed to the use of preference information of both preferred and non-preferred solutions when compared to W-Hype, where only preferred solutions were considered as the preference information from the DM. This extra information on non-preferred solutions can help the algorithm to avoid solutions in the corresponding regions in subsequent generations and converge faster to solutions in the preferred region. In addition, the DM has more options of how to express one's preferences.
To show the flexibility of the proposed algorithm, a fourth case study was performed on the ZDT4 problem (as it is easy to visualize a biobjective optimization problem), where a DM was involved. In the beginning of the solution process, the DM was asked to provide the maximum number of interactions i.e. how many times he wanted to interact with the algorithm. In addition, the flexibility is given to the DM to change the number of nondominated solutions (DA, default is maximum 5) he wanted to see during interaction. In this study, the maximum number of generations (N G) was uniformly divided into 6 times for the interaction with the DM as mentioned in the I-SIBEA algorithm. The other parameters used for this problem are shown in Table 5. The results of this study are shown in Fig. 2. The first scatter plot shows the nondominated solutions (A) before the first interaction. For the first interaction, the DM wanted to see 5 nondominated solutions and k-means clustering was used to get them. The DM then selected the preferred (AA) and non-preferred solutions (RA) which are shown in the second scatter plot. The solutions obtained before the second interaction are also plotted in the same plot to show the search direction of I-SIBEA. The solution process was then continued until the second interaction. In Fig. 2, the preferred and non-preferred solutions are shown for five interactions. In this study, the DM changed his preferences in the subsequent interactions or in other words, the DM was not consistent with his preferences as shown in Fig. 2. The I-SIBEA algorithm was found to exploit the preference information provided by the DM and generate solutions in the regions preferable to the DM. This shows that the algorithm is flexible to changes in the preferences and can find solutions in the preferred region. After completion of the maximum number of generations, the DM interacted again (6 th time) and selected the most preferred solution. This solution was then projected to the Pareto front by solving problem (3). We used here fmincon from MATLAB optimization toolbox to solve problem (3). In this study, f 1 = 0.096327 and  The proposed algorithm directed its search towards the DM's preferences and also changed its search direction with changes in the preferences. Therefore, the algorithm emphasized solutions in the preferred region and was flexible to the DM's preferences. In addition, the optimality of the chosen preferred solution was guaranteed (at least locally).

Conclusions
In this paper, an interactive simple indicator-based evolutionary algorithm called I-SIBEA is proposed. In this algorithm, the DM's preferences are taken into account in terms of preferred and/or non-preferred solutions. The information for non-preferred solutions helps the algorithm to avoid such solutions in subsequent generations. In this algorithm, the DM can decide how many times s(he) wants to interact with the algorithm and how many solutions s(he) wants to compare while interacting. Therefore, the DM does not need to compare more solutions than (s)he is able to consider at a time. In addition, the algorithm is flexible towards changes in the preferences from the DM. Hence, the algorithm does not assume that the DM has some prior information about preferred and/or nonpreferred solutions. Furthermore, unlike typical evolutionary algorithms that cannot guarantee optimality, at least local Pareto optimality of the final solution is guaranteed as it is projected to the Pareto front by optimizing an achievement scalarizing function.
We have compared the performance of I-SIBEA with the W-Hype algorithm. I-SIBEA performed equivalent or better in terms of results obtained but needed fewer function evaluations to get the final solution. In addition, the potential of the algorithm was demonstrated using a human DM to show its flexibility towards changes in the preferences. As indicator based algorithms can handle large numbers of objectives, therefore, next we plan to test the algorithm for more than three objectives and apply the DM's preferences in different ways. Additionally, we plan to develop a GUI which can be utilized with to solve real world multiobjective optimization problems.