Survey of methods to visualize alternatives in multiple criteria decision making problems

When solving decision problems where multiple conflicting criteria are to be considered simultaneously, decision makers must compare several different alternatives and select the most preferred one. The task of comparing multidimensional vectors is very demanding for the decision maker without any support. Different graphical visualization tools can be used to support and help the decision maker in understanding similarities and differences between the alternatives and graphical illustration is a very important part of decision support systems that are used in solving multiple criteria decision making problems. The visualization task is by no means trivial because, on the one hand, the graphics must be easy to comprehend and not too much information should be lost but, on the other hand, no extra unintentional information should be included. In this paper, we survey and analyze different ways of visualizing a small set of discrete alternatives graphically in the context of multiple criteria decision making. Some of the ways discussed are widely used and some others deserve to be brought into a wider awareness. This survey provides a starting point for all those who deal with multiple criteria decision making problems and need information of what kind of visualization techniques could be put to use in order to support the decision maker better.


Introduction
Multiple criteria decision making means making the best possible decisions with respect to several conflicting criteria. In such problems, it is impossible to find a solution that could simultaneously give the optimal value to all the conflicting criteria, but typically there exist many so-called Pareto optimal or efficient solutions and the best one among them is to be found by involving a human decision maker and preference information of some form. This means that the most preferred solution is identified based on the preferences of the decision maker and the decision maker needs support in this task.
Multiple criteria decision making (MCDM) problems involving multiple conflicting criteria can be divided into two distinct types depending on the properties of the feasible solutions, that is, whether we have discrete alternatives or alternatives determined by continuous variables. In the first type, referred to as multiattribute decision analysis, the set of feasible alternatives is discrete, predetermined and finite (like when choosing a car or an apartment). In the second type, in multiobjective optimization problems, the feasible alternatives are not explicitly known in advance. Instead, they are represented by (continuous, binary or integer-valued) decision variables restricted by constraints (like when choosing the dimensions of a container or how assets should be invested). In what follows, we call such problems continuous (even though they could be e.g. combinatorial), as for such problems one, actually, has infinitely many solutions satisfying the constraints. Many solution methods for continuous multiobjective optimization generate discrete sets of alternatives during the solution process and ask the decision maker to select the most preferred one among them. Such a question is often a part of an interactive solution process (Miettinen 1999;Miettinen et al. 2008), as, for example, in the surrogate worth trade-off method (Chankong and Haimes 1983, pp. 371-379), the Geoffrion-Dyer-Feinberg method (Geoffrion et al. 1972), the Tchebycheff method (Steuer 1986, pp. 419-455), the reference point method (Wierzbicki 1982), the light beam search (Jaszkiewicz and Slowiński 1999) and the NIMBUS method (Miettinen 1999;Mäkelä 2000, 2006).
It is important in both discrete and continuous cases to help the decision maker in the task of comparing different alternatives consisting of criterion values. Usually, the comparison problem is difficult to be solved directly. In these cases, we need tools that describe different features and elements involved in a simple but rigorous way. One possibility to be reckoned with is to use visualization of the alternatives. It can be used in exploring data to gain insight into the data itself as well as understanding of the underlying phenomena and the problem solving process. Besides, when data and relationships therein are represented in the form of graphs, they are easier to be comprehended and remembered. Graphs can also provide a general view and a comprehensive picture of the data. Thus, we can say that visualization is more than only graphic representation.
When solving continuous problems, visualization tools are needed, for example, with so-called a posteriori methods (see, for example, Miettinen 1999, 2008 andreferences therein) that generate a representation of the set of Pareto optimal solutions. More recent developments include hybrids of interactive and a posteriori methods where only a part of the Pareto optimal set is approximated (see, for example, Klamroth and Miettinen 2007). After the Pareto optimal set (or a part of it) has been generated, it is presented to the decision maker, who selects the most preferred one among the alternatives. Here, a valid question is how to present or display the alternatives to the decision maker in an effective way. The same goal of finding the Pareto optimal set motivates, for example, the methods of evolutionary multiobjective optimization (see, for example, Coello Coello et al. 2007 and references therein).
Large sets can be handled with different tools like data analysis (Chambers et al. 1983;Gnanadesikan 1977;Guerrero 2010) in order to decrease the amount of information available. Five ways of decreasing the number of alternatives are presented in Graves et al. (1992). Among others, the number of alternatives can be decreased using clustering (Malakooti and Raman 2000). Further tools developed for the reduction of dimensionality can be found in references devoted to multivariate analysis, for example, (Gnanadesikan 1977). In case of many alternatives, one can also locate them in the same graph and allow the decision maker to define bounds or ranges that are interesting to her/him, like in Trinkaus and Hanne (2005). In this dynamic way, uninteresting alternatives are filtered away and individual alternatives become gradually identifiable. One can even approximate the whole set of Pareto optimal solutions and use approaches based on decision maps resembling topographic maps like in Lotov et al. (2004) and Lotov and Miettinen (2008) and references therein. However, in this presentation, we concentrate on visualizing relatively small sets of alternatives (typically less than ten).
As computers and monitors have developed, visualization has increased in importance and has also become easier to produce. For example, many spreadsheet programs contain rather versatile possibilities of visualization. However, not all of them are necessarily useful for a discrete set of alternatives in multiple criteria decision making. The same can be said about several techniques for generating graphical displays in the analysis of multivariate data that are briefly surveyed in Chambers and Kleiner (1982). These treatments do not necessarily consider the special properties of data related to multiple criteria decision making when the purpose is to compare a set of alternatives. Thus, not all of the general visual display tools of quantitative data (like in Tufte 1983) are applicable. In Chambers et al. (1983), a summary of references related to plotting multidimensional data is given with short descriptions. On the other hand, for example, the annotated bibliography on scientific visualization in Voegele (1990) contains many topics not relevant for visualizing alternatives, including ray tracing and volume rendering. Tools developed for identifying outliers, patterns or trends in data are not necessarily applicable to comparing criterion values of alternatives. That is why we need special approaches.
Short summaries of visualization tools for alternatives related to multiple criteria are given, for example, in Klimberg (1992), Klimberg and Cohen (1999) and Schilling et al. (1983). However, they mostly concentrate on one or two tools and only mention some of the others. An overview related to multiple criteria decision making is given in Korhonen and Wallenius (2008) in brief and in Vetschera (1994) in German. On the other hand, some approaches originally developed, for example, for clustering can be applied in visualizing alternatives but they have not been widely brought to the awareness of people dealing with multiple criteria decision making problems.
In this paper, we treat visualization of alternatives together with related matters with the aim of supporting comparison in multiple criteria decision making. We concentrate on general visualizations and do not pay attention to various application-specific graphs that may help the decision maker of some particular application field. The goal is to serve all those who need to compare different alternatives as a part of solution methods and need an overview of the existing possibilities. That is why we here bring together many existing approaches for visualizing alternatives and give a review of them. The potential and restrictions of different graphics tools are treated and some clarifying figures are enclosed. This presentation is based on the background laid in Miettinen (1999Miettinen ( , 2003.
The rest of this paper is organized as follows. In Sect. 2, we briefly describe the basic concepts of multiple criteria decision making as the basis of our consideration. After that, in Sect. 3, we introduce many different visualization techniques that can be used for the purpose of comparing a small set of alternatives in multiple criteria decision making. We classify the techniques according to their main characteristics and illustrate many of them with examples. Section 4 is devoted to discussion related to visualization in general, the role of tables versus graphs as well as using colors. Finally, the paper is concluded in Sect. 5.

Basic concepts of multiple criteria decision making
In what follows, we assume that we have a set of alternatives that we wish to visualize but we do not fix the form of the underlying multiple criteria decision making problem. Thus, the problem may have a set of discrete alternatives describing the performances of different criteria. Alternatively, we may have a set of solutions that has been generated for a continuous problem during some (e.g., interactive) solution process. In any case, we assume that we have k criteria and a finite set of m alternatives. The alternatives consist of criterion values z i = (z i 1 , z i 2 , . . . , z i k ) T for i = 1, . . . , m, and the problem is how to display the m × k-dimensional data.
We assume here that we want to minimize all the criteria simultaneously. Thus, lower criterion values are preferred to higher. Because of the contradiction between the criteria and their possible incommensurability, it is not possible to find a single solution that would be optimal for all the criteria simultaneously. Yet, we can define a subset of solutions that can be seen optimal. This notion is usually called Pareto optimality or efficiency. A feasible criterion vector z * is Pareto optimal if there does not exist another feasible criterion vector z such that z i ≤ z * i for all i = 1, . . . , k and z j < z * j for at least one index j. In what follows, we assume that the set of alternatives to be visualized consists of Pareto optimal solutions. Because all the Pareto optimal solutions are equally good from the mathematical point of view, we need a decision maker to identify the best of them. The decision maker is a person (or a group of persons) who is supposed to have good insight into the problem and who can express preference relations between different (Pareto optimal) solutions. Provided that the problem is correctly specified, the final solution of a rational decision maker is always Pareto optimal (Miettinen 1999). Usually, the decision maker is responsible for the final solution and solving a multiple criteria decision making problem calls for the co-operation of the decision maker and an analyst. A person or a computer program responsible for the mathematical side of the solution process is here referred to as an analyst. The analyst generates information for the decision maker to consider and the decision maker selects a solution according to her/his preferences. Let us mention that sometimes the preferences of the decision maker are expressed in the form of reference points. They consist of desirable values, so-called aspiration levels, for each criterion.
In the introduction, we mentioned several interactive multiobjective optimization methods that involve a phase where the decision maker must compare alternatives which the method has generated and then select the most preferred one before the solution process can continue. Steps of most interactive methods can roughly be seen as follows : (1) Initialization. (2) Generation of a Pareto optimal starting point. (3) Asking the decision maker to specify preference information to direct the solution process. (4) Generation of new Pareto optimal alternatives according to the preferences and showing it/them to the decision maker. If several alternatives were generated, asking the decision maker to select the best alternative so far. (5) Termination, if the decision maker wants to. (6) Otherwise the solution process continues from step (3). In the interactive methods listed, visualization is a welcome way to support the decision maker in step (4). Furthermore, in Luque et al. (2011), a global formulation is given for interactive multiobjective optimization, where various different methods are incorporated. In the flowchart of the global solution scheme, steps where alternatives must be compared (and visualizations can help the decision maker) are clearly indicated. In addition, examples of multiattribute decision analysis methods where visualizations are used will be mentioned in the following section.
Often, it is good to know the ranges of the values the criteria can obtain in the Pareto optimal set. A vector consisting of the best (that is, smallest) values of each criterion is called an ideal criterion vector z ∈ R k . In continuous problems, the components z i are obtained by minimizing each of the criteria individually subject to the constraints. Even though the ideal criterion vector is not attainable, it can be considered as a goal, something to go for. From the ideal criterion vector we obtain the lower bounds of the Pareto optimal set for each criterion.
In discrete problems, the upper bounds of the Pareto optimal set, that is, the components of a nadir criterion vector z nad , can easily be identified. The task is much more difficult in continuous problems. However, they can be estimated in different ways. For more details, see, for example, (Deb et al. 2010;Miettinen 1999) and references therein.
In many occasions it is advisable to re-scale, that is, to normalize the criteria so that the criterion values are of approximately the same magnitude. If the ideal criterion vector and a good enough approximation to the nadir criterion vector are known, we can divide the difference of each criterion value and the ideal criterion value by z nad i − z i for i = 1, . . . , k. Sometimes the ideal criterion values are replaced by a so-called utopian values, which are strictly lower that the ideal criterion values. In this way one can avoid dividing by a small number if some criterion has a small range (Miettinen 1999;Steuer 1986). A simpler alternative for continuous problems for normalizing the criterion values is to divide each criterion by its (nonzero) ideal criterion value. This is not as exact as the previous methods but does not necessitate information about the nadir criterion vector.
Note that it is usually advisable to use normalized criterion values only in calculations and to display restored criterion values in the original scales to the decision maker. In this way, the different scales do not confuse computation but understandable criterion values are displayed to the decision maker. However, it is often worthwhile to use normalized values in visualizations if the ranges of the criteria are very different. Otherwise, it may be difficult for the decision maker to identify similarities and differences.
Let us finally point out that many of the visualization techniques to be discussed in the next section presume information about the ranges of the criterion values in the Pareto optimal set. In other words, some of the illustrations can only be generated if ideal and nadir criterion vectors (or their accurate enough approximations) are available.

Visualization
Visualization plays an essential role when designing modern software. As computers have developed, more attention has been paid towards the role and the possibilities of computer graphics in building human-computer interfaces. When developing graphical illustrations, many challenges exist. Information is encoded when a graph is constructed and the graphical perception of the decision maker is vital when this encoded information is decoded (see Cleveland 1994). If the decoding fails, the whole illustration is worthless.

Introduction to visualization in multiple criteria decision making
In multiple criteria decision making, the ultimate goal of visualization is to enable the decision maker to gain more and more understanding of the problem and new insight into the data about the alternatives and the underlying phenomena. The point is not only in helping decision makers in understanding subtle relationships but also in general impressions. In other words, good visualization enables graphical exploration of the problem and helps the decision maker to gain better insights into the problem and the different alternatives to be compared. It is important that the decision maker can differentiate between the alternatives, their essential similarities and differences when comparing them before making the decision. The graphics must be clear as well as easy to comprehend and interpret by the decision maker. On the one hand, not too much information should be allowed to be lost and, on the other hand, no extra unintentional information should be included in the presentation. To provoke intuitive decision making, analysts should provide information in rich and multidimensional graphic terms. One should avoid forcing decision makers to deal with visualization tools that are not intuitive to them. Non-intuitive tools may frustrate decision makers and lead to the rejection of these tools. Our inspiration here is the one stated in Hamming (1973): "The purpose of computing is insight, not numbers." Utilizing visualization does not mean that the limits on human information processing capacity are transcended. This means that there is no sense in trying to offer too much information (e.g., too many alternatives) for evaluation, no matter how clear the visualizations are. Several psychological tests are summarized in Miller (1956) to prove that the span of absolute judgment and the span of immediate memory in human beings is rather limited. We cannot receive, process or remember large amounts of information. The magical number seven plus or minus two appears in several tests and in several ways. However, no number can be regarded as an absolute limit. Everything depends on the circumstances. Still, the findings in Miller (1956) are to be kept in mind when deciding the number of alternatives to be presented to the decision maker or the number of criteria to be treated (if these can be affected). After all, the human visual system is the same as it was decades ago.
As stressed in Kok (1986), experiments in psychology indicate that the amount of information provided to the decision maker has a crucial role. If more information is given to the decision maker, the percentage of the information used decreases. In other words, more information is not necessarily better than less information. More information may increase the confidence of the decision maker in the solution obtained but the quality of the solution may nonetheless be worse. It may, for example, be advisable to hide some of the information and show it only when the decision maker requests it.
In what follows, we discuss several possibilities of visualizing a given set of alternatives, that is, criterion vectors related to some multiple criteria decision making problem. We give examples of many visualization techniques using the data of Table 1 with three alternatives involving four criteria. The ranges in the Pareto optimal set are also shown in the table.
Visualization techniques can be classified in many ways and there is no unique way to do it. Here we classify them mainly according to the principles they utilize into six classes: commonly known techniques involving bars, scatter plots or value paths, techniques using circles and polygons, icons, techniques based on hierarchical clustering, projection-based techniques and other techniques.
After we have described various visualization possibilities, we summarize some of their properties or limitations paying attention to the maximum number of criteria and maximum number of alternatives they can incorporate as well as how sensitive they are to the order of the criteria, whether information about ranges is necessary,  whether they are best suited for pairwise comparison and whether they can incorporate preference information in the form of aspiration levels.

Bar charts
A widely used general mode of visualization is to use bar charts or histograms. This means that a group of bars represents the alternative values of a single criterion, as in Fig. 1. The bars of the same color are related to one alternative. Separate ranges for criteria are possible as well. Naturally, the roles of the alternatives and the criteria can be interchanged so that bars are grouped according to alternatives instead of criteria (as in Fig. 2). The bars can also be grouped in other ways, see Jarvenpaa (1989). Bar charts take horizontally a lot of space in case of several criteria and alternatives. Thus, it is not convenient to display a lot of information simultaneously. In this case, the bars may be located, for example, in three dimensions, as in Fig. 3. Unfortunately, this brings along the problem of selecting the angle of view so that bars are not hidden behind each other.
Another possibility is to locate the bars in a horizontal instead of a vertical direction. It is naturally possible to use different groupings with horizontal bars as well, see Figs. 4 and 5. For minimization problems, it is natural that to have the shorter the bar, the better the corresponding criterion value. If the data to be visualized contains both criteria to be minimized and maximized and the ranges of criteria are know, one can locate the roots of the bars related to criteria to be maximized on the right. In this setting, the interpretation is the same: the shorter the bar, the better the corresponding value. This is used, for example, in the IND-NIMBUS optimization software (Miettinen 2006). Notice that the order of the criteria affects the general appearance of the chart in Figs. 2 and 5. Flexibility is increased if the decision maker can affect the order of alternatives in the visualizations. The decision maker may also appreciate if the ranges of each criterion can be seen, for example, as shadows in the bars.
Bar charts are called by the name profiles in Kleiner and Hartigan (1981). This term refers to the tops of the bars: if the tops are connected with a polygonal line, we have a value path in question (to be discussed later in this section).
In MS-Excel, it is possible to use cylinders, cones or pyramids instead of ordinary columns in bar charts. As far as multiple criteria decision making is concerned, bar charts are used, for example, in the interactive multiobjective optimization system WWW-NIMBUS Mäkelä 2000, 2006) operating on the Internet at http://nimbus.it.jyu.fi/ and its more advanced version IND-NIMBUS (Miettinen 2006). Instead of using separate ranges for the criteria it is possible to provide bar charts using both absolute and relative scales. This is advisable in particular if the ranges of the criteria vary widely. This option is also available in WWW-NIMBUS.

Scatter plots
According to Chambers and Kleiner (1982), multivariate plotting methods can be divided into two classes: symbolic plots and extensions of scatter plots. A scatter plot is a projection of the data (describing the alternatives) into a two-dimensional space, that is, a plane, where the coordinates can be any two of the criteria. Thus, the scatter plot matrix consists of panels each representing one pair of criteria. The dimension of the square matrix is the number of criteria. Different alternatives can be denoted by different symbols or colors. As can be seen in Fig. 6, each pair is graphed twice with the roles interchanged. This means that either the lower or the upper triangle could be dropped without losing any information. However, displaying the whole matrix makes it easier to compare the criterion values. One can conveniently measure the performance of one criterion against the other criteria by having a look at one column or one row at a time. Each criterion can naturally have a range of its own in the panels. Scatter plot matrices are available in many statistical packages. The scatter plot matrix, also known as the generalized draughtsman's view (Tukey and Tukey 1981b), generalized draftsman's display (Chambers et al. 1983) or the generalized drafter's plot (Ware 2000), is described in Cleveland (1994) as "one of the best visualization ideas around." With three criteria, this is called a draughtsman's view (Tukey and Tukey 1981b) or draftsman's display (Chambers et al. 1983).
With scatter plots, the performance of only two criteria can be studied at a time. In principle, a third criterion can be displayed in a scatter plot using some symbol and the corresponding criterion values can be represented by changing, for example, the size of that symbol. This can be called a symbolic scatter plot (Chambers et al. 1983). Alternatively, we can imagine the three-dimensional data related to three criteria as a three-dimensional scatter plot. It can then be partitioned into subsets based on the intervals of the third criterion value. In other words, the three-dimensional region is sliced into several displays. This can be called a casement display (Chambers et al. 1983).
If we have exactly four criteria to be visualized, the criterion values can be displayed using linked views. This means that two scatter plots are located near each other representing each two criterion values. Then, the points related to the same alternative are connected with a line, see Tukey and Tukey (1981b). Whether this view is more confusing than helpful is up to the decision maker and depends, naturally, on the number of alternatives.
The disadvantage of scatter plot matrices is the fact that data patterns involving more than two criteria are not easily in view. In other words, there may be data patterns that are visible only when three or more criteria can be seen simultaneously. It is suggested in Ware (2000) that color mappings can be used in scatter plot matrices to make visible data patterns involving more than two criteria. This means, for example, that for five criteria the first two criteria are mapped and displayed in the normal way but the symbols denoting the alternatives contain varying amounts of red, green and blue based on the criterion values of the third, the fourth and the fifth criterion, respectively. However, it is pointed out in Ware (2000) that even though this color-mapped scatter plot may make it easier to identify clusters and patterns, the interpretation may be difficult.
The matrix is not easy to interpret if the number of criteria is too large (according to Everitt 1978, around ten). Because scatter plot matrices become less informative with more criteria, it may be advisable not to display all the possible two-dimensional projections but to generate random views or a set of pre-chosen direct views, see (Tukey and Tukey 1981a,b). The first reference also describes several techniques related to using the data itself to help in selecting desired views of higher dimensions. In Tukey and Tukey (1981b), different possibilities for how to display the projections of three and four dimensions are also given.
The difficulty of following how some alternative or a subset of alternatives performs in different panels of the scatter plot matrix is discussed in Becker and Cleveland (1987) and so-called brushing scatterplots are suggested to be able to link alternatives across different panels. In practice, the decision maker can select any group of alternatives to be highlighted in all the panels. Several different operations are suggested for this purpose. Furthermore, the decision maker can delete alternatives instead of highlighting them or ask for the names of the selected alternatives to appear.

Value paths
One more widely used way of representing sets of criterion vectors is to use value paths, as suggested, for example, in Geoffrion et al. (1972) and Schilling et al. (1983). This means that horizontal lines of different colors or of different line styles represent the values of the criteria in different alternatives. In other words, one line is associated with one alternative. These polygonal lines are also known as linear profiles (Mezzich and Worthington 1978) or line graphs (Wickens 1992).
Value paths are depicted in Fig. 7. The bars in the figure show the ranges of the criteria in the Pareto optimal set. If the ranges are known, they give additional information about the possibilities and the limitations of the criteria. Note that each criterion can have a scale of its own in the bars, if necessary. Different scales for the criteria are demonstrated in Törn (1983). Value paths are a recommendable and effective method of visualization because they are easy to interpret without overloading the decision maker. For example, it is easy to distinguish non-Pareto optimal alternatives if they are included (by the fact that the corresponding value paths do not intersect some of the others). Because the representation is rather compact, the number of criteria can be increased to a certain degree. Having too many alternatives, that is, paths in the same figure makes interpretation and comparison difficult. To avoid this, it is possible to locate each path individually, that is, to have as many charts as alternatives. In Chambers et al. (1983), this is called a profile symbol plot. In this case, a separate key showing the assignment of the criteria to the bars can be used so that they do not have to be repeated in each chart. Note that the appearance of the visualization changes if the order of the criteria is altered.
Value paths are used, for example, in the visual interactive sensitivity analysis system VISA (Belton and Vickers 1985) and in the WWW-NIMBUS and IND-NIMBUS systems Mäkelä 2000, 2006) and (Miettinen 2006), respectively, (with both absolute and relative scales for the same reason as with bar charts). Value paths (i.e., lines) are also available in MS-Excel.
The roles of the lines and the bars can be interchanged so that bars denote alternatives and lines represent criteria, as in Fig. 8. In this case, possible different scales of the criteria have to be interpreted differently (see, e.g., Hwang and Masud 1979, p. 109). This reversal of roles has been utilized, for instance, in Geoffrion et al. (1972) in the Geoffrion-Dyer-Feinberg method and in the first implementations of the discrete reference direction approach VIMDA (Korhonen 1986). The idea in VIMDA is that when the user horizontally moves the cursor to a bar representing an alternative, the corresponding numerical criterion values are displayed.

Multiway dot plots
If we arrange the bars of the bar chart alternative-wise and locate them in the horizontal direction as in Fig. 5, we come close to a so-called multiway dot plot (Cleveland 1994). Multiway dot plots consist of panels where each panel represents one alternative and the levels, that is, the rows of the panels, represent criteria. Thus, the dots, for example, in Fig. 9 represent criterion values. If there is some reference alternative available, the criteria can be arranged so that the dots, for example, increase from bottom to top for that alternative. Then, the actual alternatives can be depicted so that the criteria are in that same order. Multiway dot plots are available e.g. in IND-NIMBUS (Miettinen 2006).

Star coordinate systems
It is suggested in Maňas (1982) that criterion vectors can be represented in a star coordinate system, star symbol plot (Chambers et al. 1983), or as star presentations (Sobol and Klein 1989). (Stars are also sometimes referred to as polygons, see Kleiner and Hartigan 1981.) In stars, we have circles where rays emanating from the origin represent criteria, the lengths of the rays depict criterion values and the end points of the rays are connected to form stars. For example, an alternative involving three criteria is represented as an irregular triangle. This requires the ideal criterion vector and the (possibly approximated) nadir criterion vector to be known. An example is given in Fig. 10. Each circle represents one alternative and the area of each star depicts the goodness of one alternative. In each ray, the ideal criterion value is located at the center and the component of the nadir criterion vector at the circumference. Instead of repeating in each chart which ray represents each criterion, a separate key can be used that shows the assignment of the criteria in the charts as in Chambers et al. (1983). Actually, stars have been used for representing data much earlier than in Maňas (1982). Earlier references dating back to 1920's are given, for example, in Chernoff (1978). Note that if there are not too many alternatives, it is possible to locate all the alternatives in the same picture. This visualization can be regarded as a variant of value paths, where the paths take a different form. In this case, it is better not to fill the areas. Such a graph can be called a circular profile (Mezzich and Worthington 1978).
Note that if the order of the criteria is altered, the shape and the area of the star change. This can be considered a weakness of the system, as stated in Tan and Fraser (1998). In some problems, there may exist some guidelines as how the criteria should be arranged in the stars. An option is not to form the stars but only to display the line segments along the rays, as in Fig. 11. In this case, the lengths of the rays indicate the performance of each criterion (Sobol and Klein 1989). This is called a whisker plot in Ware (2000) and available e.g. in IND-NIMBUS (Miettinen 2006). This formulation is not as sensitive as the star to changing the order of the criteria. The maximum number of criteria when using the star coordinate system is around ten. With significantly more criteria, the shapes will be difficult to interpret. Related to this, it is recommended in Ware (2000) that the rays emanating from the origin should be separated by at least 30 • to avoid confusion. The number of criteria to be displayed can be increased by inverting the luminance polarity of some of the rays, as suggested in Ware (2000).

Spider-web charts
Ideas similar to the star coordinate system are exploited in Kasanen et al. (1991) and an example is presented in Fig. 12, where we have three polygons and each apex of a polygon represents one criterion. (For example, with three criteria we have triangles for each alternative.) This form of a visualization can be called a spider-web because of its shape. Sometimes it is also called a radar chart. The outer polygon shows the (possibly approximated) nadir criterion vector, the inner polygon (the darkest one) stands for the ideal criterion vector and the middle polygon (the grey one) presents one alternative criterion vector. Thus, only the middle polygon varies for different alternatives. It is also possible to include aspiration values of a reference point (that is, one more profile) in the pictures. These ideas are further developed in Kasanen et al. (1991). In any case, the order of the criteria affects the general appearance of the charts as it does with the star coordinate system.
It is possible to locate the polygons of all the alternatives in the same figure as in Fig. 13 (if there are not too many of them). In this case, it is best not to fill the polygons but only to show their shapes (that is, profiles) as in MS-Excel. One can think that this is what we get if we connect the end points of value paths. This kind of a visualization is used in knowCube (Trinkaus and Hanne 2005) where the number of alternatives is unlimited but the decision maker can restrict the number of alternatives to be shown by introducing, for example, desirable bounds for different criterion values. This is also possible in IND-NIMBUS (Miettinen 2006

Petal diagrams
Somewhat parallel ideas to stars and spider-webs are suggested in Tan and Fraser (1998) where a circle is divided into k (the number of criteria) equal sectors. The size (radius) of each slice indicates the magnitude of the criterion value. This is called a petal diagram. Here we have one circle for each alternative. Each segment of the diagram, that is, each criterion can be associated with a different color, as in Fig. 14.
Notice that the order of the criteria has no effect on the actual shape of the diagram or the total area covered by the segments. Thus, the relations of the different segments are clearly shown. Petal diagrams are utilized, for example, in the CASTART system for electricity production (Gandibleux 1999) as well as in WWW-NIMBUS Mäkelä 2000, 2006) and IND-NIMBUS (Miettinen 2006). Similar ideas are utilized in Angehrn (1991)

Fig. 14 Petal diagram
It is mainly a matter of taste in the star coordinate system, the spider-web chart and the petal diagram, how the ideal criterion vector is situated. One may think that when minimizing the criteria it is logical to have the ideal area as small as possible. However, the roles can be interchanged so that the ideal criterion value is located on the circumference and the nadir criterion value at the center. In this case, the larger the area the better. If this is the setting, the ideal criterion values on the circumference can be replaced, for example, by average criterion values or aspiration level values specified by the decision maker. This means that the figures can extend beyond the circumference, stressing values better than the average or the aspiration level. This may be convenient for the decision maker because it is easy to identify alternatives better than some pre-defined level.

Icons
Criterion vectors can be represented using glyphs, that is, graphical objects or symbols illustrating the data. Using glyphs necessitates some mapping or coding between the criterion values and the properties or the graphical attributes of the glyph. This idea can also be called by the name object display (Wickens 1992) or symbolic plot (Chambers and Kleiner 1982). In any case, the motivation is that when several criteria are displayed in a single object, it is guaranteed that the decision maker processes them in parallel.

Chernoff's faces
One of the oldest ideas to visualize multidimensional data is to use Chernoff's faces (Chernoff 1973). The idea is to represent the values of up to 18 criteria as the characteristics of a face. In other words, the values of each criterion are parametrized to represent some feature of an icon, that is, a cartoon of a face. The features of the face include curvature of mouth, size of eyes, slant of eyes, position of pupils, angle of eyebrows, length of nose, width of nose, half-length of face, etc. In Fig. 15, the length of eyebrows is connected to criterion 1 and the size of eyes to criterion 2 whereas The motivation of Chernoff's faces is that people are accustomed to interpreting faces and can rather easily differentiate between them. Thus, the purpose is to allow decision makers to draw on their experience of interpreting facial expressions. The experiments reported in Jacob (1978) support this idea. These experiments involved both Chernoff's faces and the same faces located upside-down. The conventional faces performed substantially better despite the fact that both the faces had the same geometrical features. The difference was that the upside-down faces lacked the familiarity of faces and were, thus, more difficult to be interpreted.
In Pickover (1990), faces are called cartoon faces and different applications for them are discussed. It is suggested, for example, in Bruckner (1978), Huff and Black (1978) and Wang and Lake (1978) that by including ears in the faces one can represent two more parameters, that is, criterion values in our case. The above-mentioned references also describe how to control the facial features representing the criterion values. Symmetry is abandoned in Flury and Riedwyl (1981), where Chernoff's faces are transformed to be asymmetrical. In this way, up to 36 criteria can be represented.
There are some potential weaknesses or problems related to faces. Firstly, there is no unique way how to associate the criteria and the different facial features but the output depends on how the parameters are selected. In addition, because of the subjectivity of the approach, some personal emotions related to liking some features may disturb the interpretations. Furthermore, some facial features are more or less interrelated and some combination may stand out more easily than some others, for example, a happy face. The decision maker may also regard, for example, the curvature of mouth as a more meaningful feature than the length of nose. In this case, the criteria assigned to these two features may not be considered in an appropriate way. Thus, the assignment may introduce some subjective bias in the interpretation. It is suggested in Umanath and Vessey (1994) to associate the most important criteria with the curvature of mouth, obliquity of eyebrows, size of pupils, radius of ears and length of nose. On the other hand, in Everitt (1978), the bias is suggested to be avoided using several sets of faces involving different permutations of criteria associated with different features. However, this may easily increase the cognitive burden on the decision maker.

Harmonious houses
When using icon representations, the icon used must be such that the decision maker can see the icon becoming 'better' as the criterion value improves. That is why concepts like symmetry and harmony are important. An icon that people have been used to seeing in a harmonious and symmetrical form is a house. Thus, it is suggested in Korhonen (1991) to use so-called harmonious houses as icons, where criteria are associated with the corner points of the house, the door, the window or the roof. In Fig. 16, the four criteria of our example data are associated with the upper left and right corners of the roof and the house. The aim is that when the values of the criteria are close to the ideal ones, the house is quite harmonious and symmetrical. This type of a visualization has especially been intended for pairwise comparison. All in all, the success of the approaches based on icons depends on the assignment of the criteria to the parameters of the icon used. In the example given in Korhonen (1991) it is demonstrated how values of 16 criteria can be visualized.
Note that both Chernoff's faces and harmonious houses can, in theory, be used with problems involving quite many criteria. How easy it is to compare such multidimensional data depends on the number of alternatives and, naturally, on the personal capabilities of the decision maker.
The following icons can be used with a very limited number of criteria.

Metroglyphs
In metroglyphs (Anderson 1960), each alternative is represented by a glyph where rays of different lengths emanate from a circle, as shown in Fig. 17. The lengths of the rays are proportional to the criterion values in each alternative but the circle has no special role here. The clarity of the visualization can be increased using a separate key for indicating the order of the criteria, that is, which ray stands for which criterion (Anderson 1960). Metroglyphs are also briefly discussed in Gnanadesikan (1977). Originally, metroglyphs were designed for representing grades like 'high', 'medium' and 'low' instead of numerical values. In this case, a long ray was associated with a high grade and no ray at all stood for a low grade. When visualizing criterion vectors, one can associate the longest ray with the nadir criterion value and no ray at all with the ideal criterion value (or vice versa).
In the first variants of metroglyphs, rays were used around all sides of circles but this was detected to confuse the human eye. It is easier to scan the diagrams more efficiently if the rays are slanted in approximately the same way on both the left-hand and the right-hand sides of the circle. It has been noticed that metroglyphs are most efficient with no more than three to seven rays, that is, criteria. Note that the order of the criteria affects the appearance of the metroglyphs. A variant of metroglyphs, a so-called inward-pointing glyph is suggested in Jacob (1981). As the name suggests, this glyph uses rays emanating from a large circle pointing inwards.

Stick-figure icons
In Pickett and Grinstein (1988), stick-figure icons are used to represent data with the maximum of five dimensions, that is, criteria. A stick-figure consists of five line segments. In the basic form, one line segment acts as a body and the others are legs and arms. The numerical values of the criteria in the alternatives are represented as the angles of these line segments. The authors have developed 12 different ways of connecting the four limbs to the body and they suggest that it is worthwhile to study the same data using different icon types. Originally, the idea has been suggested for studying large amounts of data by locating the icons densely and the goal is to see structure in the data as texture contours. However, the stick-figure icons can be used for visualizing alternative criterion vectors, as well. In this case, their interpretation may need some practice. In Fig. 18, the smaller the angle from the body, the better the corresponding criterion value.

Boxes
Boxes, originally suggested for clustering in Hartigan (1975), are discussed in Kleiner and Hartigan (1981) . 19 Boxes dimensions of the box. Having more that three criteria available means that the three dimensions have several segments. In this case, the box seems to have been wrapped by strings. For an example, see Fig. 19, where the first box has no height because the first criterion has the best possible (i.e., ideal) value in the first alternative. Boxes are not practical with too many criteria. The order of criteria also changes the appearance of the boxes. The interpretation is easier if the criteria can be divided into three correlated clusters corresponding to the three dimensions. In this case, the criteria can be ordered so that similar criteria are close to each other.

Arrows
It is mentioned in Everitt (1978) that alternatives involving up to six criteria can be represented as arrows. Here, the first two criterion values are denoted as a point in a plane. The third criterion value is denoted by an arrow starting from the above-mentioned point, where the length is proportional to the criterion value and the direction either eastwards or westwards shows the sign being positive or negative, respectively. Correspondingly, the fourth criterion value is denoted by an arrow pointing northwards or southwards and the half-cardinal points are reserved for the fifth and the sixth criterion values. Let us add that the interpretation of such a representation may need practice.

Trees
If the number of criteria is too large, for example, for star presentations or bar charts, one can still use trees. They are introduced in Kleiner and Hartigan (1981) and discussed by the name tree symbol plots in Chambers et al. (1983). In trees, each criterion value is assigned to one branch (or a leaf) of a stylized tree and these branches are connected to limbs and the limbs to the trunk. The lengths of the branches represent criterion values and the length of each limb is determined by the average of all the branches it supports. The assignment of criteria to branches is based on a hierarchical clustering and the resulting cluster tree is the basis of the tree symbol. The clustering guarantees that the order of the criteria has no effect on the appearance of the tree. In addition, similar criteria are located close to each other in the plot. Trees are appropriate in case there are clusters of highly correlated criteria in the problem and, then, criteria in the same cluster are located in the same part of the tree. All the criterion values are assumed to be positive. This means that some scaling may be necessary. Some guidelines are given in Kleiner and Hartigan (1981) about how to choose the thickness and the length of the branches and the angles between them.

Castles
It is not too easy to compare criterion values in the same tree. If this is required, trees are suggested in Kleiner and Hartigan (1981) to be augmented by so-called castles. A castle is a combination of a tree and a bar chart. The same hierarchical clustering provides a template for the castles as it does for the trees. Thus, one can say that castles are modified bar charts, where the tree structure is visible. Instead of bars, one can use trapezoids larger at the base and call the plots as tapered castles. The idea of Kleiner and Hartigan (1981) is to use trees to get an overall impression and to use castles for more detail. Thus, when used together, they offer both global and local information. Both of these visualization tools are based on the template, and the construction of the template is rather complicated. For this purpose, a software is introduced in Kleiner and Hartigan (1981). Note that it is suggested in Jacob (1981) to use the hierarchical clustering also with other graphical displays such as bar charts, value paths, etc.

Techniques based on principal components
Projection-based approaches are more or less related to scatter plots. Principal component analysis can be used to find a plane (two dimensions) in which the criteria can be projected. The goal is to find linear transformations of the original criteria so that these transformations are uncorrelated and as little information and variation as possible is lost in two orthogonal directions. In other words, the two largest principal components are selected to form the projection plane. On this plane, projected criteria can be depicted as vectors and projected alternatives as points. One should note that the distance between two projected points approximates the Euclidean distance between the original alternatives (Everitt 1978). The weakness here is that if the criteria have nonlinear relations, principal component analysis cannot find it.
Principal components are used, for example, in the system called Geometrical Analysis for Interactive Aid (GAIA) (Mareschal and Brans 1988). It is a part of the discrete multiattribute decision analysis method PROMETHEE (Brans and Mareschal 1994). Instead of treating the actual criteria, they are first modified as generalized criteria to include some preference information of the decision maker and then normalized. These generalized criteria are linear combinations of the original criteria and they have some benefits over the original ones. Namely, they are in the same scales, big differences in the criterion values are emphasized and small differences are lessened.
If selecting the plane is managed well enough, the relations between the criteria and the alternatives can be seen in their projections. For example, if two criteria are highly conflicting, their vectors go in opposite directions, whereas independent criteria are orthogonal and similar criteria are oriented approximately in the same direction. From the location of the alternatives one can see how well they perform with respect to each criterion, that is, how near or far they are from each other. In any case, the interpretation must be done very carefully.
It seems that the GAIA plane method is a rather clear method of visualization. However, it has two main limitations. Firstly, the plane contains only a part of the information available. Secondly, the conflict characteristics of the criteria are not absolute but depend on the alternatives considered and on the preferences of the decision maker. Thus, any changes in these pieces of information will change the whole visualization.

Biplot and other related techniques
A projection-based technique called biplot is described in Everitt (1978) and Gabriel (1981). Here the word 'bi' refers to the joint display of both rows and columns of the original data matrix containing the alternatives. This graphical multivariate technique is based on the fact that any matrix can be factorized into a product of two matrices with a common rank n. This factorization assigns vectors to each of the rows and columns of the original matrix containing the criterion values. Each of these vectors is of order n and if n = 2 they can be plotted in a plane, see Everitt (1978). In other words, an exact biplot is possible only if the data matrix is of order one or two. Otherwise, the matrix must be approximated by a matrix of rank two. This is discussed in Gabriel (1981). The interpretation of biplot displays is demonstrated in Gabriel (1981) with different examples. In these examples, rows are displayed as points and columns as arrows (that is, vectors).
The same ideas of biplot graphs are utilized in Lewandowski and Granat (1991), where so-called principal component biplot is suggested for the graphical presentation of matrices where the other dimension equals two. As above, the matrix of alternatives is factorized into a product of two matrices of order two and can be plotted on a plane giving a representation of the original criterion vectors. The result is a projection on principal component directions. Dynamic BIPLOT in aspiration level-based decision support systems is also described in Lewandowski and Granat (1991). The presentation is most accurate when correlation between the criteria is concerned and less accurate when distances are to be interpreted. Another question is how much experience one must have to be able to interpret representations like these. Related ideas about projecting alternatives and criteria on a plane using a projection matrix obtained from principal component analysis are used in the MCView System in Vetschera (1994) based on the ideas of Vetschera (1992). Here, preference information like weighting coefficients is included in the projection matrix in addition to the principal components.
A somewhat similar approach to the previous ones is suggested in Raveh (2000) by the name Co-plot. As above, alternatives are mapped as points in a plane and criteria are represented by arrows. Even the interpretation corresponds to that of, for example, GAIA's. Namely, arrows of correlating criteria point in about the same direction and promising compromises are located near the center of the figure where the arrows emerge from. In Co-plot, the alternatives are first normalized based on means and standard deviations and dissimilarities between pairs of alternatives are calculated. This information is presented graphically using a multidimensional scaling method called Guttman's Smallest Space Analysis. It is shown in Raveh (2000) that the general appearance of the graphic displays in GAIA and Co-plot are similar but the latter does not necessitate preference information from the decision maker. Co-plot includes information about the goodness of fit and criteria not fitting the graphic display are recommended to be eliminated. One must note that Co-plot shares the weakness of GAIA's in the sense that changes in the set of alternatives change the whole display.
Applying multivariate statistical techniques is suggested in Losa et al. (2001). The treatment concentrates on group decision making and uses a covariance biplot technique. As in the other projection based approaches, the information is approximated by decreasing a multidimensional matrix into a two-dimensional representation. In Losa et al. (2001), criterion values are replaced by preference scores for the alternatives given by different decision makers.

Movie-type techniques
The idea in the grand tour method (Asimov 1985) is to display the data describing the criterion values of the alternatives as a continuous sequence of two-dimensional orthogonal projections, that is, scatter plots and to move through this sequence. Thus, the data is seen as a movie. The sequence of projection directions is chosen so that it is dense in the set of all possible projection directions. Projection pursuit techniques are suggested in Crawford and Fall (1990) as a direction-reduction technique. The idea is to examine linear projections of the data onto two dimensions. This involves selecting the projection directions in special ways and examining the projected data. The goal is to repeatedly select directions, examine the projections and, for example, to locate interesting structures within the data. In automatic projection pursuit techniques, the directions are identified using heuristic rules. In all, the interpretation of the resulting movie may require experience. It is in order to add that the application of these methods in visualizing alternative criterion vectors is not straight-forward.

Other techniques
Let us mention that we can combine scatter plots and the possibilities described in some of the previous subsections in order to be able to display several criteria. In other words, we can locate, for example, star presentations, petal diagrams or metroglyphs in the scatter plots to represent two or more additional criteria. Here, the roles of the first two criteria differ from the others and it may not be good for the easiness of interpretation.
In what follows, we discuss some more visualization methods.

GRADS
Ideas mentioned earlier in connection with scatter plots are introduced in Klimberg (1992) as a graphical display system called GRADS. GRADS is dynamic and can be applied to problems with about five to twelve criteria. The decision maker is first asked to indicate two criteria whose values in the different alternatives are drawn as points in a plane. The decision maker obtains information about the other criteria for one alternative at a time by indicating that point with a mouse. Then, the other criterion values are depicted as lines originating from the point considered. The lengths of the lines are proportional to the criterion values. The end points of the lines are connected as in the stars but here they form triangles of different colors. The percentage achievements of the alternative in question are also displayed. They are calculated as the difference between the nadir criterion value and the current criterion value divided by the range in the set of Pareto optimal solutions. The decision maker can change the alternative considered and the two criteria forming the base of the display. However, problems arise if the alternatives are not unique with respect to the two selected criteria. In Klimberg and Cohen (1999), options to GRADS are given so that no triangles are formed but only rays (or spokes) emitting from the current alternative are drawn (resembling the idea in Fig. 11). Clearly, the lengths of the rays are proportional to the criterion values. A third possibility is to extend the rays with another color to illustrate the underachievements from the best possible values. In other words, this means that the above mentioned percentage achievements are displayed. In this setting, longer rays are associated with better criterion values.

Andrew's plot
Transforming criterion vectors into two-dimensional curves with the aid of Fourier series was originally proposed in Andrews (1972). Later, this idea has been discussed, for example, in Brodlie et al. (1992), Everitt (1978) and Klimberg (1992). The idea is very simple: We define for −π < t < π a curve z i 1 / √ 2 + z i 2 sin(t) + z i 3 cos(t) + z i 4 sin(2t) + z i 5 cos(2t) + · · · (1) for each alternative i = 1, . . . , m. In this so-called Andrews' plot or function plot (Kleiner and Hartigan 1981), all the alternatives can be plotted on the same coordinate system for comparison. The transformation preserves Euclidean distances, in other words, criterion vectors that are close to each other are transformed into curves that are not far from each other for all values of t. An example is given in Fig. 20. In this figure, also both ideal and nadir criterion vectors are included in order to make the interpretation easier. However, the roles of different criteria are not easy to be seen. Further properties are discussed in Andrews (1972). The plot can be interpreted as in Crawford and Fall (1990): "each point on the curve is essentially a different projection direction." It is suggested in Everitt (1978) that one should locate the most important criterion as the first one in the formula, the second most important as the second one, etc. because the criteria are not treated equally in the transformation. Even though it alternative 1 alternative 2 alternative 3 ideal nadir

Fig. 20
Andrew's plot may be possible to identify clusters in the curves, the plot can otherwise be hard to interpret. Another problem is the fact that not too many alternatives can be displayed. Otherwise, it may be difficult for the decision maker to identify the curves. Variants of these linear Fourier representations (Mezzich and Worthington 1978) are suggested in Andrews (1972) and Chernoff (1973). Namely, principal components can be displayed instead of the original alternatives. The curves may also be plotted in polar coordinates. In Mezzich and Worthington (1978), this is called by the name polar Fourier representation. Alternative formulations for the function to be used instead of (1) are described in Andrews (1972).

Constellation graphical method
A constellation graphical method is proposed in Wakimoto and Taguri (1978). In this method, the criterion values are first scaled to the interval [0, π]. Let the scaled criterion value corresponding to z i j be denoted by ξ i j . Then, each alternative i is mapped as a point for i = 1, . . . , m. This means that each alternative is mapped to the upper half of the unit circle on the one-dimensional complex plane. In this setting, each point is an end point of k linked vectors with the angle equal to the scaled criterion value and the length equal to the user-specified weight w j . In this way, each alternative can be considered as a linked pattern. For further details, see Wakimoto and Taguri (1978).

Symbolic matrices
Besides tables (or matrices) and graphs, we can use so-called symbolic matrices (Chambers and Kleiner 1982). This means that each criterion value in the table is replaced by a symbol and the symbols are proportional to the criterion values. In other words, the criterion values are mapped to a discrete set of symbols. In this way, the criterion values can be expressed in a simple and compact form. This may not be enough as the only way of visualizing the data because it is rather coarse but it may be a helpful addition.

Discussion
In Table 2, we summarize information related to some of the visualization techniques discussed in the previous section. In the first column, we specify the maximum number of criteria the technique can easily represent. The second column shows whether the order of the criteria affects the outer appearance of the visualization and the third column refers to the maximum number of alternatives to be visualized. Here, "not many" means "not too many". The fourth column indicates whether the visualization needs information about the ranges of the criterion values in the Pareto optimal set. As mentioned earlier, they all can utilize information about ranges in one form or another but here "no" refers to the fact that the visualizations can be generated in absolute scales of criteria. The fifth column informs whether the visualization is best suited for pairwise comparison criterion-wise (cr) or alternative-wise (alt). Finally, the last column indicates whether preference information in the form of aspiration levels (i.e. desirable criterion values) can be conveniently represented (instead of preparing one more visualization containing the reference point consisting of the aspiration levels). The information in the table is only trend-setting and no absolute conclusions should be made based on it. For example, for some of the visualization techniques, several variants were introduced for which the information may vary. As far as the maximum number of alternatives to be visualized is concerned, ∞ means that the technique itself does not set limitations but it is up to the decision maker, how many alternatives (s)he feels comfortable comparing.
We can conclude that many visualization possibilities exist and a recommended way of presenting information to the decision maker is to offer the same data in different formats. In this way, the decision maker can choose the most illustrative and informative representations. The visualizations may also supplement each other and the decision maker can tailor the display according to her or his information processing style. While (s)he can change her or his attention from one figure to another, (s)he can also possibly hide undesirable alternatives before making the final selection (as in WWW- NIMBUS Miettinen and Mäkelä 2000, 2006and IND-NIMBUS Miettinen 2006. A simple tabular format may be one of the figures. Corresponding ideas are suggested, for instance, in Steuer (1986), pp. 520-522. Changing the order or the assignment of criteria within a certain visualization type may also emphasize different features and adduce new relationships in the data. It may also be advisable that the decision maker can select some of the alternatives to be highlighted in all the visualizations. The importance of maintaining as much consistency as possible between the different graphs is emphasized in Wickens (1992). Otherwise, the decision maker may face cognitive difficulty in making transition from one graph to another. Human tendency towards anchoring (Tversky and Kahneman 1974) must be kept in mind when presenting the alternatives. Anchoring means that the information that is seen first affects the decision most because it provides a cognitive anchor as a starting point of decision maker's beliefs and impressions. After this, the additional information is used only to shift the anchor. This means that the order of presenting different visualizations and the same alternatives in different formats and using different assignments plays a role in the decision process.
The problem of how we can determine a priori whether the graphical formats used will aid rather than hinder decision making is important in connection with visualizations. This issue is examined in Jarvenpaa (1989) by comparative studies. The conclusion is that knowledge concerning the relationship between the presentation format and the decision strategy of the decision maker can facilitate the selection of the presentation format. Special attention is given to the benefits of bar charts and grouped bar charts. Among others, it is noted that it is important how the graphical information is arranged on a display. It affects the order in which decision makers acquire information.
Literature related to comparing different types of graphic presentations of data is summarized in Lewandowski and Granat (1991). It can be concluded that the research done does not provide clear answers regarding what types of presentation to favor in the decision-making context. The conclusion in DeSanctis (1984) is still valid, namely, no practical guidelines can be given because the results of experiments are conflicting and the area has not been studied systematically. Thus, one cannot claim one approach to be more appropriate than some other for a given task.
Similar matters are handled in connection with visual interactive simulation in Bell and O'Keefe (1995), where it is concluded that the use of visual displays generates final solutions that are demonstrably better than those that make limited use of such displays. This means that different levels of usage of specific displays have an impact on the quality of the solutions generated. In the experiments, bar charts were the most favored visual displays.
According to Ware (2000), "…visual system has evolved over tens of millions of years to enable creatures to perceive and act within the natural environment. Although very flexible, the visual system is tuned to receiving data presented in certain ways, but not in others. If we can understand how the mechanism works, we can produce better displays." Thus, it is important to remember that new visualization tools need time for training the decision makers in interpreting them, see Sobol and Klein (1989). This may explain the good performance of the well-known bar charts in some experiments. Several references are also provided in DeSanctis (1984) to support the claim that the comprehension of graphical displays can be improved with practice. Yet, it is claimed that more empirical evidence is needed.
An important question to be studied is when to use tables and when visualizations. Literature related to this question (between the years 1915 and 1983) is summarized in DeSanctis (1984) and it is noted that the evidence is not conclusive. Several existing studies on the applicability of graphs versus tables are also analyzed in Vessey (1990). According to the theory developed, it is concluded that tables perform better in information acquisition tasks in both time and accuracy of performance. Thus, tables are in order when specific data values must be extracted and a simple retrieval of data is important, since they represent discrete data values. If information must be viewed at a glance, evaluated or relationships in the data are of interest, graphs are recommendable. The same conclusion is drawn in Benbasat and Dexter (1985). Thus, we can say that graphs are visually appealing but sometimes tables are easier to read since they provide exact values. Furthermore, it is concluded in Klimberg and Cohen (1999) that "graphs are superior to tables when presenting spatial data while a tabular presentation is better than graphs for displaying symbolic information." In the experiments reported in Klimberg and Cohen (1999), graphical tools (of GRADS) were concluded to be more effective than tabular values in terms of ten out of eleven performance measures used.
Instead of tables containing the numerical data in its full accuracy, it is possible to use box indices as suggested in Miettinen et al. (2009). They show only significant differences between criterion values and, thus, it is easier for the decision maker to compare alternatives. Box indices are based on evaluating the alternatives in a natural and rough enough scale in order to let the decision maker easily recognize and easily understand the main characteristics of the alternatives at a glance.
In all, one can say that graphs and tables emphasize different characteristics of the same data and they complement each other. In the experiments in Sobol and Klein (1989), tables, bar charts and star presentations were compared. According to the conclusions, tables were more accurate than the others and needed less time. On the other hand, alternatives were easiest to be ranked with star presentations. However, the problem setting of each experiment affects the results and no too strict conclusions should be drawn based on individual tests. It is suggested in Umanath and Vessey (1994) to use Chernoff's faces in the preliminary screening of the alternatives and use other illustrative tools in the final selection whereas using plain tables is not recommended. Related to these topics, the notion of cognitive fit is emphasized in Klimberg and Cohen (1999), Umanath and Vessey (1994) and Vessey (1990). It measures how well the representation format used supports the strategies required to perform the comparison task.
Using colors in visualizations has advantages and disadvantages. Usually, people prefer colorful pictures but it is not always clear that the colors will make the pictures easier to comprehend, as pointed out in DeSanctis (1984). Above all, the colors must be easy to discriminate. Another important issue is that some colors may have specific connotations to the decision maker. Such colors should be avoided as far as possible. An experimental evaluation of graphical and color-enhanced information presentation is given in Benbasat and Dexter (1985). Colors improve the readability and understandability of both symbolic and graphical displays. Colors make it easier for the decision maker to associate visually information belonging to the same context or unit (like alternative or criterion) since such data are coded in the same color. Encouraging results with multi-color reports are mentioned in Benbasat and Dexter (1985). Unfortunately, it is very easy to overload the decision maker with too much color information. Related to colors, the roles of hue, brightness and saturation are discussed in Brodlie et al. (1992). One more thing to keep in mind is the fact that colors are culture-specific and, thus, different colors symbolize different things in different cultures.
It is naturally possible to prepare interactive visualization tools where the visualizations change according to the actions of the decision maker. Some of such ideas have already been mentioned above. As mentioned at the beginning, one can have many alternatives and hide most of them based on, for example, bounds and ranges and gradually individual alternatives will become identifiable like in Trinkaus and Hanne (2005). A general rule can be to show only information that is necessary for the interpretation of data. All the other information can be hidden and shown only when the decision maker requests it. Highlighting of alternatives was already mentioned earlier.
In the interactive world, by pointing one alternative, the decision maker can highlight data related to that alternative (with different color, intensity, font or background) in all the different graphic and tabular illustrations used. Here, as in all the other cases, it is important that the graphical displays are easy for the decision maker to interpret so that the decision maker really gets deeper understanding about the relations of the alternatives. Further possibilities of interactive graphics are discussed in Eick and Wills (1995).
Finally, one must concede that when data is visualized, the interpretation must always be done with care. Furthermore, where a great number of alternatives coexists, the decision maker may get confused no matter how the alternatives are visualized. In this case, statistical tools may be useful keeping still in mind the drawbacks of compressing data. Discussion and further references related to human performance and human-machine systems can be found in Wickens (1992).
One must point out that the research in all the above mentioned fields related to graphical visualization, information acquisition and interpretation, impact of graphics on decision making, design of display format, cognitive aspects, human information processing etc. is scattered in the literature because it involves so many different disciplines. Thus, getting a general impression of the field is not easy.

Conclusions
Many solution approaches for multiple criteria decision making involve comparison of alternative criterion vectors and in this task the decision maker needs support. One possibility to help the decision maker to be able to select the most preferred alternative is to use visualization of the alternatives. Unfortunately, there are no extensive surveys available in the literature of the illustrative tools that are appropriate in these settings. That is why we have presented here a collection of different possibilities for visualization.
Human-computer interaction is important in software for multiple criteria decision making. According to Brodlie et al. (1992), "visualizations can provide extensions to mental processes." However, people have different cognitive styles and this should be taken into consideration when designing the software and its user interface, as emphasized in Sobol and Klein (1989).
In this paper, we have analyzed the advantages and weaknesses of different visualization possibilities and provided some comments about their usability. The idea has been to provide a survey as a starting point for those willing to visualize alternatives. None of the graphical representations can be claimed to be better than the others but some fit certain problem settings better than the others. It is always good to leave the final decision to the decision maker who can select those visualizations (s)he is most comfortable with. If the decision maker has time and willingness, (s)he can tailor the graphical display individually. But still, the software designer must make some kind of a preliminary choice and not to leave the decision maker alone with too many visualization possibilities because it would mean posing one more comparison and decision problem to the decision maker. Among the visualizations, the data of the alternatives in the good old table format should not be forgotten.