Visualization in Multiobjective Optimization

Final version

Bogdan Filipič 1 Tea Tušar 2,1 GECCO Tutorial, Denver, July 20, 2016 Tutorial slides are available at http://dis.ijs.si/tea/research.htm 1Department of Intelligent Systems Jožef Stefan Institute Ljubljana, Slovenia

2DOPHIN Group Inria Lille – Nord Europe Villeneuve d’Ascq, France

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s). GECCO’16 Companion, July 20-24, 2016, Denver, CO, USA ACM 978-1-4503-4323-7/16/07. 2 http://dx.doi.org/10.1145/2908961.2926994

Contents

Introduction

Visualizing approximation sets

Visualizing EAF values and differences Introduction

Summary

References

3 Introduction Introduction

Multiobjective optimization problem Visualization in multiobjective optimization Minimize Useful for different purposes [13] f: X → F • Analysis of solutions and solution sets f:(x ,..., x ) 7→ (f (x ,..., x ),..., f (x ,..., x )) 1 n 1 1 n m 1 n • Decision support in interactive optimization • Analysis of algorithm performance • X is an n-dimensional decision space ⊆ Rm ≥ • F is an m-dimensional objective space (m 2) Visualizing solution sets in the decision space • Problem-specific → Conflicting objectives a set of optimal solutions • If X ⊆ Rm, any method for visualizing multidimensional • Pareto set in the decision space solutions can be used • Pareto front in the objective space • Not the focus of this tutorial

4 5

Introduction Introduction

Visualization can be hard even in 2-D Stochastic optimization algorithms Visualizing solution sets in the objective space • Single run → single approximation set • Interested in sets of mutually nondominated solutions called approximation sets • Multiple runs → multiple approximation sets • Different from ordinary multidimensional solution sets Single run Three runs Ten runs • The focus of this tutorial 1.2 1.2 1.2 1 1 1 0.8 0.8 0.8 Challenges 0.6 0.6 0.6 0.4 0.4 0.4 • High dimension and large number of solutions 0.2 0.2 0.2 0 0 0 • Limitations of computing and displaying technologies 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Run 1 Run 3 Run 5 Run 7 Run 9 • Cognitive limitations Run 2 Run 4 Run 6 Run 8 Run 10

Visualization of the Empirical Attainment Function (EAF) can be used

6 in such cases 7 Introduction

This tutorial is not about • Visualization for decision making purposes [26] • Visualization in the decision space • General multidimensional visualization methods not previously used on approximation sets Visualizing approximation sets

This tutorial covers • Visualization in the objective space • Visualization of separate approximation sets [1] • Visualization of EAF values and differences in EAF values [2]

Methodology Benchmark approximation sets

Two different sets that can be instantiated in any dimension [1]

Comparing visualization methods • Linear with a uniform distribution of solutions • No existing methodology for comparing visualization methods • Spherical with a nonuniform distribution of solutions (more at • Propose benchmark approximation sets (analog to benchmark the corners and less at the center) problems in multiobjective optimization) • Sets are intertwined • Visualize the sets using different methods Size of each set • Observe which set properties are distinguishable after visualization • 2-D: 50 solutions • 3-D: 500 solutions • 4-D: only 300 solutions since most methods cannot handle more

9 10 Benchmark approximation sets Visualizing approximation sets

Desired properties of visualization methods 1 Linear Linear Spherical • Preservation of the Spherical 1 0.8 • Dominance relation 0.8 • Front shape 0.6 1 0.6 f3 0.8 • Objective range f 2 0.4 2 0.6 f 0.4 • Distribution of solutions 0.4 0.2 0.2 0 • Robustness 0.2 0.2 • Handling of large sets 0.4 0.6 • Simultaneous visualization of multiple sets 0 f 0.8 0 0.2 0.4 0.6 0.8 1 1 1 • Scalability in number of objectives f1 • Simplicity

11 12

Visualizing approximation sets General methods

• Scatter plot matrix • Bubble chart Existing methods • Radial coordinate visualization [16, 36] Showing only methods previously used in multiobjective • Parallel coordinates [17] optimization • Heatmaps [29] • General methods • Sammon mapping [30, 33] • Specific methods – designed for visualizing approximation sets • Neuroscale [24, 10] • Self-organizing maps [18, 27] Demonstration on 4-D benchmark approximation sets • Principal component analysis [39] • Isomap [31, 21]

13 14 Scatter plot matrix Scatter plot matrix

1 1 1 2 3 4 Most often f 0.5 f 0.5 f 0.5

• Scatter plot in a 2-D space 0 0 0 0 0.5 1 0 0.5 1 0 0.5 1 • Matrix of all possible combinations f1 f1 f1 1 1 → m(m−1) • m objectives 2 different combinations 3 4

f 0.5 f 0.5

Alternatively 0 0 0 0.5 1 0 0.5 1 f f • Scatter plot in a 3-D space 2 2 1 → m(m−1)(m−2) • m objectives 6 different combinations 4

Linear f 0.5 Spherical

0 0 0.5 1

f3 15 16

Scatter plot matrix Bubble chart

Linear 1 1 Spherical

f4 0.5 f4 0.5

0 0

0.5 0.5 4-D objective space f 1 f 1 2 0.5 1 0.5 1 0 1 0 f3 f3 • Similar to a 3-D scatter plot 1 1 • Fourth objective visualized with point size

f4 0.5 f3 0.5

0 0 5-D objective space 0.5 0.5 f 1 f 1 1 0.5 1 0.5 1 0 1 0 • Fifth objective visualized with colors f2 f2

Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈ ≈

17 18 Bubble chart Radial coordinate visualization

0.8 Linear Also called RadViz Spherical 0.6 • Inspired from physics f3 f1 0.4 • Objectives treated as anchors, 0.2 equally spaced around the 0 circumference of a unit circle 0.2 • Solutions attached to anchors with f2 f4 0.4 ‘springs’ f1 0.6 1 0.8 0.8 • Spring stiffness proportional to the 0.6 0.4 1 0.2 objective value f2 f3 • Solution placed where the spring

Preservation of the forces are in equilibrium Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈ ≈

19 20

Radial coordinate visualization Parallel coordinates

f 1 Linear Spherical • m objectives → m parallel axes • Solution represented as a polyline with vertices on the axes • Position of each vertex corresponds to that objective value • No loss of information f3 f2

1.0

0.8

0.6

0.4

0.2

f4 0.0 f1 f2 f3 f4 Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈ 21 22 Parallel coordinates Heatmaps

Linear Spherical 1 1

0.8 0.8

0.6 0.6 • m objectives → m columns 0.4 0.4 • One solution per row

0.2 0.2 • Each cell colored according to objective value

0 0 • No loss of information f1 f2 f3 f4 f1 f2 f3 f4

Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈

23 24

Heatmaps Sammon mapping

Linear Spherical 1.0 0.8

0.9 0.7 0.8 • A non-linear mapping 0.6 0.7 • Aims to preserve distances between solutions 0.6 0.5 d∗ x x 0.5 0.4 • ij distance between solutions i and j in the objective space • d distance between solutions x and x in the visualized space 0.4 0.3 ij i j 0.3 0.2 • Stress function to be minimized 0.2 0.1 0.1 ∑ ∑ S = (d∗ − d )2 0.0 0.0 ij ij f1 f2 f3 f4 f1 f2 f3 f4 i j>i

• Minimization by gradient descent or other (iterative) methods Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization

25 26 Sammon mapping Sammon mapping

0.8 Linear Spherical

0.4 0.8

0.4

0 Third 0 coordinate -0.4 0.8

Second coordinate 0.4 -0.8 -0.8 0 -0.4 -0.4 Second 0 -0.4 coordinate First 0.4 -0.8 coordinate 0.8 Linear Spherical -0.8 -0.8 -0.4 0 0.4 0.8 Preservation of the Handling of Simultaneous First coordinate dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈

27 28

Neuroscale Neuroscale

1.2

0.8

0.4 • A non-linear mapping • Aims to minimize the same stress function as Sammon mapping 0 • Uses a radial basis function neural network to model the -0.4 projection Second coordinate

-0.8 Linear Spherical -1.2 -1.2 -0.8 -0.4 0 0.4 0.8 1.2 First coordinate

29 30 Neuroscale Self-organizing maps

Linear Spherical

0.8 • Self-organizing maps (SOMs) are neural networks 0.4 • Nearby solutions are mapped to nearby neurons in the SOM Third 0 coordinate • A SOM can be visualized using the unified distance matrix -0.4 0.8 • Distance between adjacent neurons is denoted with color 0.4 -0.8 -0.8 0 • Similar neurons → light color -0.4 Second 0 -0.4 0.4 coordinate • Different neurons (cluster boundaries) → dark color First 0.8 -0.8 coordinate 1.2

Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈

31 32

Self-organizing maps Principal component analysis

Linear Spherical

• Principal components are linear combinations of objectives that maximize variance (and are uncorrelated with already chosen components) • They are the eigenvectors with the highest eigenvalues of the covariance matrix

Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈

33 34 Principal component analysis Isomap

0.8

0.4 • Assumes solutions lie on some low-dimensional manifold and the distances along this manifold should be preserved

0 • Creates a graph of solutions, where only the neighboring solutions are linked

-0.4 • The geodesic distance between any two solutions is calculated Second principal component as the sum of Euclidean distances on the shortest path between Linear the two solutions Spherical -0.8 -0.8 -0.4 0 0.4 0.8 • Uses multidimensional scaling to perform the mapping based First principal component on these distances

Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈

35 36

Isomap Isomap

0.8 Linear Spherical

0.8 0.4 0.4

Third 0 coordinate 0 -0.4 0.8 0.4 -0.8 -0.8 0

Second coordinate -0.4 Second 0 -0.4 coordinate -0.4 First 0.4 -0.8 coordinate 0.8

Linear Spherical -0.8 Preservation of the Handling of Simultaneous -0.8 -0.4 0 0.4 0.8 dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization First coordinate ≈ ≈ ≈

37 38 Summary of the general methods Speciﬁc methods

• Distance and distribution charts [4] • Interactive decision maps [23]

Preservation of the • Hyper-space diagonal counting [3] Method Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization Scatter plot matrix ≈ ≈ ≈ • Two-stage mapping [20] Bubble chart ≈ ≈ ≈ Radial coordinate visual. ≈ ≈ • Level diagrams [6] Parallel coordinates ≈ ≈ Heatmaps Sammon mapping ≈ ≈ • Hyper-radial visualization [8] Neuroscale ≈ ≈ Self-organizing maps ≈ Principal component analysis ≈ ≈ • Pareto shells [35] Isomap ≈ ≈ ≈ • Seriated heatmaps [36] • Multidimensional scaling [36] • Prosections [1]

39 40

Distance and distribution charts Distance and distribution charts

1 Linear 0.8 Spherical

• Plot solutions against their distance to the Pareto front and 0.6

distance to other solutions 0.4 Distance • Distance chart 0.2 0 • Plot distance to the nearest non-dominated solution 0 50 100 150 200 250 300

• Distribution chart 1 Linear • Sort solutions w.r.t. first objective 0.8 Spherical • Plot distances between consecutive solutions 0.6 0.4

• For the first/last solution, compute distance to first/last Distribution non-dominated solution 0.2 0 • k solutions → k + 1 distances 0 50 100 150 200 250 300

• All distances normalized to [0, 1] Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈

41 42 Interactive decision maps Interactive decision maps

Linear Spherical 1 1 The Edgeworth-Pareto hull (EPH) of an approximation set A contains 0.8 0.8 all points in the objective space that are weakly dominated by any solution in A. 0.6 0.6 2 2 f f 0.4 0.4 Interactive decision maps 0.2 0.2 • Visualize the surface of the EPH, not the actual approximation 0 0 set 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 f f • Plot a number of axis-aligned sampling surfaces of the EPH 1 1

f3 f3 • Color used to denote third objective 0 0.5 1 0 0.5 1 f4 = 0.5 f4 = 0.5 • Fixed value of the forth objective

Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈ ≈ 43 44

Hyper-space diagonal counting Hyper-space diagonal counting

Linear • Inspired by Cantor’s proof that shows |N| = |N2| = |N3| ... Spherical

...

(1, 3) (2, 3)

(1, 2) (2, 2) (3, 2) 6 80 (1, 1) (2, 1) (3, 1) (4, 1) 4

2 60

0 40 • Discretize each objective (choose a number of bins) f3f4 bins Number of solutions 0 20 20 • In the 4-D case 40 f f bins 60 0 • Enumerate the bins for objectives f1 and f2 1 2 80

• Enumerate the bins for objectives f3 and f4 • Plot the number of solutions in each pair of bins Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈ 45 46 Two-stage mapping Two-stage mapping

0.6 Linear Steps Spherical

• Split solutions to nondominated and dominated solutions 0.5

• Compute r as the average norm of nondominated solutions 0.4 • Find a permutation of nondominated solutions that minimizes implicit dominance errors and sum of distances between 0.3 consecutive solutions 0.2 • First stage: distribute nondominated solutions on the circumference of a quarter-circle with radius r in the order of 0.1 the permutation and with distances proportional to their 0 distances in the objective space 0 0.1 0.2 0.3 0.4 0.5 0.6 • Second stage: map each dominated solution to the minimal Preservation of the Handling of Simultaneous point of all nondominated solutions that dominate it dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈

47 48

Level diagrams Level diagrams

1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2 Linear Linear

Distance to the ideal point Spherical Distance to the ideal point Spherical 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

f1 f2 → • m objectives m diagrams 1 1

0.8 0.8 • Plot solutions with objective fi on the x axis and distance to the ideal point on the y axis 0.6 0.6 0.4 0.4

0.2 0.2 Linear Linear

Distance to the ideal point Spherical Distance to the ideal point Spherical 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

f3 f4

Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈

49 50 Hyper-radial visualization Hyper-radial visualization

0.8 Linear Spherical 0.7

0.6

0.5 4 f

3 0.4 f

• Solutions preserve distance (hyper-radius) to the ideal point 0.3

• Distances are computed separately for two subsets of objectives 0.2

• Indifference curves denote points with the same preference 0.1

0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

f1f2

Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈ ≈

51 52

Pareto shells Pareto shells

• Use nondominated sorting to split solutions to Pareto shells

• Represent solutions in a graph Shell 0 Shell 1 • Connect dominated solutions to those that dominate them (we

show only one arrow per dominated solution) Linear Spherical

Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization

53 54 Seriated heatmaps Seriated heatmaps

Linear Spherical 300 300

250 250 • Heatmaps with rearranged objectives and solutions 200 200 • Similar objectives and similar solutions are placed together 150 150 • Ranks are used instead of actual objective values for a more uniform color usage 100 100

• Similarity can be computed using 50 50 • Euclidean distance 0 0 • Spearman’s footrule f1 f4 f2 f3 f1 f4 f2 f3 • Kendall’s τ metric

Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈

55 56

Multidimensional scaling Multidimensional scaling

0.4 • Classical multidimensional scaling aims at preserving similarities between solutions 0.2 • Here, dominance distance is used to measure similarity • Two solutions are similar if they share dominance relationships 0 with a third solution

Second coordinate -0.2 1 ∑m S(a, b; z) = [I((a < z ) ∧ (b < z )) + I((a = z ) ∧ (b = z )) m i i i i i i i i i=1 -0.4 linear spherical + I((ai > zi) ∧ (bi > zi))] -0.4 -0.2 0 0.2 0.4 1 ∑ D(a, b) = (1 − S(a, b; z)) First coordinate k − 2 ∈{ } z/ a,b Preservation of the Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization ≈

57 58 Prosections Prosections

300 solutions 3000 solutions • Visualize only part of the objective space Linear Linear Spherical Spherical • Dimensionality reduction by projection of solutions in a section 1 1 • Need to choose prosection plane, angle and section width 0.8 0.8 0.6 1 0.6 1 f4 0.8 f4 0.8 0.4 f3 0.6 0.4 f3 0.6 Linear 1 0.4 0.4 Spherical Linear 0.2 0.2 0.2 0.2 Spherical 1 0.8 0 0 0.8 0.2 0.2 0.6 0.6 1 0.4 0.4

f 3 3 0.8 f f 0.6 0.6 0.4 2 0.6 0.4 0.4 f1f2 0.8 f1f2 0.8 0.2 0.2 1 1 0 0.2 0.2 0.4 0 f1 0.6 0 0.2 0.4 0.6 0.8 1 0.8 f1f2 Preservation of the 1 Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity relation range of solutions large sets visualization Before prosection After prosection ≈ ≈

59 60

Summary of the speciﬁc methods Other (newer) methods

• Tetrahedron coordinates model [5] • Distance-based and dominance-based mappings [11] Preservation of the Method Handling of Simultaneous dominance front shape objective distribution Robustness Scalability Simplicity • Aggregation trees [12] relation range of solutions large sets visualization Distance and distrib. charts ≈ ≈ Interactive decision maps ≈ ≈ ≈ • Trade-off region maps [28] Hyper-space diagonal count. ≈ ≈ Two-stage mapping ≈ ≈ Level diagrams ≈ ≈ • Treemaps [37] Hyper-radial visualization ≈ ≈ Pareto shells • MoGrams [32] Seriated heatmaps ≈ Multidimensional scaling ≈ Prosections ≈ ≈ • Polar plots [15] • Level diagrams with asymmetric norm [7] • Visualization following Shneiderman mantra [19]

61 62 Empirical attainment function

Goal-attainment • Approximation set A • A point in the objective space z is attained by A when z is weakly dominated by at least one solution from A Visualizing EAF values and 1.2

differences 1

0.8

0.6

0.4

0.2

0 0 0.2 0.4 0.6 0.8 1 1.2

Empirical attainment function Empirical attainment function

EAF values [14] Differences in EAF values [22] A • Algorithm A, approximation sets A1, A2,..., Ar • Algorithm , approximation sets A1, A2,..., Ar B • EAF of z is the frequency of attaining z by A1, A2,..., Ar • Algorithm , approximation sets B1, B2,..., Br • Summary (or k%-) attainment surfaces • Visualize differences between EAF values

1.2 1.2 1.2 1.2

1 1 1 1

1 0.8 0.8 1 0.8 0.8 1/2 0.6 0.6 2/3 0.6 0.6 0 1/3 0.4 0.4 -1/2 0 0.4 0.4 1st run of A 25%-att. surf. 1st run best -1 0.2 0.2 0.2 2nd run of A 0.2 50%-att. surf. 2nd run median 1st run of B 75%-att. surf. 3rd run worst 2nd run of B 100%-att. surf. 0 0 0 0 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2

64 65 Visualization of 3-D EAF Benchmark approximation sets

Need to compute and visualize a large number (over 10 000) of Sets of approximation sets cuboids • 5 linear approximation sets with a uniform distribution of solutions (100 solutions in each) Exact case • 5 spherical approximation sets with a nonuniform distribution • EAF values: Slicing [2] of solutions (100 solutions in each) • EAF differences: Slicing, Maximum intensity projection [38, 2]

1 Linear Spherical 0.8 Approximated case 0.6 • EAF values: Slicing, Direct volume rendering [9, 2] 0.4 0.2 1 • EAF differences: Slicing, Maximum intensity projection, Direct 0 0 0.2 0.8 0.4 0.6 volume rendering 0.6 0.4 0.8 0.2 1 0

66 67

Exact 3-D EAF values and differences Exact 3-D EAF values and differences

Slicing

1.2 1.2 1.2 Slicing 1 1 1 0.8 0.8 0.8 • Visualize cuboids intersecting the slicing plane 0.6 0.6 0.6 0.4 0.4 0.4

• Need to choose coordinate and angle 0.2 0.2 0.2

0 0 0 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2

o Slice of Lin Slice of Sph Slice of Lin-Sph and ◦ ◦ ◦ o′ at φ = 5 at φ = 5 Sph-Lin at φ = 5 1.2 1.2 1.2 f3 1 1 1

0.8 0.8 0.8 ′ z z 0.6 0.6 0.6 ϕ f2 r1 0.4 0.4 0.4 f1 0.2 0.2 0.2

0 0 0 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Slice of Lin Slice of Sph Slice of Lin-Sph and ◦ ◦ ◦ 68 at φ = 45 at φ = 45 Sph-Lin at φ = 45 69 Exact 3-D EAF differences Exact 3-D EAF differences

Maximum intensity projection Maximum intensity projection • Volume rendering method for spatial data represented by voxels • Suitable for visualizing EAF differences (focus on large • Simple and efficient differences) • No sense of depth, cannot distinguish between front and back • Sorting w.r.t. EAF differences (smaller to larger) • Plot on top of previous ones

Projection plane Viewpoint

70 71

The approximated case Approximated 3-D EAF differences

Discretization into voxels Maximum intensity projection • Discretization of cuboids • Plots produced using Voreen [25, 34] • Discretization from the space of EAF values/differences • Some loss of information Slicing

1 1 1 1

0.8 0.8 0.8 0.8

0.6 0.6 0.6 0.6

0.4 0.4 0.4 0.4

0.2 0.2 0.2 0.2

0 0 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Exact 643 voxels 1283 voxels 2563 voxels

72 73 Approximated 3-D EAF values and differences Approximated 3-D EAF differences

Direct volume rendering of Lin-Sph

Direct volume rendering • Volume rendering method for spatial data represented by voxels • A transfer function assigns color and opacity to voxel values 1/5 2/5 3/5 • Enables to see “inside the volume” • Requires the definition of the transfer function

4/5 5/5 1/5 and 5/5

74 75

Approximated 3-D EAF differences Approximated 3-D EAF values

Direct volume rendering of Sph-Lin

Direct volume rendering of Sph

1/5 2/5 3/5

1/5 and 5/5

4/5 5/5 1/5 and 5/5

76 77 Summary – Visualization of approximation sets

General methods Speciﬁc methods

• Scatter plot matrix • Distance and distribution charts • Bubble chart • Interactive decision maps • Radial coordinate visualization • Hyper-space diagonal counting Summary • Parallel coordinates • Two-stage mapping • Heatmaps • Level diagrams • Sammon mapping • Hyper-radial visualization • Neuroscale • Pareto shells • Self-organizing maps • Seriated heatmaps • Principal component analysis • Multidimensional scaling • Isomap • Prosections

Summary – Visualization of EAFs Summary

Exact 3-D case Approximated 3-D case EAF values EAF values • Visualization in multiobjective optimization needed for various • Slicing • Slicing purposes • General methods fail to address the peculiarities of EAF differences • Direct volume rendering approximation set visualization • Slicing EAF differences • Customized methods give more information and are currently • Maximum intensity projection • Slicing gaining attentions • Maximum intensity projection • Direct volume rendering

79 80 Acknowledgement

This work was partially funded by the Slovenian Research Agency under research program P2-0209.

This work is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation References program under grant agreement No. 692286.

SYNERGY Synergy for Smart Multi-Objective Optimization www.synergy-twinning.eu

References I References II

[3] G. Agrawal, C. L. Bloebaum, and K. Lewis. [1] T. Tušar and B. Filipič. Intuitive design selection using visualized n-dimensional Visualization of Pareto front approximations in evolutionary Pareto frontier. multiobjective optimization: A critical review and the American Institute of Aeronautics and Astronautics, 2005. prosection method. [4] K. H. Ang, G. Chong, and Y. Li. IEEE Transactions on Evolutionary Computation, 19(2):225-245, Visualization technique for analyzing nondominated set 2014. comparison. [2] T. Tušar and B. Filipič. SEAL ’02, pages 36–40, 2002. Visualizing exact and approximated 3D empirical attainment [5] X. Bi and B. Li. functions. The visualization decision-making model of four objectives Mathematical Problems in Engineering, Article ID 569346, 18 based on the balance of space vector. pages, 2014. IHMSC 2012, pages 365–368, 2014.

82 83 References III References IV

[6] X. Blasco, J. M. Herrero, J. Sanchis, and M. Martínez. A new graphical visualization of n-dimensional Pareto front for [9] K. Engel, M. Hadwiger, J. M. Kniss, C. Rezk-Salama, and decision-making in multiobjective optimization. D. Weiskopf. Information Sciences, 178(20):3908–3924, 2008. Real-time Volume Graphics. A. K. Peters, Natick, MA, USA, 2006. [7] X. Blasco, G. Reynoso-Mezab, E. A. Sanchez Perez, and J. V. Sanchez Perez. [10] R. M. Everson and J. E. Fieldsend. Asymmetric distances to improve n-dimensional Pareto fronts Multi-class ROC analysis from a multi-objective optimisation graphical analysis. perspective. Information Sciences, 340-341:228–249, 2016. Pattern Recognition Letters, 27(8):918–927, 2006. [8] P.-W. Chiu and C. Bloebaum. [11] J. E. Fieldsend and R. M. Everson. Hyper-radial visualization (HRV) method with range-based Visualising high-dimensional Pareto relationships in preferences for multi-objective decision making. two-dimensional scatterplots. Structural and Multidisciplinary Optimization, 40(1–6):97–115, EMO 2013, pages 558–572, 2013. 2010.

84 85

References V References VI

[12] A. R. R. de Freitas, P. J. Fleming, and F. G. Guimaraes. [15] Z. He and G. G. Yen. Aggregation trees for visualization and dimension reduction in Visualization and performance metric in many-objective many-objective optimization. optimization. Information Sciences, 298:288–314, 2015. IEEE Transactions on Evolutionary Computation, 20(3):386–402, [13] S. Greco, K. Klamroth, J. D. Knowles, and G. Rudolph. 2016. Understanding complexity in multiobjective optimization [16] P. E. Hoffman, G. G. Grinstein, K. Marx, I. Grosse, and E. Stanley. (Dagstuhl seminar 15031). DNA visual and analytic data mining. Dagstuhl Reports, pages 96–163, 2015. Conference on Visualization, pages 437–441, 1997. [14] V. D. Grunert da Fonseca, C. M. Fonseca, and A. O. Hall. [17] A. Inselberg. Inferential performance assessment of stochastic optimisers Parallel Coordinates: Visual Multidimensional Geometry and its and the attainment function. Applications. EMO 2001, pages 213–225, 2001. Springer, New York, NY, USA, 2009.

86 87 References VII References VIII

[21] F. Kudo and T. Yoshikawa. [18] T. Kohonen. Knowledge extraction in multi-objective optimization problem Self-Organizing Maps. based on visualization of Pareto solutions. Springer Series in Information Sciences, 2001. CEC 2012, 6 pages, 2012. [19] R. H. Koochaksaraei, R. Enayatifar, and F. G. Guimaraes. [22] M. López-Ibáñez, L. Paquete, and T. Stützle. A new visualization tool in many-objective optimization Exploratory analysis of stochastic local search algorithms in problems. biobjective optimization. HAIS 2016, pages 213–224, 2016. Experimental Methods for the Analysis of Optimization [20] M. Köppen and K. Yoshida. Algorithms, pages 209–222, 2010. Visualization of Pareto-sets in evolutionary multi-objective [23] A. V. Lotov, V. A. Bushenkov, and G. K. Kamenev. optimization. Interactive Decision Maps: Approximation and Visualization of HIS 2007, pages 156–161, 2007. Pareto Frontier. Kluwer Academic Publishers, Boston, MA, USA, 2004.

88 89

References IX References X

[24] D. Lowe and M. E. Tipping. [27] S. Obayashi and D. Sasaki. Feed-forward neural networks and topographic mappings for Visualization and data mining of Pareto solutions using exploratory data analysis. self-organizing map. Neural Computing & Applications, 4(2):83–95, 1996. EMO 2003, pages 796–809, 2003. [25] J. Meyer-Spradow, T. Ropinski, J. Mensmann, and K. H. Hinrichs. [28] R. L. Pinheiro, D. Landa-Silva, and J. Atkin. Voreen: A rapid-prototyping environment for Analysis of objectives relationships in multiobjective problems ray-casting-based volume visualizations. using trade-off region maps. IEEE Computer Graphics and Applications, 29(6):6–13, 2009. GECCO 2015, pages 735–742, 2015. [26] K. Miettinen. [29] A. Pryke, S. Mostaghim, and A. Nazemi. Survey of methods to visualize alternatives in multiple criteria Heatmap visualisation of population based multi objective decision making problems. algorithms. OR Spectrum, 36(1):3–37, 2014. EMO 2007, pages 361–375, 2007.

90 91 References XI References XII

[33] J. Valdes and A. Barton. [30] J. W. Sammon. Visualizing high dimensional objective spaces for A nonlinear mapping for data structure analysis. multiobjective optimization: A virtual reality approach. IEEE Transactions on Computers, C-18(5):401–409, 1969. CEC 2007, pages 4199–-4206), 2007. [31] J. B. Tenenbaum, V. de Silva, and J. C. Langford. [34] Voreen, Volume rendering engine. A global geometric framework for nonlinear dimensionality http://www.voreen.org/ reduction. [35] D. J. Walker, R. M. Everson, and J. E. Fieldsend. Science, 290(5500):2319–2323, 2000. Visualisation and ordering of many-objective populations. [32] K. Trawinski, M. Chica, D. P. Pancho, S. Damas, and O. Cordon. CEC 2010, 8 pages, 2010. moGrams: A network-based methodology for visualizing the set [36] D. J. Walker, R. M. Everson, and J. E. Fieldsend. of non-dominated solutions in multiobjective optimization. Visualizing mutually nondominating solution sets in CoRR abs/1511.08178, 2015. many-objective optimization. IEEE Transactions on Evolutionary Computation, 17(2):165–184, 2013.

92 93

References XIII

[37] D. J. Walker. Visualising multi-objective populations with treemaps. GECCO 2015, pages 963–970, 2015. [38] J. W. Wallis, T. R. Miller, C. A. Lerner, and E. C. Kleerup. Three-dimensional display in nuclear medicine. IEEE Transactions on Medical Imaging, 8(4):297–230, 1989. [39] M. Yamamoto, T. Yoshikawa, and T. Furuhashi. Study on effect of MOGA with interactive island model using visualization. CEC 2010, 6 pages, 2010.