Design of Experiments for Model-Based Optimization Experimenteel ontwerp voor modelgebaseerde optimalisatie Prashant Singh Promotoren: prof. dr. ir. T. Dhaene, dr. D. Deschrijver Proefschrift ingediend tot het behalen van de graad van Doctor in de Ingenieurswetenschappen: Computerwetenschappen Vakgroep Informatietechnologie Voorzitter: prof. dr. ir. D. De Zutter Faculteit Ingenieurswetenschappen en Architectuur Academiejaar 2015 - 2016 ISBN 978-90-8578-896-6 NUR 950, 980 Wettelijk depot: D/2016/10.500/28 Universiteit Gent Faculteit Ingenieurswetenschappen en Architectuur Vakgroep Informatietechnologie Promotoren: prof. dr. ir. Tom Dhaene dr. Dirk Deschrijver Universiteit Gent Faculteit Ingenieurswetenschappen en Architectuur Vakgroep Informatietechnologie Tech Lane, Ghent Science Park - Campus A, iGent, Technologiepark Zwijnaarde 15, B-9052 Zwijnaarde, Belgie¨ Tel.: +32-9-331.49.00 Fax.: +32-9-331.48.99 Proefschrift tot het behalen van de graad van Doctor in de Ingenieurswetenschappen: Computerwetenschappen Academiejaar 2015-2016 Dankwoord The journey of successfully conducting the research leading to this dissertation would not have been possible without the help and support of people important to me. I thank my supervisors Prof. Tom Dhaene and Dr. Dirk Deschrijver for believing in me, granting me the freedom to express myself and most of all, for teaching me the values of conducting good scientific research. Knowledge can be had from books, but values only come from teachers leading by example. I am proud and fortunate to have Prof. Dhaene and Dr. Deschrijver as my teachers, who are amongst the kindest people I know. I also thank Dr. Ivo Couckuyt for his constant support and guidance. I have often turned to Dr. Couckuyt for sound scientific and technical advice, and working with him has been thoroughly enjoyable. What I have learned during the course of my stay in Ghent will stay with me forever, and I am certain that I will miss many pleasant facets of being a PhD student at SUMO lab, Ghent University. The staff at IBCN did their utmost in taking care that my stay here was comfortable, and I would like to extend my gratitude to Ms. Martine Buysse, Ms. Davinia Stevens and Ms. Bernadette Becue for their kind support. Spending considerable amount of time away from family can be hard, but thankfully I always had my wonderful colleagues and friends ensuring that my stay was pleasant. I thank Dr. Krishnan Chemmangat for always being there like a guardian angel, right from the moment he received me at Gent-Sint-Pieters train station on my arrival. My life here would not have been as easy without him (I hope it is not the other way around for him). Thank you Krishnan and Sreeda for everything! Sreeda’s weekend cooking has long been dearly missed. I also thank Dr. Selvakumar Ulaganathan for the many adventures we have had together, and for his constant companionship. Our long evening walks are a lasting memory I fondly remember. I would also like to thank Dr. Elizabeth (a.k.a. Chechi) and John Samuel, and their cute little daughter Joanne for their company, and the many delicious lunches and dinners they invited me to. Arun and Amrita Narayanan are also dearly missed, and I thank them for many memorable discussions over meals. I have Joachim van der Herten to thank for many stimulating and intellectual conversations about our research, and in general. Sharing our office was so much fun! Thanks also to Joeri Ruyssinck (who has an excellent supply of poor jokes to keep up the spirits), Arun Kaintura (the best male cook I know), Yinghao Ye (Mr. Nice and Cute), Leen De Baets (Miss. Pleasant), Domenico Spina (Mr. Funny), Keiichi Ito, Sean Stijven and Sofie van Gassen (The Invisibles). It was also a pleasure to interact with Natalia Leszczynska and Fabio Passos during their short but pleasant stays here. I thank my office colleagues at the Zuiderpoort - Jeroen ii van der Hooft, Cedric De Boom, Lander van Herzeele, Rein Houthooft, Jolien Coenraets, Dr. Matthias Strobbe, Elias De Konick, Sam Leroux and Pieter-Jan Maenhaut for the excellent environment we always had in our office, and for some very fond memories. I thank the chairman of the jury, Prof. Luc Taerwe for devoting his valuable time for my examination process. I thank the jury for their valuable feedback and suggestions -Prof. Thomas Bartz-Beielstein, Prof. Guillaume Crevecoeur, Prof. Nobby Stevens, Dr. Ivo Couckuyt, Dr. Dirk Deschrijver and Prof. Tom Dhaene. I deeply appreciate the presence of Prof. Thomas Bartz-Beielstein in my internal and public defenses’, given that he had to travel to Ghent from Cologne. I also thank him for the valuable discussions on current and future research over time. I thank my collaborations for their cooperation - Dr. Francesco Ferranti, Prof. Khairy Elsayed, Prof. Davy Pissoort, Tim Claeys, Xu Gong, Jens Trogh, Marco Rossi and Prof. Hendrik Rogier. The seeds of the PhD dream are courtesy of my MSc thesis supervisor Dr. Vasudha Bhatnagar, and for that I will always be thankful to her. I deeply admire her for her confidence and conviction in me, and thank her for motivating and inspiring me to follow my own path. I dedicate this dissertation to my wonderful loving family. It is as much their journey as it has been mine. I feel proud and lucky of my parentage. I thank my parents - Prof. Raghuvir Singh and Mrs. Sadhna Singh for making me who I am, and for always giving me the freedom, space and means to be myself and grow as a person. My sister Manisha has been a constant source of support and adventure ever since I opened my eyes, and I thank her, my brother-in-law Neeraj and my cute, energetic, naughty niece Manasi for many wonderful moments. Last, but not the very least, I would like to thank my wife Ekta for watching Top Gear with me, being a solid source of support, putting up with my eccentricities (which there are many) and sharing many enjoyable adventures and travails (not necessarily in the same order). Gent, May 2016 Prashant Singh Table of Contents Dankwoordi Samenvatting xxi Summary xxv 1 Introduction1 1.1 Challenges, Research Contributions and Outline..........3 1.1.1 The Inverse Problem....................3 1.1.2 Exploration v/s Exploitation Tradeoff...........4 1.1.3 Constrained Multi-Objective Optimization........5 1.1.4 Multiple Fidelities of Data and Large Datasets......5 1.1.5 The Curse of Dimensionality................6 1.1.6 Line-based Design of Experiments.............6 1.2 Publications.............................6 1.2.1 Publications in international journals (listed in the Science Citation Index)............8 1.2.2 Publications in international conferences (listed in the Science Citation Index)............9 1.2.3 Publications in national conferences............9 References................................. 11 2 A Sequential Sampling Strategy for Adaptive Classification of Compu- tationally Expensive Data 13 2.1 Introduction............................. 14 2.2 Adaptive Classification....................... 15 2.3 Related Work on Data Sampling.................. 16 2.4 Neighborhood-Voronoi Sequential Sampling Algorithm...... 17 2.4.1 Exploitation......................... 19 2.4.2 Exploration......................... 22 2.4.3 Combining exploitation and exploration score....... 24 2.5 Examples.............................. 25 2.5.1 Example: Non-Linearly Separable Classification Problem 25 2.5.2 Effect of Noise....................... 28 2.5.3 Example: Nowacki Beam Problem............. 29 iv 2.5.4 Example: Disconnected Feasible Regions......... 31 2.6 Conclusion and Future Work.................... 33 References................................. 34 3 Adaptive Classification Algorithm for EMC Compliance Testing of Electronic Devices 39 3.1 Introduction............................. 40 3.2 Goal Statement........................... 40 3.3 Adaptive Classification Algorithm................. 40 3.4 Sequential Sampling Scheme.................... 41 3.5 Experiment............................. 42 3.6 Conclusion............................. 44 References................................. 45 4 A Balanced Sequential Design Strategy for Global Surrogate Modeling 47 4.1 Introduction............................. 48 4.2 Sequential Design Methods..................... 49 4.3 Balancing Schemes......................... 50 4.4 Experiments and Results...................... 52 4.4.1 SUMO Toolbox....................... 52 4.4.2 Benchmark Functions................... 52 4.4.3 Numerical Results..................... 53 4.5 Conclusions............................. 53 References................................. 56 5 Accurate Hotspot Localization by Sampling the Near-Field Pattern of Electronic Devices 57 5.1 Introduction............................. 58 5.2 Goal statement and preliminaries.................. 58 5.3 Kriging Models........................... 59 5.4 Statistical Criteria.......................... 59 5.4.1 Generalized Probability of Improvement.......... 60 5.4.2 Minimum Distance Criterion................ 60 5.5 Example : Measured Microstrip Lines............... 60 5.6 Example : Printed Circuit Board.................. 62 5.7 Conclusion............................. 62 5.8 Software implementation...................... 63 References................................. 67 6 A Constrained Multi-Objective Surrogate-Based Optimization Algo- rithm 69 6.1 Introduction............................. 70 6.2 Multi-Objective Surrogate Based Optimization.......... 70 6.3 Efficient Constrained Multi-Objective Optimization Algorithm (ECMO) 72 6.3.1 Kriging..........................