Alma Mater Studiorum - Universit`adi Bologna DOTTORATO DI RICERCA IN COMPUTER SCIENCE AND ENGINEERING Ciclo: XXXI Settore Concorsuale : 01/B1 Settore Scientifico Disciplinare: INF/01 Innovative Applications of Constraint Programming Presentata da: Tong Liu Coordinatore Dottorato: Relatore: Paolo Ciaccia Maurizio Gabbrielli Correlatore: Jacopo Mauro Esame finale anno 2019 Abstract Constraint programming (CP) is a declarative paradigm that enables us to model a problem in the form of constraints to be satisfied. It offers powerful constraint solvers which, by implementing general-purpose search techniques, are fast and robust to address complex constraint models automatically. Constraint programming has attracted the attention of people from various domains. By separating the definition of a problem from its solution, it is more natural for people to implement the program directly from the problem specification, reducing the cost of development and future maintenance significantly. Furthermore, CP provides the flexibility of choosing a suitable solver for a problem of a given nature, which overcomes the limitations of a unique solver. Thanks to this, CP has allowed many non-domain experts to solve emerging problems efficiently. This thesis studies the innovative applications of CP by examining two topics: constraint modeling for several novel problems, and automatic solver selection. For the modeling, we explored two case studies, namely the (sub)group activity optimization problem, and the service function chaining deployment problem that comes from the Software Defined Network (SDN) domain. Concerning the solver selection, we improved an algorithm selection technique called \SUNNY", which generates a schedule of solvers for a given problem instance. In this work, we demonstrate with empirical experiments that the procedure we have designed to configure SUNNY parameters is effective, and it makes SUNNY scalable to an even broader range of algorithm selection problems not restricted to CP. iii iv Acknowledgements I would like to express my deep gratitude to Professor Maurizio Gabbrielli, my research supervisor, for his endless advice and patient guidance throughout my time as a graduate student. I thank also my co-supervisor Professor Jacopo Mauro; although we have been working remotely, I learned enormously from many enlightening discussions with him, which has significantly influenced my thinking and research. I am greatly indebted to Professor Gabbrielli and Professor Mauro for all that I learned from them both in terms of technical knowledge and in terms of research style. Their willingness to give their availability so generously has been very much appreciated. This work would have been impossible without their help and support. I am also very grateful to Dr. Roberto Amadini, Professor Roberto Di Cosmo and Professor Franco Callegati for their support and advices; working with them was also a hugely positive and enriching experience. My experience as a Ph.D. student at University of Bologna would have been much poorer but for the colleagues and friends that I had in Bologna. I would like to thank particularly Mariagrazia Amato, Luca Bedogni, Giacomo Bergami, Flavio Bertini, Marcos Caetano, Stefano Cammelli, Angelo Di Iorio, Adrien Durier, Andrea Melis, Federico Montori, Andrea Nuzzolese, Francesco Gavazzo, Saverio Giallorenzo, Micheal Lodi, Vicenzo Lomonaco, Stefano Pio Zingaro, Stefano Rizzo, Rahimeh Rouhi, Daniel Russo, Wilson Sakpere, Luca Sciullo, Angelo Trotta, YF Chen, YC Fan, XL Xu, Chun Tian and YW Zhang. I extend my thanks also to the friends I met at the Doctoral Training Center in Paris (EIT Digital), Wenqin Shao, Jos´e v Horta, Gianni Barlacchi, Mourad Leslous and Quang Pham. The author acknowledges the Ph.D. fundings sponsored by the Italian Ministry of Education (MIUR) and trainings offered by the European Doctoral School, EIT Digital. vi List of Acronyms AI Artificial Intelligence AS Algorithm Selection ASlib Algorithm Selection Library COSEAL COnfiguration and SElection of ALgorithms CP Constraint Programming CSP Constraint Satisfaction Problem ETSI European Telecommunications Standards Institute ILP Integer Linear Programming k-NN k-Nearest Neighbors algorithm - a non-parametric machine learning method LCG Lazy Clause Generation MANO Management and Orchestration MILP Mixed Integer Linear Programming MP Mathematical Programming NBI North Bound Interface NFV Network Function Virtualization OASC Open Algorithm Selection Challenge QoS Quality of Service SA Simulated Annealing - an approximative search technique in AI vii SBI South Bound Interface SDN Software Defined Network SFC Service Function Chain SUNNY SUb-portfolio Nearest Neighbor lazY - a technique for algorithm selection VIM Virtualized Infrastructure Managers VNF Virtual Network Function WAN Wide Area Network viii List of Figures 2.1 Model for the Algorithm Selection Problem. 23 3.1 Graphic User Interface of NightSplitter. 30 3.2 CP solvers comparison for NightSplit.................. 46 3.3 CP vs SA in terms of solution quality for NightSplit.......... 47 3.4 CP vs SA in terms of runtime for NightSplit.............. 48 3.5 Comparison of CP and SA varying the number of subgroups. 48 3.6 Comparison of CP and SA varying preference weight η......... 49 4.1 General concept of NFV. 57 4.2 General concept of MANO. 58 4.3 General example of dynamic Service Function Chaining. 59 4.4 SFCtree example. 67 4.5 Solvers comparison for SFC design problem............... 74 4.6 System response time varying instance size for SFC design problem. 75 A.1 Solvers comparison two for SFC design problem............ 113 A.2 Solvers performance varying domain constraints for SFC design problem.114 A.3 Percentage of failed runs whSolvers performance varying domain constraints for SFC design problem................... 114 ix List of Tables 2.1 Technical comparison of Mathematical Programming and Constraint Programming. 21 3.1 Summary of the experiment dataset for NightSplit........... 45 3.2 Summary of NightSplit instances in the MiniZinc Challenge 2017. 49 5.1 Toy example of AS technique SUNNY. 87 5.2 Scenarios in the Open Algorithm Selection Challenge. 89 5.3 Comparison of split modes of sunny-as2................. 94 5.4 Performances of sunny-as2 varying number of training instances. 95 5.5 Neighborhood size k used by sunny-as2 for scenarios in OASC. 97 5.6 Performances of sunny-as2 by varying feature limit. 97 5.7 Performances of sunny-as2 with different combinations of feature limit and training instances limit. 99 5.8 Results of sunny-as2 by tuning the schedule size parameter. 99 5.9 Results with different combinations of training and test functions. 101 5.10 Time spent for training by various evaluation functions of sunny-as2. 102 5.11 Comparisons of basic modalities offered by sunny-as2......... 102 xi Contents Abstract iii Acknowledgements v 1 Introduction 1 1.1 Thesis outline and contributions . .4 2 Background 7 2.1 Brief History of CP . .8 2.2 Constraint Satisfaction Problems (CSPs) . .9 2.2.1 Constraint Optimization Problem . 10 2.3 Solving CSP . 11 2.3.1 Consistency Techniques . 13 2.3.2 Constraint Propagation . 14 2.3.3 Lazy Clause Generation (LCG) . 16 2.4 Applications of CP . 16 2.4.1 Limitations . 17 2.5 Other techniques to Solve CSP . 18 2.5.1 Operations Research (Mathematical Programming) . 18 Linear Programming . 19 Constraint Programming vs. Mathematical Programming . 20 xiii 2.5.2 Local Search . 22 2.5.3 Portfolio Approaches . 23 I Applications of Constraint Programming 25 3 CP for (sub)group activity optimization 27 3.1 Problem Introduction . 27 3.2 NightSplitter . 29 3.3 NightSplit . 31 3.3.1 Useful Extensions . 37 3.4 Solution Approaches . 37 3.4.1 NightSplit and Constraint Programming . 38 3.4.2 NightSplit and Simulated Annealing . 42 3.5 Empirical Experiment . 43 3.5.1 NightSplit in MiniZinc Challenge 2017 . 49 3.6 Related work . 50 3.7 Summary and future prospectives . 51 4 Flexible Service Function Chaining Deployment with CP 53 4.1 Problem Introduction . 54 4.2 Application Context: NFV/SDN Networking . 56 4.3 Problem Definition . 60 4.3.1 Service Function Chain specification . 62 4.3.2 SFC design problem . 65 4.4 SFC modeling with Constraint Programming . 70 4.5 Empirical Validations . 72 4.6 Summary . 75 xiv II Constraint Solver Selection with SUNNY 77 5 SUNNY for Algorithm Selection 79 5.1 Introduction to Algorithm Selection . 80 5.2 Related work . 82 5.3 Preliminaries . 84 5.3.1 Algorithm Selection Problem . 84 5.3.2 Feature Selection . 85 5.3.3 SUNNY and sunny-as ...................... 86 5.3.4 2017 OASC challenge . 88 5.4 sunny-as2 ................................. 89 5.4.1 Evaluation function . 90 5.4.2 sunny-as2 and its execution modalities . 91 5.5 Empirical Validation . 93 5.5.1 Stratified vs Random Cross Validation . 94 5.5.2 Number of Training Instances . 95 5.5.3 Limit on the Number of Features . 96 5.5.4 Schedule size λ for greedy-SUNNY ................ 99 5.5.5 greedy-SUNNY vs SUNNY . 100 5.5.6 Comparison of execution modalities . 102 5.6 Chapter Summary . 103 6 Conclusions and future extensions 105 A Appendix 109 A.1 Approximation algorithm for NightSplit ................ 109 A.2 Hardness of SFC design problem .................... 111 A.3 Extended experiments for SFC design problem ............ 113 References 115 xv xvi Chapter 1 Introduction This thesis studies the application of constraint programming (CP) in solving novel problems and the improvement of solving efficiency with the aid of solver selection. As mentioned in a Chinese idiom that \Nothing can be accomplished without norms or standards" (Mencius, 300 BC), it can be noted that humans are surrounded by constraints, which arise naturally from their endeavors. The constraints, being a medium of expression for formalizing regularities, limit, oblige, or even prevent people when they make decisions. Today, with the use of of artificial intelligence (AI), many problems that involve constraints can be modeled as constraint satisfaction problems (CSPs). Within this framework, a problem can be represented concisely by a set of constraints over a set of variables with a finite domain, and a solution includes values that are assigned to all variables that contemporaneously satisfy all the constraints.