University of Calgary PRISM: University of Calgary's Digital Repository Graduate Studies The Vault: Electronic Theses and Dissertations 2018-07-25 EEIA: The Extended Efficiency Improvement Advisor Nygren, Nicholas Nygren, N. (2018). EEIA: The Extended Efficiency Improvement Advisor (Unpublished master's thesis). University of Calgary, Calgary, AB. doi:10.11575/PRISM/32705 http://hdl.handle.net/1880/107524 master thesis University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission. Downloaded from PRISM: https://prism.ucalgary.ca UNIVERSITY OF CALGARY EEIA: The Extended Efficiency Improvement Advisor by Nicholas Nygren A THESIS SUBMITTED TO THE FACULTY OF GRADUATE STUDIES IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE GRADUATE PROGRAM IN COMPUTER SCIENCE CALGARY, ALBERTA JULY, 2018 c Nicholas Nygren 2018 Abstract In the past, the Efficiency Improvement Advisor (EIA) has been successfully applied to several dynamic problems. By learning recurring tasks, it was able to correct inefficient behavior in multi- agent systems. We present an extension to the advisor which allows certain known-ahead knowl- edge to be exploited. This extension unobtrusively guides autonomous agents to follow a plan, while retaining the dynamic abilities of those agents. Unlike other similar approaches which in- troduce planning functionality, this does not require always-on communications. The extended advisor’s planning abilities work in tandem with the original learning abilities to create additional efficiency gains. The abilities of the extended advisor (including the introduction of planning, the preservation of dynamism, and mixing certain knowledge with learned knowledge) are eval- uated in 2 different problem domains. First, the advisor is applied to the familiar arcade game: Whack-a-mole. Then, Pickup and Delivery is considered, which is similar to coordinating a taxi service. ii Table of Contents Abstract ........................................... ii Table of Contents . iii List of Tables . v List of Figures . vi 1 Introduction . 1 2 Definitions and Basic Concepts . 4 2.1 The Problem: Dynamic Task Fulfillment . 4 2.2 Multi Agent Systems . 7 2.3 The Efficiency Improvement Advisor . 10 2.3.1 The extract step . 13 2.3.2 The optimize step . 15 2.3.3 The derive step . 17 3 The Extended Efficiency Improvement Advisor . 19 3.1 The Working Cycle of the EEIA . 20 3.2 The derive step . 22 3.3 Simple Example . 25 4 Simple Application: Whack-a-mole . 29 4.1 The Environment . 33 4.2 Dynamic Task Fulfillment Perspective . 35 4.2.1 Task Similarity . 35 4.2.2 Optimization Criterion . 36 4.2.3 Bounding Function . 38 4.3 Agents . 39 4.4 Required Modifications of Agents for Working with An Advisor . 43 4.4.1 Providing Histories . 43 4.4.2 Handling Rules . 45 4.4.3 Rule Conflicts . 48 4.4.4 New Agents . 50 5 Complex Application: Pickup and Delivery . 53 5.1 The Environment . 57 5.2 Dynamic Task Fulfillment Perspective . 59 5.2.1 Task Similarity . 60 5.2.2 Optimization Criterion . 60 5.3 Agents . 64 5.4 Required Modifications of Agents for Working with An Advisor . 68 5.4.1 Providing Histories . 68 5.4.2 Handling Rules . 69 5.4.3 Rule Conflicts . 70 6 Experiments . 71 6.1 Generating Base Run Instances . 72 6.1.1 Whack-a-mole . 72 6.1.2 Pickup and Delivery . 74 iii 6.2 Degree of Dynamism . 76 6.2.1 Whack-a-mole . 78 6.2.2 Pickup and Delivery . 81 6.3 Static Variant . 84 6.3.1 Whack-a-mole . 85 6.3.2 Pickup and Delivery . 86 6.4 Full Runs . 88 6.4.1 Whack-a-mole . 91 6.4.2 Pickup and Delivery . 93 6.4.3 More Vehicles . 93 7 Related Works . 100 7.1 Control Theory . 100 7.2 Whack-a-mole . 102 7.3 Pickup and Delivery . 104 7.3.1 How pre-determined are an agent’s actions? . 106 7.3.2 How does the system handle new requests mid-execution? . 107 7.3.3 What is the impact of an agent/component failing? . 108 7.3.4 What happens when communication fails mid-execution? . 109 7.3.5 Which agents require constant communication? . 111 7.3.6 How much variation is there in vehicle size? . 112 7.3.7 How can tasks be given emphasis or priority? . 113 7.3.8 What is the optimization criterion? . 113 7.3.9 What limitations are there on simultaneous jobs? . 115 7.3.10 What ability is there to cancel a task? . 116 8 Conclusion . 117 Bibliography . 121 A Optimization Methods . 126 A.1 Genetic Algorithm . 126 A.2 Branch and bound . 130 iv List of Tables 5.1 Customer requests made in advance, to allow planning. 54 5.2 Based on the known-ahead requests, the driver can make a plan. 54 5.3 In retrospect, all fares fulfilled, including dynamic ones. 56 6.1 Advised single hammer FIFO in the static variant of Whack-a-mole, quality rela- tive to a plan. 85 6.2 Advised 2 hammer NN in the static variant of Whack-a-mole, quality relative to a plan. 85 6.3 Average efficiency of executing plans in the static variant of PDPTW. 87 6.4 The effect of d on the knowledge set. 90 6.5 Comparison of the quality (Avg. Hits better than base system) of solutions achieved by base system (single hammer FIFO), system with EIA and system with EEIA for larger values of m.................................... 94 6.6 Comparison of the quality (Avg. Hits better than base system) of solutions achieved by base system (multi-hammer NN), system with EIA and system with EEIA for larger values of m.................................... 95 6.7 Comparison of the quality (total time over all run instances of an experiment) for Pickup and Delivery with 2 vehicles and measuring efficiency by time, varying values of m....................................... 96 6.8 Comparison of the quality (total time over all run instances of an experiment) for Pickup and Delivery with 2 vehicles and measuring efficiency by distance, varying number of vehicles. 99 A.1 Some parameters adjusted for problem size, where m is the number of tasks. 130 v List of Figures and Illustrations 2.1 A Multi Agent System. 7 2.2 A Multi Agent System under advice. 10 2.3 The actions of the original advisor, the EIA (from [SD+10]). 11 2.4 Types of Exception Rules . 17 3.1 The actions of the EEIA. 21 3.2 High-level example run instance. 27 4.1 Example mole schedule, and comparison of hammer choices. 31 4.2 The Whack-a-mole environments, varying sizes (are not required to be square). 33 fifo 4.3 The application of an ignore rule, ata j , to a queue for g . 46 : A nn 4.4 The application of an ignore rule, ata , to a list of tasks distances for g . 47 : j A 4.5 The application of an proactive rule to a queue for gfifo. 47 4.6 The application of an proactive rule to a list of tasksA distances for gnn. 48 4.7 An Ignore rule conflicting with a proactive rule for the same task.A This is the difference made by order of rule application. 49 5.1 Larger cities and towns on the Trans Canada, labeled with distances separating them. 53 5.2 The plan, shown as the graph of the highway extended over the day (a space-time diagram). A blue line is used to represent the lifeline/trajectory of the vehicle. 54 5.3 An example fare,.