LEARNING TO RANK IMPROVING THE PERFORMANCE OF AN ENTROPY DRIVEN ADVISORY SYSTEM USING IMPLICIT USER FEEDBACK By: Seth Kingma Studentnumber 0735272 [email protected] Supervisors: Dr. M.A. Wiering (Artificial Intelligence, University of Groningen) Prof. Dr. L.R.B. Schomaker (Artificial Intelligence, University of Groningen) ARTIFICIAL INTELLIGENCE UNIVERSITY OF GRONINGEN CONTENTS LIST OF TABLES............................................................... V LIST OF FIGURES ............................................................ VI LIST OF EQUATIONS ....................................................... VII ABSTRACT .................................................................. VIII PREFACE ...................................................................... IX ACKNOWLEDGEMENTS .................................................................X 1. INTRODUCTION ......................................................... 1 1.1. RESEARCH QUESTION ........................................................ 1 1.2. METHODS ........................................................................ 2 1.3. SCIENTIFIC RELEVANCE FOR ARTIFICIAL INTELLIGENCE............ 2 1.4. STRUCTURE OF THE THESIS ................................................. 3 2. THE BICYCLE ADVISORY SYSTEM ................................. 4 2.1. INSPIRATION..................................................................... 4 2.1.1. Decision Trees ....................................................................4 2.1.2. Constructing Decision Trees .................................................5 2.1.2.1. Entropy .................................................................................6 2.1.2.2. Information Gain ....................................................................7 2.2. CONCEPTUAL DESIGN AND IMPLEMENTATION ......................... 8 2.2.1. Activation Values ................................................................8 2.2.2. Automatic Selection of the Most Relevant Question ............... 10 2.2.2.1. Calculation of the Entropy ..................................................... 10 2.2.2.2. Information Gain .................................................................. 11 2.2.2.3. Determination of the Most Relevant Bicycles............................ 11 2.2.3. The WIZARD Algorithm ...................................................... 12 2.2.4. Implementation................................................................. 13 2.3. FUTURE WORK................................................................ 13 2.3.1. Weighted Activation Values ................................................ 14 2.3.2. Weighted Expected Entropy................................................ 14 2.3.3. Entropy and Order ............................................................ 14 2.4. SUMMARY ...................................................................... 15 3. FEEDBACK ..............................................................16 3.1. EXPLICIT FEEDBACK ........................................................ 16 3.2. IMPLICIT FEEDBACK ........................................................ 16 3.3. USING FEEDBACK IN THE ADVISORY SYSTEM ........................ 17 3.3.1. The FAIRPAIRS Algorithm................................................... 17 3.3.2. Application to the Advisory System: FAIRSHARES .................. 18 3.3.3. Implementation................................................................. 19 3.4. SUMMARY ...................................................................... 20 4. LEARNING ALGORITHMS ............................................21 4.1. ARTIFICIAL NEURAL NETWORKS ......................................... 21 4.1.1. The Artificial Neuron......................................................... 21 4.1.2. Single Layer Feed-Forward Networks................................... 23 iii 4.1.2.1. Widrow-Hoff or Delta Rule Learning....................................... 24 4.1.3. Multilayer Feed-Forward Networks ..................................... 26 4.1.3.1. The BACKPROPAGATION Algorithm.......................................... 26 4.1.4. Neural Networks applied to the Advisory System ................... 28 4.1.4.1. Single Layer Feed-Forward Network ....................................... 28 4.1.4.2. Multilayer Feed-Forward Network .......................................... 30 4.2. BAYESIAN LEARNING ........................................................ 30 4.2.1. Bayes Theorem ................................................................. 31 4.2.2. Naïve Bayes Classifiers....................................................... 31 4.2.3. Bayesian Learning applied to the Advisory System ................ 32 4.3. CONCLUSION .................................................................. 34 4.4. SUMMARY ...................................................................... 34 5. CONCEPTUAL DESIGN & IMPLEMENTATION ...................35 5.1. CONCEPTUAL DESIGN ...................................................... 35 5.1.1. Preference Pairs................................................................ 36 5.1.1.1. Obtaining Preference Pairs .................................................... 36 5.1.1.2. Obtaining Training Data for Relative Ranking.......................... 36 5.1.1.3. Obtaining Training Data for Absolute Ranking ......................... 37 5.1.2. Ranking Cost Function....................................................... 38 5.1.3. The Bipolar Sigmoid Activation Function............................. 39 5.1.4. Learning to Rank............................................................... 39 5.2. IMPLEMENTATION ........................................................... 41 5.3. SUMMARY ...................................................................... 42 6. RESULTS ................................................................43 6.1. DATA PREPARATION ........................................................ 43 6.2. EXPERIMENTS................................................................. 43 6.2.1. Learning the Expert’s Opinion ............................................ 43 6.2.2. Relative Ranking ............................................................... 45 6.2.2.1. Setting the Network Parameters.............................................. 45 6.2.2.2. Learning Relative Ranking ..................................................... 45 6.2.2.3. Why Try to Learn the Obvious?............................................... 46 6.2.2.4. Comparing the Approaches.................................................... 46 6.2.3. Absolute Ranking .............................................................. 48 6.3. SUMMARY ...................................................................... 49 7. DISCUSSION ............................................................50 7.1. EVALUATION .................................................................. 50 7.2. FUTURE WORK................................................................ 51 7.2.1. Learning to Cluster............................................................ 51 7.2.2. Consumer Price ................................................................ 51 7.2.3. Multilayer Networks .......................................................... 52 7.3. CONCLUSION .................................................................. 52 REFERENCES .................................................................53 iv LIST OF TABLES CHAPTER 2 2.1 Training examples for the target concept BicycleRacingWeather 2.2 Is D20 a good day to take the racing bicycle out for a ride? 2.3 Summary of the ID3 algorithm specialized for learning Boolean-valued functions 2.4 Summary of the WIZARD algorithm behind the advisory system CHAPTER 3 3.1 Summary of the FAIRPAIRS algorithm for obtaining unbiased clickthrough data 3.2 Summary of the FAIRSHARES algorithm CHAPTER 4 4.1 Summary of the STOCHASTIC-GRADIENT-DESCENT algorithm 4.2 Summary of the BACKPROPAGATION algorithm CHAPTER 5 5.1 Summary of the OBTAIN-PREFERENCES procedure 5.2 Summary of the OBTAIN-RELATIVE-TRAINING-DATA procedure 5.3 Summary of the OBTAIN-ABSOLUTE-TRAINING-DATA procedure 5.4 Calculating the ranking error terms for a given set of preference pairs 5.5 Summary of the STOCHASTIC-GRADIENT-DESCENT-RANKING algorithm CHAPTER 6 6.1 Testing different combinations of sigmoid α and learning speed η parameters 6.2 Testing various sizes for training and test sets in relative ranking 6.3 Simplifying the FAIRSHARES interpretation 6.4 Using the activation values of the advisory system as initial weight values 6.5 Testing the three approaches for relative ranking 6.6 Testing various sizes for training and test sets in absolute ranking v LIST OF FIGURES CHAPTER 2 2.1 Decision tree for the target concept BicycleRacingWeather 2.2 Managing the activation values 2.3 The advisory system at work CHAPTER 3 3.1 Feedback in the advisory system CHAPTER 4 4.1 Nonlinear model of a neuron 4.2 Threshold and sigmoid activation functions 4.3 Feed-forward single layer neural network for the concept BicycleRacingWeather 4.4 The advisory system as a single layer linear network 4.5 The advisory system as a multilayer network CHAPTER 5 5.1 Bipolar sigmoid activation function CHAPTER 6 6.1 Total rank error per sample for the base preferences of the advisory system 6.2 Total number of conflicting preference pairs in learning the base preferences 6.3 Total percentage of correctly arranged test sample pairs in relative ranking 6.4 Total number of incorrectly arranged pairs per test sample in relative ranking 6.5 Total percentage