AN EVALUATION of the FDS Revision 11 2 16
Total Page：16
File Type：pdf, Size：1020Kb
! AN EVALUATION OF SPORTRADAR’S FRAUD DETECTION SYSTEM by David Forrest Professor of Economics, University of Liverpool, UK and Ian G. McHale Professor of Sports Analytics, University of Salford, UK September, 2015 ! ! CONTENTS SUMMARY ............................................................................................................................... 1! 1! INTRODUCTION ............................................................................................................... 2! 1.1! The purpose and scope of this Report .............................................................................................. 2! 1.2! Conceptual framework ..................................................................................................................... 3! 1.3! Criteria for assessing screening systems .......................................................................................... 6! 1.4! Structure of the Report ..................................................................................................................... 9! 2! THE QUALITY, SCOPE AND RELIABILITY OF THE DATA USED IN THE FDS .. 12! 2.1! Introduction .................................................................................................................................... 12! 2.2! Odds Data ....................................................................................................................................... 12! The scope of the data ........................................................................................................................... 12! The quality of the data ......................................................................................................................... 15! 2.3! Sports Data ..................................................................................................................................... 16! 2.4! Synchronisation of Odds and Sports Data ..................................................................................... 17! 3! THE MATHEMATICAL MODELS OF THE FDS .......................................................... 19! 3.1! The role of the mathematical models ............................................................................................. 19! 3.2! The pre-match model ..................................................................................................................... 20! 3.3! The in-play model .......................................................................................................................... 23! The in-play model in practice .............................................................................................................. 24! Testing the efficacy of the in-play model ............................................................................................. 27! 4! THE ALERT PROCESS: STAGE 1 IN THE FDS ........................................................... 29! 4.1! The triggering of alerts pre-match ................................................................................................. 29! The general approach ......................................................................................................................... 29! Criteria for creation of an alert .......................................................................................................... 30! Additional criteria for alerts ............................................................................................................... 32! Setting thresholds for alerts (‘configuration’) .................................................................................... 33! 4.2! The triggering of alerts in-play ...................................................................................................... 35! 4.3! Are the thresholds set appropriately? ............................................................................................. 38! 5! STAGE 2 IN THE FDS: HOTLISTING AND ESCALATION ........................................ 40! i ! ! 5.1! Introduction .................................................................................................................................... 40! 5.2! Hotlisting ....................................................................................................................................... 41! Description of the process ................................................................................................................... 41! Are the analysts adequately provided with information? .................................................................... 42! Are the analysts appropriately qualified to make the judgement? ...................................................... 44! 5.3! The escalation process ................................................................................................................... 45! The input from freelancers .................................................................................................................. 47! The decision meeting ........................................................................................................................... 50! The report on a match classified as suspicious ................................................................................... 51! Conclusion ........................................................................................................................................... 53! 6! CASE STUDIES: THE FDS IN ACTION ........................................................................ 55! 6.1! Introduction .................................................................................................................................... 55! 6.2! Case studies .................................................................................................................................... 56! Australia .............................................................................................................................................. 56! Austria ................................................................................................................................................. 58! Estonia and Latvia .............................................................................................................................. 59! 7! SOME REFLECTIONS ..................................................................................................... 62! 8! CONCLUSIONS ............................................................................................................... 65! Appendix A:! List of bookmakers monitored within the FDS .............................................. 67! Appendix B:! Odds database and screenshot details ............................................................. 71! Appendix C:! Empirically testing the in-play model. ........................................................... 77! Appendix D:! Procedures for hotlisting and escalation. ........................................................ 81! ii ! ! SUMMARY In this Report, we examine the likely efficacy of Sportradar’s Fraud Detection System (FDS), which monitors betting markets for indications that a (football) match may have been manipulated. As with any screening system, its efficacy is appropriately evaluated by reference to its sensitivity (what proportion of manipulated matches does it identify?) and its specificity (what proportion of matches identified as likely to have been manipulated are true cases of manipulation?). Our evaluation was based principally on analysing the reliability in construction and execution of each component of the system, both those based on statistical algorithms and those where expert analysts form a final judgement. We also considered case studies of matches known, from external evidence, to have been manipulated Our conclusion is, first, that, while the level of sensitivity cannot be evaluated precisely, the FDS is likely to identify correctly a significant proportion of manipulated matches. Second, given the quite high proportion of matches identified as potentially suspicious by the statistical algorithms operated by the FDS and the relatively low number of reports finally issued to client organisations, we view analysts at the FDS as being both thorough and cautious when the final decision is taken on whether to classify a match as suspicious. Therefore we believe that the specificity of the screen is likely to be high and that few false positives will be presented. ! 1 ! ! 1 INTRODUCTION 1.1 The purpose and scope of this Report During the last five years in particular, there has been growing awareness of the scale and level of threat to the integrity of organised sport, including from international organised crime. Several significant Reports on the problem of match fixing have been produced, for example by l’Institut de Relations Internationales et Stratégiques1 and by the Sorbonne University2. Increasingly, governments have also recognised the need for protective measures to be developed, as evidenced by the Council of Europe Convention on the Manipulation of Sports Competitions, adopted by the Committee of Ministers in July 2014. Against this background, Sportradar offers integrity services to sports federations and competitions and state and law enforcement agencies. Its Fraud Detection System (FDS) monitors betting markets for abnormal activity with a view to identifying fixtures where there is evidence suggestive of manipulation