Assessing Team Strategy using Spatiotemporal Data Patrick Lucey Dean Oliver Disney Research ESPN Pittsburgh, PA, USA Bristol, CT, USA [email protected] [email protected] Peter Carr Joe Roth Iain Matthews Disney Research Disney Research Disney Research Pittsburgh, PA, USA Pittsburgh, PA, USA Pittsburgh, PA, USA [email protected] [email protected] [email protected] ABSTRACT Canada USA In CVPR,2009. 9(3) Shots (on Goal) 12(4) [16] W. Lu, J. Ting, K. Murphy, and J. Little. Identifying The Moneyball revolution coincided with a shift in the way 13 Fouls 11 Players in Broadcast Sports Videos using Conditional professional sporting organizations handle and utilize data in 3 Corner Kicks 8 Random Fields. In CVPR,2011. 1O↵sides 4 [17] P. Lucey, A. Bialkowski, P. Carr, E. Foote, and terms of decision making processes. Due to the demand for 38% Time of Possession 62% I. Matthews. Characterizing Multi-Agent Team better sports analytics and the improvement in sensor tech- 3 Yellow Cards 1 Behavior from Partial Team Tracings: Evidence from 0 Red Cards 0 nology, there has been a plethora of ball and player tracking the English Premier League. In AAAI,2012. 4 Saves 3 information generated within professional sports for analyt- [18] L. Madden. NFL to Follow Army’s Lead on Helmet ical purposes. However, due to the continuous nature of the (a) (b)Sensors in Attempt to Prevent Head Injury. data and the lack of associated high-level labels to describe www.forbes.com/sites/lancemadden/2012/07/16/ 8. REFERENCESFigure 1: (a) An example of standard soccernfl-to-follow-armys-lead-on-helmet-sensors-in/ statis- it - this rich set of information has had very limited use espe-[1] S. Alitics and based M. Shah. on Floor hand-labeled Fields for Tracking event in High data which-attempt-to-prevent-head-injury/ describe 16 July 2012. ” cially in the analysis of a team's tactics and strategy. In this Densitywhat Crowdhappened. Scenes. In ECCV (b),2008. Spatiotemporal data[19] has R. Masheswaran, the po- Y. Chang, A. Henehan, and paper, we give an overview of the types of analysis currently[2] N. Allen,tential J. Templon, to describe P. McNally, the L. Birnbaum,where and and how, butS. Danesis. as it Destructing is the Rebound with Optical K. Hammond. StatsMonkey: A Data-Driven Sports performed mostly with hand-labeled event data and high- a continuous signal which is not associatedTracking with Data. a In MIT Sloan Sports Analytics Narrative Writer. In AAAI Fall Symposium Series, Conference,2012. light the problems associated with the influx of spatiotem- 2010.fixed event, using this data for analysis[20] is V. difficult. Morariu and L. Davis. Multi-Agent Event poral data. By way of example, we present an approach[3] BBC-Sports. Footballers may trial wearing microchips Recognition in Structured Scenarios. In CVPR,2011. which uses an entire season of ball tracking data from the to monitor health. [21] T. Moskowitz and L. Wertheim. Scorecasting: The www.bbc.co.uk/sport/0/football/21460038 English Premier League (2010-2011 season) to reinforce the 1. INTRODUCTION ,14Feb Hidden Influences Behind How Sports Are Played and 2013. Games Are Won. Crown Publishing Group, 2011. common held belief that teams should aim to \win home[4] M. Beetz,In N. his von 2003 Hoyningen-Huene, book Moneyball B. Kirchlechner,[14], Michael Lewis docu- mented how Oakland A's General Manager Billy[22] NBA Beane Shot was Charts. www.nba.com/hotspots. games and draw away ones". We do this by: i) forming a S. Gedikli, F. Siles, M. Durus, and M. Lames. [23] D. Oliver. Basketball on Paper: Rules and Tools for representation of team behavior by chunking the incoming ASPOGAMO:able to Automatedeffectively Sports use Game metrics Analysis derived from hand-craftedPerformance Analysis. Brassey’s, Incorporated, 2004. Models. International Journal of Computer Science in spatiotemporal signal into a series of quantized bins, and ii) statistics to exploit the inefficiencies in the[24] value D. Oliver. of indi- Guide to the Total Quarterback Rating. Sport,8(1),2009. espn.go.com/nfl/story/_/id/6833215/ generate an expectation model of team behavior based on[5] P. Carr,vidual Y. Sheikh, baseball and I.players. Matthews. Around Monocular the same time, Basketball on Paper [24] was published which outlined methodsexplaining-statistics-total-quarterback-rating for , a code-book of past performances. We show that home ad- Object Detection using 3D Geometric Primitives. 4 August 2011. vantage in soccer is partly due to the conservative strategy 2012.valuing player performance in basketball which[25] is Opta a far Sports. morewww.optasports.com. of the away team. We also show that our approach can flag[6] K. Goldsberry.challenging CourtVision: problem New because Visual and it is Spatial a continuous[26]team S. Pellegrini, sport. A. Ess, K. Schindler, and L. van Gool. Analytics for the NBA. In MIT Sloan Sports Analytics You’ll Never Walk Alone: Modeling Social Behavior anomalous team behavior which has many potential appli- ConferenceDue to,2012. the popularity and effectiveness of the tools that em- for Multi-Target Tracking. In CVPR,2009. [7] A. Gupta,anated P. Srinivasan,from these J. Shi, works, and L. there Davis. has been enormous interest cations. [27] M. Perse, M. Kristan, S. Kovacic, and J. Pers. A Understanding Videos, Constructing Plots: Learning a in the field of sports analytics over the last 10Trajectory-Based years with Analysis of Coordinated Team Visuallymany Grounded organizations Storyline Model (e.g. from professional Annotated teams, mediaActivity groups) in Basketball Game. Computer Vision and Categories and Subject Descriptors Videos.housing In CVPR their,2009. own analytics department. However,Image nearly Understanding all ,2008. [8] Hawk-Eye. www.hawkeyeinnovations.co.uk. H.4 [Information Systems Applications]: Miscellaneous; of the analytical works have dealt solely with[28] hand-labeled Prozone. www.prozonesports.com. [9] D. Henschen. IBM Serves New Tennis Analytics At [29] B. Siddiquie, Y. Yacoob, and L. Davis. Recognizing I.2.6 [Learning]: General event datawww.informationweek.com/software/ which describes what happened (e.g. basketball Wimbledon. Plays in American Football Videos. Technical report, business-intelligence/ - rebounds, points scored, assists, football - yardsUniversity per carry, of Maryland, 2009. ibm-serves-new-tennis-analytics-at-wimbl/ tackles, sacks, soccer - passes, shots, tackles (see[30] Figure SportsVision. 1(a))).www.sportsvision.com. Keywords 240002528, 23 June 2012. [31] STATS SportsVU. www.sportvu.com. Sports Analytics, Spatiotemporal Data, Representation [10] A. HervieuOnce andthe P. data Bouthemy. is in Understandingthis form, most sports approaches just relate videoto using parsing players in trajectories. the relevant In J. Zhang, data L. from Shao, a database,[32] Statsheet. then ap-www.statsheet.com. L. Zhang,plying and sport-based G. Jones, editors, rulesIntelligent and standard Video statistical[33] D. methods,Stracuzzi, A. Fern, K. Ali, R. Hess, J. Pinto, N. Li, T. Konik, and D. Shapiro. An Application of Transfer Eventincluding Analysis and regression Understanding and. Springeroptimization. Berlin / Heidelberg, 2010. to American Football: From Observation of Raw Permission to make digital or hard copies of all or part of this work for[11] S. IntilleAs and most A. Bobick. sporting A Framework environments for Recognizing tend to be dynamicVideo to with Control in a Simulated Environment. AI personal or classroom use is granted without fee provided that copies are Multi-Agentmultiple Action players from continuously Visual Evidence. moving In AAAI and, competingMagazine against,32(2),2011. not made or distributed for profit or commercial advantage and that copies 1999.each other, simple event statistics do not capture[34] X. Wei, the P.com- Lucey, S. Morgan, and S. Sridharan. bear this notice and the full citation on the first page. To copy otherwise, to[12] K. Kim, M. Grundmann, A. Shamir, I. Matthews, Sweet-Spot: Using Spatiotemporal Data to Discover plex aspects of the game. To gain an advantageand over Predict the Shots in Tennis. In MIT Sloan Sports republish, to post on servers or to redistribute to lists, requires prior specific J. Hodgins, and I. Essa. Motion Fields to Predict Play Evolutionrest of in Dynamicthe field, Sports sporting Scenes. organizations In CVPR,2010. have recentlyAnalytics looked Conference,2013. permission and/or a fee. [35] C. Xu, Y. Zhang, G. Zhu, Y. Rui, H. Lu, and KDD’13, August 11–14, 2013, Chicago, Illinois, USA. [13] M. Lewis.to employMoneyball: commercial The Art of Winning tracking an technologies Unfair which can lo- Gamecate. Norton, the position 2003. of the ball and players at each timeQ. Huang. instant Using Webcast Text for Semantic Event Copyright 2013 ACM 978-1-4503-2174-7/13/08 ...$15.00. Detection in Broadcast. T. Multimedia,10(7),2008. [14] R. Li and R. Chellappa. Group Motion Segmentation Using a Spatio-Temporal Driving Force Model. In [36] Zonalmarking. www.zonalmarking.net. CVPR,2010. [15] R. Li, R. Chellappa, and S. Zhou. Learning Multi-Modal Densities on Discriminative Temporal 1366Interaction Manifold for Group Activity Recognition. in professional leagues [29, 32, 9, 26]1 - to determine where alizations for the television broadcasters [9]. Partial data and how events happen. Even though there is potentially sources normally generated by human annotators such as an enormous amount of hidden team behavioral information shot-charts in basketball and ice-hockey are often used for to mine from such sources, due to the sheer volume as well analysis [23], as well as passing and shot charts in soccer as the noisy and variable length of the data, methods which [26]. Recently, ESPN developed a new quarterback rating can adequately represent team behaviors are yet to be devel- in American Football called\TotalQBR"[25] which attempts oped.