Human Pose Recovery and Behavior Analysis Group ChaLearn Looking at People 2015 new competitions: Age Estimation and Cultural Event Recognition Sergio Escalera, UB & CVC & ChaLearn Jordi Gonzàlez, UAB & CVC Xavier Baró, UOC & CVC Pablo Pardo, UB Junior Fabian, UAB & CVC Marc Oliu, UB & CVC Hugo J. Escalante, INAOE & ChaLearn Ivan Huerta, University of Venezia Isabelle Guyon, ChaLearn http://gesture.chalearn.org/ All rights reserved HuBPA© Human Pose Recovery and Behavior Analysis Group http://gesture.chalearn.org/ Index • ChaLearn Looking at People events • ChaLearn Looking at People Cultural Event Recognition • ChaLearn Looging at People Age Recognition • Ongoing and upcoming events 2 Human Pose Recovery and Behavior Analysis Group ChaLearn events http://gesture.chalearn.org/ ChaLearn Gesture Recognition Challenges and Workshops •Quantitative competition: CVPR 2011 - Workshop and Challenge •One-shot learning on Gesture Recognition •New depth-rgb data set CVPR 2012 - Workshop and Challenge •Dictionaries among 5-8 on Gesture Recognition gesture categories •Leveinstein: recognizing list ICPR 2012 - Workshop and Challenge of sequences within each on Gesture Recognition sequence •Quantitative competition: ICMI 2013 - Workshop and •User independent multiple instance Challenge on Gesture learning Recognition •New depth-rgb-mask-skeleton- ECCV 2014 – Workshop and audio data set Challenge on Human Pose, •Dictionary of 20 gesture categories Action and Gesture •Leveinstein: recognizing list of Recognition gestures within each sequence 3 Human Pose Recovery and Behavior Analysis Group ChaLearn events http://gesture.chalearn.org/ ECCV 2014 – Workshop and Challenge on Human Pose, Action and Gesture Recognition CVPR 2015- ChaLearn Looking at People 2015 - Action spotting and cultural event recognition ICCV 2015- ChaLearn Looking at People 2015 - Age recognition and extended cultural event recognition 4 Human Pose Recovery and Behavior Analysis Group ChaLearn events http://gesture.chalearn.org/ Winners and Invited speakers ICMI 2013 winners ECCV 2014 winners CVPR 2015 winners Invited speakers: Antonis Argyros, Takeo Kanade, Deva Ramanan, Stan Sclaroff, Cordelia Schmid, Fernando de la Torre, Jeffrey Cohn, Tinne Tuytellars, Leonid Sigal, Larry Davis, among others. Special issues: JMLR, TPAMI, (IJCV LaP opened until february 2016) 5 Human Pose Recovery and Behavior Analysis Group Cultural event http://gesture.chalearn.org/ Challenge on cultural event recognition •Track on Cultural Event Recognition: More than 10,000 images corresponding to 50 different cultural event categories will be considered. Examples of cultural events will be Carnival (Brasil, Italy, USA), Oktoberfest (Germany), San Fermin (Spain), Maha-Kumbh- Mela (India) and Aoi-Matsuri (Japan), among others. 6 Human Pose Recovery and Behavior Analysis Group Cultural event http://gesture.chalearn.org/ •Track on Cultural Event Recognition: More than 10,000 images corresponding to 50 different cultural event categories will be considered. • First dataset on cultural events •10.000 images corresponding to 50 cultural events. • Person related events. • High intra and low inter-class variability. • Different cues can be exploited like garments, human poses, crowds analysis, objects and background scene. 7 Human Pose Recovery and Behavior Analysis Group Cultural event http://gesture.chalearn.org/ •Track on Cultural Event Recognition: More than 10,000 images corresponding to 50 different cultural event categories will be considered. Inter-class variability 8 Human Pose Recovery and Behavior Analysis Group Cultural event http://gesture.chalearn.org/ •Track on Cultural Event Recognition: More than 10,000 images corresponding to 50 different cultural event categories will be considered. Inter-class variability Carnival of Dunkerque Carnival of Rio Carnival of Venice Carnival of Helsinki Nothing Hill Carnival Carnival of Quebec 9 Human Pose Recovery and Behavior Analysis Group Cultural event http://gesture.chalearn.org/ •Track on Cultural Event Recognition: More than 10,000 images corresponding to 50 different cultural event categories will be considered. Inter-class variability Quebec Winter Carnival Harbin Ice and Snow Festival Average Precision evaluation For each image, participants submit their confidence for all the categories (Average Precision). 10 Cultural Event Country #Images 1. Annual Buffalo Roundup USA 334 2. Ati-atihan Philippines 357 3. Ballon Fiesta USA 382 4. Basel Fasnacht Switzerland 310 5. Boston Marathon USA 271 6. Bud Billiken USA 335 Human Pose Recovery and Behavior Analysis Group 7. Buenos Aires Tango Festival Argentina 261 Cultural8. Carnival of Dunk erqueevent France 389 9. Carnival of Venice Italy 455 10. Carnival of Rio Brazil 419 11. Castellers Spain 536 12. Chinese New Year China http://gesture.chalearn.org/296 13. Correfocs Catalonia 551 •Track on Cultural Event Recognition: More 14.thanDesert Festi10val,of000Jaisalmer imagesIndia 298 15. Desfile de Silleteros Colombia 286 corresponding to 50 different cultural event categories will16. D´ıa bede los consideredMuertos . Mexico 298 Cultural Event Country #Images 17. Diada de Sant Jordi Catalonia 299 18. Diwali Festival of Lights India 361 1. Annual Buffalo Roundup USA 334 19. Falles Spain 649 2. Ati-atihan Philippines 357 20. Festa del Renaixement Tortosa Catalonia 299 3. Ballon Fiesta USA 382 21. Festival de laMarinera Peru 478 4. Basel Fasnacht Switzerland 310 22. Festival of the Sun Peru 514 5. Boston Marathon USA 271 23. Fiesta de laCandelaria Peru 300 6. Bud Billiken USA 335 24. Gion matsuri Japan 282 7. Buenos Aires Tango Festival Argentina 261 25. Harbin Ice and Snow Festival China 415 8. Carnival of Dunkerque France 389 26. Heiva Tahiti 286 9. Carnival of Venice Italy 455 27. Helsinki Samba Carnival Finland 257 10. Carnival of Rio Brazil 419 28. Holi Festival India 553 11. Castellers Spain 536 29. Infiorata di Genzano Italy 354 12. Chinese New Year China 296 30. LaTomatina Spain 349 13. Correfocs Catalonia 551 31. Lewes Bonfire England 267 Figure14. Desert3. CulturalFestival eofventsJaisalmersample images.India 298 15. Desfile de Silleteros Colombia 286 32. Macys Thanksgiving USA 335 16. D´ıa de los Muertos Mexico 298 33. Maslenitsa Russia 271 2011-12, the number17. Diadaofde SantimagesJordi is similar,Cataloniaaround 11, 000299 , 34. Midsommar Sweden 323 but the number18.ofDiwcateali goriesFestival ofisLightshere increasedIndiamore than361 5 35. Notting hill carnival England 383 36. Obon Festival Japan 304 times. 19. Falles Spain 649 20. Festa del Renaixement Tortosa Catalonia 299 37. Oktoberfest Germany 509 38. Onbashira Festival Japan 247 4. Pr otocol and21. Festievvalaluationde laMarinera Peru 478 22. Festival of the Sun Peru 514 39. Pingxi Lantern Festival Taiwan 253 This section23.introducesFiesta de laCandelariathe protocol and ePeruvaluation met-300 40. Pushkar Camel Festival India 433 41. Quebec Winter Carnival Canada 329 rics for both tracks.24. Gion matsuri Japan 282 25. Harbin Ice and Snow Festival China 415 42. Queens Day Netherlands 316 4.1. Evaluation26. Heiprva ocedure for action/interactionTahiti 286 43. Rath Yatra India 369 track 27. Helsinki Samba Carnival Finland 257 44. SandFest USA 237 28. Holi Festival India 553 45. San Fermin Spain 418 To evaluate29.theInfiorataaccuracdi Genzanoy of action/interactionItaly recogni-354 46. Songkran Water Festival Thailand 398 tion, we use the30.JaccardLaTomatinaIndex, the higher theSpainbetter. Thus,349 47. St Patrick’s Day Ireland 320 31. Lewes Bonfire England 267 48. The Battle of the Oranges Italy 276 Figure 3. Cultural events sample images. for then action and interaction categorieslabeled for aRGB sequence s, the32.JaccardMacys ThanksgiIndexvingisdefined as: USA 335 49. Timkat Ethiopia 425 33. Maslenitsa Russia 271 50. Viking Festival Norway 262 11 34. Midsommar T Sweden 323 Table 3. List of the 50 Cultural Events. 2011-12, the number of images is similar, around 11, 000, As,n B s,n 35. NottingJ hill= carnival S , England 383(1) but the number of categories is here increased more than 5 s,n 36. Obon FestivalAs,n B s,n Japan 304 times. where As,n isthe37. Oktoberfestground truth of action/interactionGermany n at509se- Jaccard Index among all categories for all sequences, where 38. Onbashira Festival Japan 247 quence s, and B s,n istheprediction for such an action at se- motion categories are independent but not mutually exclu- 4. Pr otocol and evaluation 39. Pingxi Lantern Festival Taiwan 253 quence s. As,n and B s,n are binary vectors where 1-values si ve (in a certain frame more than one action, interaction, This section introduces the protocol and evaluation met-correspond to frames40. PushkarinCamelwhichFestitheval n− th actionIndiaisbeing 433per- gesture class can be active). 41. Quebec Winter Carnival Canada 329 rics for both tracks. formed. The participants were evaluated based on the mean In the case of false positives (e.g. inferring an action 42. Queens Day Netherlands 316 4.1. Evaluation procedure for action/interaction 43. Rath Yatra India 369 track 44. SandFest USA 237 45. San Fermin Spain 418 To evaluate the accuracy of action/interaction recogni- 46. Songkran Water Festival Thailand 398 tion, we use the Jaccard Index, the higher the better. Thus, 47. St Patrick’s Day Ireland 320 for then action and interaction categorieslabeled for aRGB 48. The Battle of the Oranges Italy 276 sequence s, the Jaccard Index isdefined as: 49. Timkat Ethiopia 425 50. Viking Festival Norway 262 T Table 3. List of the 50 Cultural Events. A s,n B s,n Js,n