Looking for the Movie Seven or Sven from the Movie Frozen? A Multi-perspective Strategy for Recommending Queries for Children I ON M ADRAZO A ZPIAZU, N EVENA D RAGOVIC, O GHENEMARO A NUYAH & S OLE P ERA Department of Computer Science Boise State University Boise, Idaho, USA Research supported in part by National Science Foundation, Award # 1565937. Background: Who is Sven? 2 Let’s search for sven 3 Do query suggestions help? 4 YUM Easier querying YUM Search as learning Easier understanding 5 Proposed method Which are the main characters of Did you mean? Frozen movie? “ Frozen main Search Candidate Readability Topical characters ” Intent Ranking Generation Analysis Filtering Analysis “ Frozen character costumes “ “ Disney characters “ ReQuIK “Frozen” Children Search Intent module UberSuggest Children Resources WordNet Flesh-Kincaid Corpora 7 Search intent identification Initial Query Sven the amaaazing raindeer of Frozen Stop-word removal Sven amaaazing raindeer Frozen Spelling for children Sven amazing reindeer Frozen Children’s relevant term detection Sven reindeer Frozen Final Query Sven reindeer Frozen QuIK “Is Sven Seven?” A Search Intent Module for Children, Nevena Dragovic, Ion Madrazo Azpiazu, Maria Soledad Pera, SIGIR 2016 Search Candidate Reading Ranking Diversity Intent Generation Level 8 Candidate generation • Sven and ole Sven a • Sven and olaf • Sven berg Sven b • Sven buerki • Sven coop Sven c • Sven counters … • Sven zatec Sven z • Sven zimmer Search Candidate Reading Ranking Diversity Intent Generation Level 9 Ranking Search Candidate Reading Ranking Diversity Intent Generation Level 10 Ranking: Deep Model ❑ Word Embeddings ❑ Recurrent Long Short Term Memory Layer ❑ Fully Connected Search Candidate Reading Ranking Diversity Intent Generation Level 11 Ranking: Wide Model th Wide model features; where q represents a query, qi is the i non-stop word, lemmatized term in q, and |q| is the length of non-stop words in q. Search Candidate Reading Ranking Diversity Intent Generation Level 12 Reading level filtering 8th Grade 12th Grade 15th Grade 11th Grade Search Candidate Reading Ranking Diversity Intent Generation Level 13 Ensuring diversity Wordnet Based Similarity Image courtesy: FreeSandal.org Search Candidate Reading Ranking Diversity Intent Generation Level 14 Evaluation Not an option Module-wise evaluation 15 Data: Query gathering lollipop How long are… ? cheetas frozen batman Star wars K-9 Idaho Teachers 50 Children 300 open queries Publicly Available Dataset available at: http://scholarworks.boisestate.edu/cs_scripts/5/ 16 Data: Other resources 1M sentences based on children content 17 Compared strategies 18 Evaluation: Search intent Google Bing Yahoo Kidzsearch CQS ReQuik 46% 36% 65% 76% 57% 94% Percentage of queries that trigger a recommendation in each compared system. 19 Classifying children queries Wide Deep Wide and Deep Accuracy 68% 92% 94% Performance of diverse ranking strategies. 20 Can we use this model for ranking? Precision@K, where K is defined as the Model assessment based on Precision@K, percentage of queries and sentences analyzed. where K is the number of queries examined. (K is defined as a percentage, as the raw counts of sentences and queries are not comparable otherwise) 21 Readability of retrieved documents ReQuik Google Bing Yahoo! CQS 7.71 12.46 19.96 11.42 11.82 AskKids Ipl2 Sweet Search KidRex Kiddle 13.3 10.9 12.3 12.7 12.83 Average Readability of top-3 documents retrieved for test query recommendations. 22 Online assessment ❑ 11 Teachers ❑ 5 schools ❑ 10 initial queries ❑ Each teacher could select at most 2 queries 23 Online assessment 0.6 Overall Unigram Bigram N-gram 0.51 0.5 0.45 0.42 0.4 0.4 0.36 0.33 0.34 0.3 0.3 0.27 0.28 Accuracy 0.2 0.12 0.1 0.05 0.06 0.06 0.03 0.02 0 CQS Google Kidzsearch ReQuIK Comparison of query recommendation systems. 24 Conclusions And Future Work Main Contributions: ❑ReQuIK ❑Deep & wide model – trained on children sentences; to account for limited query log availability ❑Readability-based prioritization of queries ❑Children query dataset Future Work ❑More exhaustive evaluation ❑Gather more queries ❑Better suitability analysis 25 Thank You! LOOKING FOR THE MOVIE SEVEN OR SVEN FROM THE MOVIE FROZEN? A Multi-perspective Strategy for Recommending Queries for Children Ion Madrazo Azpiazu Nevena Dragovic Oghenemaro Anuyah Maria Soledad Pera Research supported in part by National Science Foundation, Award # 1565937. 26.