Looking for the Movie Seven or Sven from the Movie Frozen? A Multi-perspective Strategy for Recommending Queries for Children
I ON M ADRAZO A ZPIAZU, N EVENA D RAGOVIC, O GHENEMARO A NUYAH & S OLE P ERA Department of Computer Science Boise State University Boise, Idaho, USA Research supported in part by National Science Foundation, Award # 1565937. Background: Who is Sven?
2 Let’s search for sven
3 Do query suggestions help?
4 YUM
Easier querying
YUM Search as learning
Easier understanding
5 Proposed method
Which are the main characters of Did you mean? Frozen movie?
“ Frozen main Search Candidate Readability Topical characters ” Intent Ranking Generation Analysis Filtering Analysis “ Frozen character costumes “
“ Disney characters “ ReQuIK “Frozen”
Children Search Intent module UberSuggest Children Resources WordNet Flesh-Kincaid Corpora
7 Search intent identification
Initial Query Sven the amaaazing raindeer of Frozen
Stop-word removal Sven amaaazing raindeer Frozen
Spelling for children Sven amazing reindeer Frozen
Children’s relevant term detection Sven reindeer Frozen
Final Query Sven reindeer Frozen QuIK “Is Sven Seven?” A Search Intent Module for Children, Nevena Dragovic, Ion Madrazo Azpiazu, Maria Soledad Pera, SIGIR 2016
Search Candidate Reading Ranking Diversity Intent Generation Level 8 Candidate generation
• Sven and ole Sven a • Sven and olaf
• Sven berg Sven b • Sven buerki
• Sven coop Sven c • Sven counters …
• Sven zatec Sven z • Sven zimmer
Search Candidate Reading Ranking Diversity Intent Generation Level 9 Ranking
Search Candidate Reading Ranking Diversity Intent Generation Level 10 Ranking: Deep Model
❑ Word Embeddings ❑ Recurrent Long Short Term Memory Layer ❑ Fully Connected
Search Candidate Reading Ranking Diversity Intent Generation Level 11 Ranking: Wide Model
th Wide model features; where q represents a query, qi is the i non-stop word, lemmatized term in q, and |q| is the length of non-stop words in q.
Search Candidate Reading Ranking Diversity Intent Generation Level 12 Reading level filtering
8th Grade 12th Grade
15th Grade
11th Grade
Search Candidate Reading Ranking Diversity Intent Generation Level 13 Ensuring diversity
Wordnet Based Similarity Image courtesy: FreeSandal.org
Search Candidate Reading Ranking Diversity Intent Generation Level 14 Evaluation
Not an option Module-wise evaluation
15 Data: Query gathering
lollipop How long are… ? cheetas frozen batman Star wars
K-9 Idaho Teachers 50 Children 300 open queries Publicly Available
Dataset available at: http://scholarworks.boisestate.edu/cs_scripts/5/
16 Data: Other resources
1M sentences based on children content
17 Compared strategies
18 Evaluation: Search intent
Google Bing Yahoo Kidzsearch CQS ReQuik 46% 36% 65% 76% 57% 94%
Percentage of queries that trigger a recommendation in each compared system.
19 Classifying children queries
Wide Deep Wide and Deep Accuracy 68% 92% 94%
Performance of diverse ranking strategies.
20 Can we use this model for ranking?
Precision@K, where K is defined as the Model assessment based on Precision@K, percentage of queries and sentences analyzed. where K is the number of queries examined. (K is defined as a percentage, as the raw counts of sentences and queries are not comparable otherwise)
21 Readability of retrieved documents
ReQuik Google Bing Yahoo! CQS 7.71 12.46 19.96 11.42 11.82 AskKids Ipl2 Sweet Search KidRex Kiddle 13.3 10.9 12.3 12.7 12.83
Average Readability of top-3 documents retrieved for test query recommendations.
22 Online assessment
❑ 11 Teachers ❑ 5 schools ❑ 10 initial queries ❑ Each teacher could select at most 2 queries
23 Online assessment
0.6 Overall Unigram Bigram N-gram 0.51 0.5 0.45 0.42 0.4 0.4 0.36 0.33 0.34 0.3
0.3 0.27 0.28 Accuracy 0.2 0.12 0.1 0.05 0.06 0.06 0.03 0.02 0 CQS Google Kidzsearch ReQuIK Comparison of query recommendation systems.
24 Conclusions And Future Work Main Contributions: ❑ReQuIK ❑Deep & wide model – trained on children sentences; to account for limited query log availability ❑Readability-based prioritization of queries ❑Children query dataset Future Work ❑More exhaustive evaluation ❑Gather more queries ❑Better suitability analysis
25 Thank You!
LOOKING FOR THE MOVIE SEVEN OR SVEN FROM THE MOVIE FROZEN? A Multi-perspective Strategy for Recommending Queries for Children
Ion Madrazo Azpiazu Nevena Dragovic Oghenemaro Anuyah Maria Soledad Pera
Research supported in part by National Science Foundation, Award # 1565937.
26