View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Queen Mary Research Online Probabilistic retrieval models - relationships, context-specific application, selection and implementation Wang, Jun The copyright of this thesis rests with the author and no quotation from it or information derived from it may be published without the prior written consent of the author For additional information about this publication click this link. https://qmro.qmul.ac.uk/jspui/handle/123456789/655 Information about this research object was correct at the time of download; we occasionally make corrections to records, please therefore check the published record when citing. For more information contact [email protected] 1 Probabilistic Retrieval Models - Relationships, Context-specific application, Selection and Implementation Jun Wang Thesis submitted for the degree of Doctor of Philosophy at Queen Mary, University of London December 2010 2 3 Declaration of originality I hereby declare that this thesis, and the research to which it refers, are the product of my own work, and that any ideas or quotations from the work of other people, published or otherwise, are fully acknowledged in accordance with the standard referencing practices of the discipline. The material contained in this thesis has not been submitted, either in whole or in part, for a degree or diploma or other qualification at the University of London or any other University. Some parts of this work have been previously published as: • Jun Wang and Thomas Roelleke, “Statistical query features and correlation to performance of TF-IDF and language modelling” (Submitted) • Thomas Roelleke, Jun Wang, Jan Frederick Forst and Hengzhi Wu, “Informativeness and Probability Mixtures: On the Symmetry of TF-IDF and Language Modelling” (In prepar- ing for submission) • Thomas Roelleke and Jun Wang, “TF-IDF Uncovered: A Study of Theories and Proba- bilities” in 31st ACM SIGIR Conference on Research and Development in Information Retrieval, Singapore, 2008 • Thomas Roelleke, Heng Zhi Wu, Jun Wang, and Hany Azzam. “Modelling retrieval mod- els in a probabilistic relational algebra with a new operator: The relational bayes”, VLDB Journal, 2008. • Thomas Roelleke and Jun Wang, “Probabilistic Logical Modelling of the Binary Inde- pendence Retrieval Model” in 1st international conference on the theory of information retrieval, Budapest, 2007 • Thomas Roelleke and Jun Wang, “A Parallel Derivation of Probabilistic Information Re- trieval Models” in 29th ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, Seattle, 2006 4 Declaration of originality • Jun Wang and Thomas Roelleke, “Context-specific Inverse Document Frequency for Struc- tured Document Retrieval” in European Conference on Information Retrieval, London , 2006 jun wang 5 Abstract Retrieval models are the core components of information retrieval systems, which guide the doc- ument and query representations, as well as the document ranking schemes. TF-IDF, binary independence retrieval (BIR) model and language modelling (LM) are three of the most influ- ential contemporary models due to their stability and performance. The BIR model and LM have probabilistic theory as their basis, whereas TF-IDF is viewed as a heuristic model, whose theoretical justification always fascinates researchers. This thesis firstly investigates the parallel derivation of BIR model, LM and Poisson model, wrt event spaces, relevance assumptions and ranking rationales. It establishes a bridge between the BIR model and LM, and derives TF-IDF from the probabilistic framework. Then, the thesis presents the probabilistic logical modelling of the retrieval models. Vari- ous ways of how to estimate and aggregate probability, and alternative implementation to non- probabilistic operator are demonstrated. Typical models have been implemented. The next contribution concerns the usage of of context-specific frequencies, i.e., the frequen- cies counted based on assorted element types or within different text scopes. The hypothesis is that they can help to rank the elements in structured document retrieval. The thesis applies context-specific frequencies on term weighting schemes in these models, and the outcome is a generalised retrieval model with regard to both element and document ranking. The retrieval models behave differently on the same query set: for some queries, one model performs better, for other queries, another model is superior. Therefore, one idea to improve the overall performance of a retrieval system is to choose for each query the model that is likely to perform the best. This thesis proposes and empirically explores the model selection method according to the correlation of query feature and query performance, which contributes to the methodology of dynamically choosing a model. In summary, this thesis contributes a study of probabilistic models and their relationships, the probabilistic logical modelling of retrieval models, the usage and effect of context-specific frequencies in models, and the selection of retrieval models. 6 7 Contents 1 Introduction 21 1.1 Contributions ................................... 24 1.1.1 Relationships between Retrieval Models . ....... 24 1.1.2 Probabilistic Modelling of Retrieval Models . ......... 24 1.1.3 Context-specific Frequencies in Retrieval Models . .......... 24 1.1.4 Model Selection by Statistic Query Features . ........ 25 1.2 OverviewoftheThesis ..... ...... ..... ...... ..... .. 25 2 Background 27 2.1 History of Retrieval Models . ..... 27 2.2 BooleanModel ................................... 28 2.3 VectorSpaceModel................................ 29 2.3.1 General Vector Space Model And Latent Semantic Analysis ....... 30 2.4 Probabilistic Retrieval Model . ....... 31 2.4.1 Probabilistic Ranking Principle (PRP) . ....... 31 2.4.2 Binary Independence Retrieval (BIR) Model . ...... 32 2.4.3 Variations of the BIR Term Weight . 35 2.4.4 BIR without Relevance Information . 36 2.4.5 BIR with Little Relevance Information . ..... 36 2.5 PoissonModel ................................... 37 2.6 BM25........................................ 39 2.7 Divergence From Randomness (DFR) Model . ..... 41 2.8 LanguageModelling ............................... 43 2.8.1 SmoothinginLM.............................. 44 2.8.2 RelevanceinLM .............................. 45 2.9 Term Weighting and Probabilistic Estimation . .......... 46 8 Contents 2.9.1 Probability Estimation . 46 2.9.2 Entropy................................... 48 2.9.3 Inverse Document Frequency . 50 2.10 Normalization.................................. 51 2.11Summary ...................................... 54 3 Relationships between Models 55 3.1 Probability Spaces for the IR Models . ....... 56 3.1.1 SampleSpaceoftheModels . 56 3.1.2 Probability Estimation . 57 3.1.3 Relevance Assumptions . 59 3.2 RankingRationales ............................... 60 3.3 PoissonBridgeBIRandLM . 60 3.4 TF-IDF’sExplanationwithBIRandLM . ..... 62 3.4.1 Dual Application of BIR and LM Parameters . 62 3.4.2 Dual Representation with IDF and ILF . 63 3.5 Explanation of TF-IDF with Term Disjointness or Independence Assumption . 63 3.5.1 Independent Terms: P(q|d,c)........................ 63 3.5.2 Independent Terms: P(d|q,c)........................ 65 3.5.3 Independent Terms: O(r|d,q)........................ 67 3.5.4 Disjoint Terms: P(q|d,c) .......................... 69 3.5.5 Disjoint Terms: P(d|q,c) .......................... 70 3.5.6 Disjoint Terms: P(d,q|c) .......................... 70 3.6 Document and Query Independence (DQI) Model . ....... 72 3.6.1 DQIandTF-IDF .............................. 73 3.6.2 DQIandMutualEntropy. 74 3.7 Summary ...................................... 75 4 Implementing Retrieval Models with High Abstract Languages 77 4.1 HighAbstractLanguages . 78 4.1.1 Probabilistic Relation Algebra (PRA) . ...... 78 4.1.2 ProbabilisticSQL(PSQL) . 85 9 4.2 Probabilistic Relational Modelling of Retrieval Models .............. 86 4.2.1 SimpleModellingExample . 86 4.2.2 TF-IDFModelling ............................. 88 4.2.3 Binary Independence Retrieval Model . ..... 93 4.2.4 LanguageModelling ............................ 99 4.2.5 BM25....................................105 4.2.6 Divergence From Randomness . 109 4.3 Modelling Precision and Recall . 110 4.4 Summary ......................................113 5 Context-specific Frequencies in Probabilistic Retrieval Models 115 5.1 Structured Document Retrieval . 116 5.2 Motivation of Context-specific Frequencies in Structure Document Retrieval . 118 5.2.1 Inverse Document or Element Frequency? . 118 5.2.2 Element Ranking with Context-specific Frequencies . .........120 5.3 Retrieval in Structured Document Collections . ...........121 5.3.1 Document Ranking with Context-specific IDF in Multi-collection Retrieval122 5.3.2 Collection Ranking with Context-specific Frequencies in Multi- collection Retrieval . 124 5.4 Related Work on Structured Document Retrieval . .........125 5.5 Context-specific Frequencies Definition . .........126 5.6 RSV with Context-specific Frequencies . .......128 5.7 Experiments with Context-specific Frequencies . ...........130 5.7.1 Experimental Settings . 130 5.7.2 Document Retrieval with TF-IDF in the Multi-collections Organized by Subject ...................................131 5.7.3 Analysis of TF-IDF Document Retrieval Results from the Multi- collections Organized by Subject . 134 5.7.4 Document Retrieval with TF-IDF in Multi-collections Organized without