M´ario Amado Alves
Adaptive Hypertext. The shattered document approach
Departamento de Ciˆenciade Computadores da Faculdade de Ciˆenciasda Universidade do Porto 2013-03-13 ii M´ario Amado Alves
Adaptive Hypertext. The shattered document approach
Tese submetida `aFaculdade de Ciˆenciasda Universidade do Porto para obten¸c˜aodo grau de Doutor em Ciˆenciade Computadores
Departamento de Ciˆenciade Computadores da Faculdade de Ciˆenciasda Universidade do Porto
2013-03-13 iv To the memory of my parents Irene and Marius. vi Abstract
We study how adaptive hypertext may improve the utilization of large online docu- ments. We put forth the inter-related concepts of shattered documents, and renoding: splitting a document into components smaller than the page, called noogramicles, and creating each page as a new assemblage of noogramicles each time it is accessed. The adaptation comes from learning the navigation patterns of the usors (authors and readers), and is manifested in the assemblage of pages. Another essential trait of our work is the utilization of user simulation for testing our hypotheses. We have created software simulators and conducted experiments with them to compare several adaptive and non-adaptive configurations. Yet another important aspect of our work was the study and adoption of the technique of spreading activation to explore the network database of the learnt model of travels. We have realised a quantitative evaluation based on utilization quality measures adapted to the problem: session size, session cost.
vii viii Resumo
Estudamos como o hipertexto adaptativo pode melhorar a utiliza¸c˜aode documen- tos em-linha de grande dimens˜ao. Apresentamos os conceitos interrelacionados de document fragmentado e rela¸cagem (recria¸c˜aode n´os):separa¸c˜aodo documento em componentes de dimens˜aoinferior `ap´agina,chamados noogram´ıculos, e cria¸c˜aode cada p´aginacomo uma nova montagem de noogram´ıculosde cada vez que ´eacedida, deste modo criando novos n´osna rede hipertextual, `amedida que a utiliza¸c˜aoprogride. A adapta¸c˜aoprov´emde aprender os padr˜oesde navega¸c˜aodos utilizadores (autores e leitores) e manifesta-se na montagem das p´aginas. Outro tra¸coessencial do nosso trabalho ´eo recurso `a simula¸c˜ao para testar as nossas hip´oteses.Cri´amossimuladores em software e realiz´amosexperiˆenciascom eles para comparar v´ariasconfigura¸c˜oes adaptativas e n˜ao-adaptativas. Ainda outro aspeto importante deste trabalho ´eo estudo e ado¸c˜aoda t´ecnicade propaga¸c˜aoda ativa¸c˜ao para explorar a base-de-dados reticular do modelo de viagens aprendidas. Realiz´amosuma avalia¸c˜ao quantitativa baseada em medidas de qualidade da utiliza¸c˜aoadaptadas ao problema: tamanho da sess˜ao,custo da sess˜ao.
ix x R´esum´e
On ´etudiecomme l’hypertexte adaptatif peut am´eliorerl’utilisation de documents en- ligne de grande dimension. On pr´esente les concepts, en rapport entre eux-m`emes,du document fragment´e et le rela¸cage (recr´eationde noeuds): la s´eparationdu document en des composants de dimension inf´erieure `acelle de la page, nom´esles noogramicules, et la cr´eationde chaque page comme un nouvel assemblage de noogramicules `achaque fois qu’on l’acc`ede, cr´eant ainsi de nouveaux noeuds sur le r´eseauhypertextuel, au fur et `amesure du progr`esde l’utilisation. L’adaptation vient de l’apprentissage des mod`eles de navigation des utilisateurs (auteurs et lecteurs) et elle se r´ev`ele`atravers l’assemblage des pages. Un autre trais essentiel de notre travail, c’est le recours `a la simulation afin de tester nos hypoth`eses. On cr´eades simulateurs logiciels et on r´ealisades exp´eriencesavec eux pour comparer plusieurs configurations adaptatives et non adaptatives. Un autre aspect important de ce travail c’est l’´etudeet l’adoption de la technique de propagation de l’activation pour exploiter la base de donn´esr´eseau du mod`elede voyages appris. On r´ealizaune ´evaluation quantitative support´eepar des mesures de qualit´ed’utilisation, adapt´eesau probl`eme:taille de s´ession,coˆutde s´ession.
xi xii Acknowledgements
I love deadlines. I like the whooshing sound they make as they fly by. Douglas Adams
I am extremely indebted to my advisor Doctor Al´ıpioJorge and co-advisor Doctor Z´e Paulo Leal.
Al´ıpiois the most wise person I know. Each of his numerous advice was always entirely pertinent and convenient. Had I followed them all and this thesis would have been a work of absolute perfection. Al´ıpio’ssupport of my work stood unabated through the unending stream of deadline missing after deadline missing from my part. I was as surprised as I was thankful for this continued support. I was surprised because Al´ıpio is the most wise person I know, and I would expect even a mildly clever person to spot a clear lost case and hastily detach themselves from the dead weight. Paradoxically, the fact that you are now reading these lines on an acceptable if imperfect thesis is a result of Al´ıpiobeing the most wise person I know.
I am indebted to Professor Pavel Brazdil for being the excellent leader of LIAAD - INESC Porto (formerly NIAAD - LIAAC), the laboratory where this whole business started.
I am indebted to Rodolfo Matos, the prolific sysadmin of LIAAD, for his prompt support and enduring friendship.
I am grateful to all NIAAD and DCC members for their support and comradeship.
I am indebted to the Funda¸c˜aopara a Ciˆenciae Tecnologia for supporting four years of doctoral research.
I am indebted to Ada Europe, APPIA, Prolearn, and Universidade Aberta, for conference-
xiii going support.
I am grateful to Cec´ılia.
With the grace of God.
xiv Contents
Abstract vii
Resumo ix
R´esum´e xi
Acknowledgements xiii
List of Tables xxiii
List of Figures xxvii
1 Introduction 1
1.1 Context ...... 1
1.1.1 Rationale for hypertextualization ...... 2
1.1.2 Limitations of hypertext ...... 3
1.2 Solutions ...... 4
1.2.1 Our solution ...... 6
1.2.2 Hypotheses of this work ...... 6
1.3 Main contributions ...... 7
1.4 Structure of the text ...... 8
1.4.1 Notation ...... 9
xv 2 Hypertext 11
2.1 Definition ...... 11
2.2 Terminology ...... 12
2.3 Forms of hypertext ...... 13
2.3.1 The words used ...... 19
2.3.2 Items vs. connections ...... 20
2.3.3 The Monographic Principle ...... 22
2.3.4 Summary of hypertext history ...... 24
2.3.5 The document dogma ...... 24
2.3.6 Information search—the impossible that is done ...... 25
2.3.7 Aporias of adaptation ...... 26
2.3.8 Minor issues ...... 27
2.4 Structure of documents ...... 36
2.4.1 Traditional document structure ...... 36
2.4.2 Standard hypertextualization ...... 37
2.5 Learning systems ...... 40
2.5.1 Adaptive hypertext techniques ...... 41
2.5.2 Learning Systems highlights ...... 44
2.6 Summary ...... 46
3 A new model for adaptive hypertext 49
3.1 Motivation ...... 49
3.1.1 Information, not documents ...... 49
3.1.2 Guidelines for adaptive hypertext ...... 50
3.2 Model design ...... 54
3.2.1 The Shattered Documents model ...... 54
xvi 3.2.2 Adaptive information, and author as first reader ...... 56
3.2.3 Interface design ...... 57
3.2.4 Detailed design with a network data model ...... 60
3.3 Techniques and tools reused ...... 60
3.3.1 A unified model of spreading activation ...... 63
3.3.2 A didactical example ...... 63
3.3.3 Benefits of spreading activation for information retrieval . . . . 66
3.3.4 The generic model ...... 67
3.3.5 About the implementation ...... 70
3.3.6 Leaky Capacitor Model (LCM) ...... 71
3.3.7 Reverberative Circles (RC) ...... 73
3.3.8 Waterline ...... 74
3.4 Algorithms ...... 74
3.4.1 Overview ...... 75
3.4.2 Formalization ...... 76
3.4.3 Start page algorithms ...... 76
3.4.4 Recentring algorithms ...... 77
3.4.5 Learning algorithms ...... 79
3.5 Summary ...... 80
4 Experimental methodology 81
4.1 Simulation ...... 82
4.1.1 Formalization ...... 83
4.2 Experiments ...... 84
4.3 Parameter settings ...... 85
4.3.1 The document ...... 88
xvii 4.3.2 One thousand nodes ...... 88
4.4 Evaluation methodology and measures ...... 91
4.4.1 Session size ...... 91
4.4.2 Session cost ...... 91
4.5 Statistics ...... 92
4.5.1 Common and bottom line statistics ...... 94
4.5.2 Statistics in the Outcomes tables ...... 95
4.5.3 Statistics in the Results tables ...... 96
4.5.4 Statistics in the Evolution tables ...... 96
4.5.5 Alternate terms ...... 97
4.6 Summary ...... 97
5 Results 99
5.1 Main results ...... 100
5.1.1 Session size and success rate ...... 101
5.1.2 Evolution ...... 102
5.2 Testing link types ...... 103
5.3 Testing adaptative techniques ...... 105
5.4 Summary ...... 105
6 Conclusions 107
6.1 Main conclusion ...... 107
6.2 The pros and cons of simulating ...... 107
6.3 Paths not taken ...... 108
6.4 Future work ...... 109
A Electronic archive 127
xviii B Program listings 129
B.1 Package Arm05 Model (spec) ...... 129
B.2 Package Arm05 Model (body) ...... 131
B.3 Procedure Arm05 Model.Get Info (body only) ...... 134
B.4 Package Kasim2 (spec) ...... 137
B.5 Package Kasim2 (body) ...... 143
B.6 Package Kasim2.Activation (spec) ...... 151
B.7 Package Kasim2.Activation (body) ...... 152
B.8 Package Kasim2.Comparate (spec) ...... 160
B.9 Package Kasim2.Comparate (body) ...... 161
B.10 Procedure Kasim2.Comparate.Experiment (body only) ...... 186
B.11 Package Kasim2.Markov (spec) ...... 187
B.12 Package Kasim2.Markov (body) ...... 187
B.13 Package Kasim2.Markov With Heuristics (body) ...... 190
B.14 Package Kasim2.Markov With Heuristics (body) ...... 193
B.15 Package Kasim2.Structural (spec) ...... 197
B.16 Package Kasim2.Structural (body) ...... 197
C Supplemental Items 201
C.1 Software requirements for the Knowledge Atoms design ...... 201
C.2 Kasim 1. First cycle of experiments ...... 202
C.2.1 Second round: 60 atoms, 10 oracles ...... 205
C.2.2 Conclusion of cycle one ...... 207
C.3 Notes on Network Data Models ...... 207
C.3.1 Primary concepts of network data structures ...... 208
C.3.2 The untyped network hypothesis ...... 209
xix C.3.3 Other designs ...... 214
C.3.4 A network calculus ...... 215
C.3.5 Fundamental entities, definitions ...... 215
C.3.6 Useful theorems ...... 216
C.3.7 Proofs, lemmas, axioms ...... 217
D Detailed results of experiments 219
D.0.8 Common and bottom line statistics ...... 221
D.0.9 Statistics in the Outcomes tables ...... 222
D.0.10 Statistics in the Results tables ...... 223
D.0.11 Statistics in the Evolution tables ...... 223
D.0.12 Alternate terms ...... 224
D.1 Complete results of configuration Shattered Document ...... 225
D.2 Complete results of configuration Shattered Document with Random Clicks ...... 233
D.3 Complete results of configuration Shattered Document with Random Pages...... 241
D.4 Complete results of configuration Shattered Document with Random Learning ...... 249
D.5 Complete results of configuration Shattered Document with Random Pages and Random Learning ...... 257
D.6 Complete results of configuration Shattered Document with Random Document (fixed) ...... 265
D.7 Complete results of configuration Structural ...... 273
D.8 Complete results of configuration Shattered Document with high max- imum cost ...... 281
D.9 Complete results of configuration Structural with high maximum cost . 289
xx D.10 Complete results of configuration Shattered Document with Child links only ...... 297
D.11 Complete results of configuration Shattered Document, no Next links, weak Residual ...... 305
D.12 Complete results of configuration Shattered Document, weak Next links 313
D.13 Complete results of configuration Shattered Document with Strong Legacy ...... 321
D.14 Complete results of configuration Shattered Document, dense start . . 329
D.15 Complete results of configuration Shattered Document with Markov Chains ...... 337
D.16 Complete results of configuration Shattered Document with Markov Chains with heuristics ...... 345
D.17 Complete results of configuration Shattered Document, Trimmed Tree . 353
D.18 Complete results of configuration Structural, Trimmed Tree ...... 361
D.19 Complete results of configuration Shattered Document with Markov Chains, Trimmed Tree ...... 369
D.20 Complete results of configuration Shattered Document with Markov Chains with heuristics, Trimmed Tree ...... 377
xxi xxii List of Tables
2.1 Summary of hypertext history ...... 24
4.1 Some scalar parameters of the simulator ...... 87
4.2 Distribution of nodes per level in the ARM ...... 88
4.3 Random sample of sections ...... 90
4.4 Distribution of nodes per level in the Cut ARM ...... 90
4.5 Distribution of nodes per level in the Trimmed ARM ...... 91
5.1 Main results...... 100
5.2 P-values for the main results...... 101
5.3 Results for maximum cost = 6...... 102
5.4 Varying the link weights ...... 104
5.5 Markov chain results ...... 105
xxiii xxiv List of Figures
1.1 Adaptive hypertext process...... 2
1.2 Example of online shop recommendations...... 5
2.1 Screenshot of a Xanadu prototype on Nelson 2007a...... 15
2.2 Illustration of an advanced form of hypertext on Nelson 2007b...... 16
2.3 SWI-Prolog help panel...... 17
2.4 SWI-Prolog help panel for consult/1 ...... 18
2.5 Dictionary...... 19
2.6 SWI-Prolog help for catch/3...... 28
2.7 Alternative to the design in figure 2.6 ...... 28
2.8 The web interface of the network router Linksys WRT54G...... 29
2.9 How compound terms are (mis)treated in Dictionary...... 30
2.10 The Back button aporia. In certain contexts the user expects a chrono- logical behaviour. 33
2.11 The Back button aporia. Why the Back button should not behave chronologically. 33
2.12 Standard hypertextualization of the sequential structure...... 39
2.13 Real look of the first page in figure 2.12...... 39
2.14 The Knowledge Sea interface (KnowledgeSea)...... 46
xxv 2.15 Detail of The Knowledge Sea interface (KnowledgeSea)...... 47
3.1 Model of the same document in figure 2.12 but with the shattered document approach and the two types of connection Next (N) and Child (C)...... 55
3.2 Page made up of document fragments...... 55
3.3 Example design of our own adaptation model...... 59
3.4 Relating the web page as seen and the graph model underneath. . . . . 61
3.5 Travelling ...... 62
3.6 Toy network, for a general understanding of spreading activation . . . . 64
3.7 Instances of spreading activation over the toy network (transcripts of sessions with the Minibrain program)...... 65
3.8 Step algorithm...... 70
3.9 Minibrain usage...... 70
3.10 Super Page algorithm ...... 76
3.11 Main recentring algorithm ...... 77
3.12 Pure Markov chains recentring algorithm ...... 78
3.13 Heuristical Markov chains recentring algorithm ...... 78
4.1 Choose algorithm ...... 84
4.2 Evolution of session size in an exploratory experiment ...... 85
4.3 Top level models and respective components of the experimental setup. 86
4.4 Partial configuration map of the simulator...... 87
4.5 Hierarchical structure of the ARM (excluding annexes)...... 89
5.1 Session cost evolution for the shattered document D.9 ...... 103
5.2 Session costs and sizes for the original structure D.8 ...... 103
xxvi C.1 Hierarchical structure of document X3 (CHILD links) ...... 203
C.2 Complete structure of document X3 ...... 204
C.3 Evolution of session size for each oracle ...... 205
C.4 Hierarchical structure of document X4 (CHILD links) ...... 206
C.5 Evolution of session size ...... 207
C.6 A subset of XQuery use case 1.1.2 (Chamberlin et al. 2005) represented in (a) the original XML format, (b) a typed network a la RDF. This example illustrates various issues discussed in the text...... 209
C.7 The direct attribute structure equates a connection...... 211
C.8 Untyped base structure to represent an explicit, extensively defined set
X = {x1, . . . , xm}...... 212
C.9 The road intersection pattern (a), for representing an (abstract) typed connection (b), in the untyped network base...... 212
C.10 A subset of XQuery use case 1.1.2 (Chamberlin et al. 2005) represented in: (a) the original XML format; (b) a typed network; (c) the untyped network base. This example illustrates various issues discussed in the text...... 213
xxvii xxviii Chapter 1
Introduction
Traveller, there are no paths. Paths are made by walking. Ant´onioMachado
1.1 Context
This thesis studies the process of hypertextualization and hypertext adaptation, as a means to improve the utilization of large online documents. Figure 1.1 illustrates this two-phase transformation process from a traditional document, or set of items, to hypertext, and therefrom to a different, adapted, hypertext—hopefully a better one. The diagram depicts the overall process; the techniques named therein are just a few of many possible ones, all addressed later.
We study the different parts of the process. In general, we delve more deeply into the adaptive parts of the process, situated on the right-hand half of the diagram. Our main original contributions are the concept of renoding, or shattered documents, and a simulator of user behaviour, which we use to automate the evaluation of different hypertext systems.
In sum, we study the large online document, and how its utilization might be improved by means of adaptive hypertext features. Large means an extent such that the document cannot be seen all at once. In other words: large = (much) larger than a screenful.
1 2 CHAPTER 1. INTRODUCTION
Figure 1.1: Adaptive hypertext process. The horizontal direction represents transformation. Vertical arrows represent data flow. See text.
1.1.1 Rationale for hypertextualization
It is a fair assumption that large online documents have much to gain from being structured as hypertext. The amount of hypertext present in the Word “Wild” Web, its sheer weight, supports this assumption. (However its relative weight is small, 44% only, against 56% of PDF documents. More on this later.)
The collective mind of Web authors, manifested in the authored Web links, has also been shown a useful resource for exploring the Web space. The current epitome of this approach is the famous PageRank algorithm, which arranges Google results (Page et al. 1999).
Since the Web can be seen as a huge hypertext document, we may transpose these facts to the general concept of large document, and hypothesise that any large online document will gain by taking the form of hypertext and being adapted to—and by—its users and authors, or usors1.
More generally, hypertextualization can be seen as an effect of the desire to augment intelectual power via technology (Bush 1945). This movement is pervasive to all mankind, and clearly justifiable. This rationale of intelect augmentation is adressed lengthly in chapter 2.
1After having arrived at the concept and term of usor, we have found the term wreader being used for a similar concept, as reported by Weel 2006. We have stayed with usor: its morphology is more regular than that of wreader, and its sound is more distinguishable from user than the sound of wreader from reader. 1.1. CONTEXT 3
1.1.2 Limitations of hypertext
The main problem with hypertext is an increased utilization effort relative to normal text. This accrued effort is two-fold: there is an increased effort of authoring, and there is an increased effort of navigation.
1.1.2.1 Hypertext is costly to build
There is an increased authoring effort of creating the pages and the links—accrued relatively to the work of producing normal text (Bra et al. 1999). The hypertext author has to organize or divide the content into pages, and create links between the pages. The number of possible page partitions and manner of linking is virtually infinite, which puts upon the author the burden of making an innordinate number of decisions that are difficult, violent (Belo 1991)—and therefore very costly.
One might think that normal text has a specific cost of its own too: because of the sequential nature of normal text, special devices of anaphor, narrative, prosody, punctuation, textual cohesion—rhetorical devices in whole—must come into play to make for a good sequential reading. But it has been shown that such factors play their part in hypertext too (Mancini 2004). For one, hypertext is still text. Even if the pages are short, they are still large enough for most rhetorical devices to operate in—and be required to.
1.1.2.2 Hypertext is costly to navigate
There is an increased reading effort of navigating an hypertext—additional to the effort of reading normal text. Essentially, the user is put in charge of constructing their own unique pathway through a variety of options (Lawless et al. 2003, Raskin 2000). This activity incurs the extra cognitive cost of making choices, from a sometimes unwiedy number of options, of links on a page.
A related, often purported limitation of hypertext is the lost in hyperspace syndrome. We discuss this on chapter 2, particularly on section 2.3.8.6. 4 CHAPTER 1. INTRODUCTION 1.2 Solutions
A number of automatic adaptive techniques have been tried or proposed as a means to alleviate the costs of both authoring and reading hypertext (Bra et al. 1999). Such techniques are analysed at length on chapter 2, particularly on section 2.5. Most if not all such proposals revolve around the concept of relinking—creating or changing links automatically, with the objective of helping users in their navigation. Those proposals are academic, stemming mostly from the Intelligent Tutoring Systems area, and are still confined to the laboratory. So is our own proposal.
In the real world, there is Google and Amazon. These resources can be seen as large hypertexts too, because they have pages and links. Google results are automatically created links from a search expression to related items on the Web. Amazon recom- mendations are automatically created links to related items on the store. Amazon and similar sites are adaptive hypertexts.
The principal part of Google algorithm is the PageRank algorithm that order the results (Page et al. 1999). PageRank relies entirely on the authoring effort of others, namely of all authors of all the pages on the Web, because it draws results from the analysis of the links between the pages, and such links and pages were created at a cost to the authors, as we have discussed. So PageRank is not really a solution to the hypertextualization cost problem, it just moves the problem out of its sight. Without authored links, there would be no Google.
The recommendations at Amazon, or at other similar online shops, are computed based on the recorded navigation data, using a technique called collaborative filtering (Goldberg et al. 1992). The navigation data includes conversion information: wheather the user has bought the item or not. See figure 1.2 for an example; note the recomenda- tions at the bottom of the large central zone; note the recommendations clearly marked as derived from the choices made by other users: “Customers who were interested in Startone CG 851 4/4 bought the following products”, and the items recommended by some other means, simply titled “related products”.
The recorded navigation data represent the actions made by users. Using this data is indeed likely to provide a means of alleviating the hypertextualization effort. Actually our own proposal in this thesis can be seen as a variant of this approach, as we shall see. That is, our system can partially be seen as a recommendation system. Among other similarities of our proposal with online shop recommendations, there is the obvious alignment between the concepts of conversion (online shop) and successful session 1.2. SOLUTIONS 5
Figure 1.2: Example of online shop recommendations. 6 CHAPTER 1. INTRODUCTION
(our system).
Recommendation systems have yielded good results in online shopping—we have not seen the method applied to documents. The laboratorial, academic adaptive tech- niques (cf. chapter 2, particularly section 2.5), have indeed been aplied to documents, with results that are also, so far, generally positive. However, it is hard to draw convincing comparisons because there is no common methodology, no common mea- sures of hypertext usability used across the different proposals. We wanted to improve on this situation. In particular, we wanted to explore the following observation: all existing systems, laboratorial or real, treat the pages as given, integral wholes.
1.2.1 Our solution
We questioned the presumed atomicity of pages, on the assumption that it might be an obstacle to better adaptation. Our rationale is based greatly on the observation that when we consult a document for reference, we end up selecting a small part of the document to satisfy our precise information need. So perhaps we could split, or shatter, a document into such small parts, or noogramicles, in order to adapt pages made of them towards the (computed) needs of the reader.
So, whereas in all existing systems (laboratorial or real) the pages are given, integral wholes, in our system the pages, or nodes, are an assemblage of small document parts, which we call noogramicles. Some noogramicles are shown contracted and actually constitute links to their fully expanded expression, or view. So a page in our system is actually a set of items of two types: document parts, links to other document parts.
Each page is created anew upon each request. So we have called this approach renoding, as a paraphrase of relinking. The constituend noogramicles of each page are selected based on usage, or navigation data, as in recommendation systems. This selection is done using a spreading activation algorithm, described later.
1.2.2 Hypotheses of this work
Naturally we expect our approach to improve upon a non-adaptive document, and so our main hypothesis is expressed thus:
Shattered Document Hypothesis (main hypothesis). The navigation effort of 1.3. MAIN CONTRIBUTIONS 7
a user in a shattered adaptive hypertext document is reduced with respect to the original hypertext document.
We also hope our spreading activation algorithm to fare better than a standard adaptive technique like Markov chains.
Spreading Activation Hypothesis (supplemental hypothesis). Spreading acti- vation is a better adaptation model for shattered documents than first order Markov chains.
Finally, a part of our method consists in treating all relations, or links, equally. Namely, we propose that, for adaptation purposes, the original document relations between noogramicles, namely the relations of sequence, hierarchy, and cross-reference, equate each other and, more importantly, they equate the relation established by the user when travelling between noogramicles. This amounts to viewing the author simply as the first user, or usor of the document, i.e. we unify the concepts of author and reader.
Usor Hypothesis (supplemental hypothesis). Varying the weights of links ac- cording to their type does not have an influence on results.
1.3 Main contributions
The main contributions of this thesis are concentrated in the new concept of renoding. Renoding is a paraphrase of relinking. Relinking is the adaptive hypertext state-of- the-art item that consists of changing, creating, or deleting hyperlinks (Bollen 2001). A foundational observation of our research is that there is no a priori reason why links should be thus changed and pages not. In particular, we observe that the state-of-the- art tacitly subsumes pages as given, integral items.
Renoding is necessarily achieved by means of shattering the document into small pieces, or noogramicles, for adaptation. It is very important to keep in mind that renoding and shattering are done for the purpose of adaptation only. That is, renoding is strictly an adaptation technique.
Given the extremely networked nature of the data structures involved, we have decided to use spreading activation as a technique to explore these data graphs. The use of 8 CHAPTER 1. INTRODUCTION spreading activation for data analysis is still an evolving research issue, therefore our findings in this area are also a contribution of this thesis.
As an experimental test bed, we have built a user simulator. Although potentially imperfect and necessarily simplified with respect to reality, the simulator enables extensive testing with different variants and can be regarded as a contribution per se. It also provides a well defined characterization of the assumptions under which the proposed approaches can operate. The evaluation of selected techniques, as well as the callibration of the simulator, on real user experiments has been left for future work. Using the proposed simulator, we have compared our approach against non- adaptive variants, and also against adaptive variants using a standard recommendation algorithm based on Markov Chains. Our approach has indeed fared the best in our main measure of session cost, ultimately verifying all our hypotheses.
1.4 Structure of the text
This thesis is organized into the following main parts.
Chapter 1. Introduction. Main theme and hypotheses briefly presented and mo- tivated.
Chapter 2. Hypertext. Discussion of hypertext in general, its history, including background and related work, and further motivation for the approaches in this thesis.
Chapter 3. A new model for adaptive hypertext. Description of the shattered document approach, including algorithms.
Chapter 4. Experimental methodology. Definition of the experimental setup, in- cluding simulation algorithms.
Chapter 5. Results Presentation of the results of the experiments.
Chapter 6. Conclusions Conclusions and future work.
References. Bibliographical descriptions of cited or studied works, websites, software works.
Appendix A. Electronic archive. DVD-ROM or card with the software created in this thesis, or indication of where on the web to obtain it. 1.4. STRUCTURE OF THE TEXT 9
Appendix B. Program listings. Listings of the principal source code units cre- ated.
Appendix C. Supplemental writings. A few writings that did not fit on the main body of the thesis, including a description of the first, small scale, exploratory experiments and simulator.
Appendix D. Detailed results of experiments. Complete result data of the large scale experiments reported, in the form of tables and charts.
Physical copies of this thesis are normally divided in volumes (the page numbers and quantities are approximate):
Volume Page numbers Qt. of pages I. Main text and appendix A i–xxv, 1–125 150 II. Appendices B–D 125–375 250 total = 400
1.4.1 Notation
Forms like Rome 2003 are bibliographical references. Forms like Turbo C, undated, are references to entities of a different type than documents, but still referable, like software works or websites. All references are collected and described on the respective section.
Single word, or expression, quotations take this form. Short, in-text quotations “take this form”. Single word or expression quotations take the same form as emphasis, not to overload the text with too many distinct notations. All short or long quotations are accompanied by the respective bibliographical reference.
“A long quotation takes the form of this paragraph. A long quotation takes the form of this paragraph. A long quotation takes the form of this paragraph.” 10 CHAPTER 1. INTRODUCTION Chapter 2
Hypertext
Our writing equipment takes part in the forming of our thoughts. F. Nietzsche
In this chapter we define hypertext, and discuss the aporias, or theoretical difficul- ties, of text, hypertext, and adaptive hypertext. Occasionally, we lay down certain assumptions, which we have found necessary in order to proceed with our work. We also chart the history of hypertext in this chapter.
2.1 Definition
An entirely consensual definition of hypertext does not exist. Below (section 2.3) we shall discuss this issue in detail and arrive at a working definition, which we preview here for convenience:
Hypertext is an interface to interconnected items of textual or pictorial information, which interface lets the user follow any connection, and also records the connections followed and lets the user relive them at will, typically by means of a “back button”.
11 12 CHAPTER 2. HYPERTEXT 2.2 Terminology
We use the following non-trivial or specific terms in this thesis. Terms marked with an asterisk have a specific meaning in this thesis i.e. we give the meaning specifically used in this thesis and particulary in the shattered document design.
Atom* See noogramicle.
Codex The current form of the physical book: a set of rectangular leaves of paper bound together along one side.
Context The wider or historical circunstances. Other authors use the term user context instead of user input.
Cybernetics The science that studies the abstract principles of organization in com- plex systems. (Heylighen & Joslyn 2001)
Distal content The page at the other end of the link. (Olson & Chi 2003)
Fan-in The quantity of afferent (incoming) immediate connections from a node in a tree or graph data structure. A.k.a. in-degree.
Fanout The quantity of efferent (outgoing) immediate connections from a node in a tree or graph data structure. A.k.a. out-degree.
Information need The information need of a user of an information repository. Syn- onymous with the term and concept of reference question from library science. The information need triggers, and sustains, a search session.
Noogramicle* The smallest constituent of meaning on a document. A caption, a figure, a formula, a footnote, a heading, a listing, a paragraph, a sentence, a table, a title, etc. Synonyns used at some time in our work include atom, knowledge atom, paragraph, extended paragraph; these forms may surface for historical reasons on software items; please adjust.
Oracle* The noogramicle, or noogramicles, that constitute the answer to the refer- ence question. We chose this term over the more common target or goal, to facilitate distinction from other meanings of the latter, notably the end point of a link, or the head of a directed connection in a data graph, which are concepts operating in this thesis also. 2.3. FORMS OF HYPERTEXT 13
Page* An assemblage of noogramicles. More precisely, of views of noogramicles.
Reference question See information need.
Session A finite episode of utilization of an hypertext system, by a user with a reference question in mind. The session represents the user searching for the answer to the reference question. The session terminates upon finding the answer (successful session) or giving up (unsuccessful session).
Session cost* Cognitive effort associated with the navigation aspect of a session. A rough measure is Session size. Finer measures takes into account other factors like page size, need for scrolling, etc.
Session size Session size = number of pages visited = number of clicks + 1.
View* The view of a noogramicle is the readable representation of the noogramicle. It is either expanded or contracted. The expanded or full view represents the noogramicle entirely. The contracted view, or label, serves as the anchor of a link. For indicative noogramicles like headings and titles normally the full view is used as the label i.e. there is no contraction proper. For propositional noogramicles like paragraphs normally the first few words are used to form the label; for items with a caption like tables and figures normally the caption is used for this purpose.
2.3 Forms of hypertext
The first aporia of hypertext concerns the notion itself. Very different kinds of hypertext have been proposed, some even antagonistic between them. For example, Nelson’s conception is consistently put in dire opposition to Berners-Lee’s, or HTML. This state of affairs makes the use, the denotation, of the word hypertext inaccurate, to say the least.
In the 1960’s Ted Nelson coined the very word hypertext, with the general meaning of
“a body of written or pictorial material interconnected in such complex way that it could not conveniently be presented or represented on paper.” (Nelson 1965) 14 CHAPTER 2. HYPERTEXT
The word has thriven, as we know. In the course of these four or five decades, it has become an essential part of both popular and scientific culture. Today, ar- guably the most common species of hypertext is that inhabiting the World Wild Web. N¨urnberg et al. 1996 integrate well from several notable sources (Engelbart & English 1968, Halasz 1988, Marshall et al. 1994), for a definition of such kind of hypertext:
“Information realized in the interface by connected “pages” of text and graphics traversed through a “point-and-click” navigation mechanism.” (N¨urnberg et al. 1996)1
Surveying an even wider range of sources, Mancini 2004 arrives at a tripartite classifi- cation, of which we will attempt a synthetic formulation2
• page-based hypertext: the common, web-like hypertext
• semantic hypertext, characterized by having typed connections
• spatial hypertext: semantic hypertext visualized graphically.
Mancini 2004 stands as one of the best works we have studied on the subject of hyper- text, and one of the very few correctly acknowledging the existence of various kinds of hypertext. Nevertheless, we observe that, strangely, she colocates Ted Nelson’s hypertext alongside web hypertext, in the page-based category. This might be unfair towards Nelson’s original proposal, called Xanadu:
“I believe it was in 1968 that I presented the full 2-way Xanadu3 design to a university group, and they dismissed it as “raving”; whereupon I dumbed it down to 1-way links and only one visible window. When they asked how the user would navigate, I suggested a backtrackable stack of recently visited addresses. I believe that this dumbdown, through the various pathways of projects imitating one another, became today’s general design, and I am truly sorry for my role in it.” (Nelson 1999)
1Not surprisingly, this definition approximates closely that of dicionaries, cf. figure 2.5. 2This formulation is our own very subjective reinterpretation of Mancini 2004, because we were not able to locate, on this part of Mancini’s text, a clear statement of a sharp, essential distinction between the proposed classes. 3Xanadu is the design already presented on Nelson 1965. 2.3. FORMS OF HYPERTEXT 15
Figure 2.1: Screenshot of a Xanadu prototype on Nelson 2007a.
Xanadu presents multiple windows at the same time, and visible connections between them. See also figures 2.1 and 2.2. Clearly, spatial hypertext in Mancini’s classifica- tion, not page-based.
Incidentally, note how Nelson, according to the account above, invented the Back Button right there. Let us take notice that the associated requirement—navigational memory—has been a staple of hypertext since its inception on Bush 1945. And many other essential sources, notably Nelson 1963, indicate clearly that users are supposed to be able to backtrack to any past point in their travels.
And indeed the Back Button has proven itself a formidable weapon against the lost in hyperspace syndrom, discussed later (section 2.3.8.6. It accounts for up to 42% of user actions with web browsers (Cockburn et al. 2002). If there ever was a silver bullet of interfaces, this was it. So we must make the Back Button—or some other form of access to the navigation history of the user—a strict requirement of our working definition of hypertext. 16 CHAPTER 2. HYPERTEXT
Figure 2.2: Illustration of an advanced form of hypertext on Nelson 2007b. The name is now transliterature, but the concept is essentially the same as Xanadu and Nelson 1965. 2.3. FORMS OF HYPERTEXT 17
Figure 2.3: SWI-Prolog help panel. In the help text, the terms set in bold face and coloured green are clickable connections. For example, clicking on consult/1 (under the arrow on the picture) replaces the page with the one on figure 2.4
Above, and on other sources, the Web has been given as an example of page-based hypertext. It must be noted that page-based hypertext is also common in offline programs, for example in help systems, multimedia encyclopedias, etc. Recall pro- gramming in the early 1980’s—before the Web—with the Turbo C compiler and environment. With the cursor upon a keyword or library entity in our program text, a hit of the F1 key would bring up instant documentation about it, and these help panels contained similar linkage to others. For a current, after the Web, example of offline programming help let us refer to the help system of SWI-Prolog, cf. figures 2.3 and 2.4.
We observe that the SWI-Prolog help system, like many such systems, unfortunately, lacks the Back Button, or any other navigational memory.
For another example of offline hypertext consider the Dictionary application, depicted in figure 2.5. In Dictionary, every word is clickable, so the designers chose not to use any special typography—wisely, because that would only produce unecessary visual 18 CHAPTER 2. HYPERTEXT
Figure 2.4: SWI-Prolog help panel for consult/1
clutter. Note also that, unlike SWI-Prolog help, Dictionary features Back and Forward buttons, and is therefore an hypertext system by all accounts.
In sum, there are various kinds of hypertext. We can define the invariants as follow:
• hypertext is an interface to interconnected items
• the items are of information, textual or pictorial
• the interface lets the user follow any connection
• the interface records the connections followed, and lets the user relive them at will; in particular, the interface provides a back button
And this will be our working definition of hypertext. 2.3. FORMS OF HYPERTEXT 19
Figure 2.5: Dictionary.
2.3.1 The words used
Nelson 1965 introduces other hyper concepts: hypermedia, hyperfilm. But does not offer a rigourous ontology. The words hypermedia, hypertext are used interchangeably in the research literature i.e. they mean the same. The word hypermedia seems to be preferential in the learning systems4 area, cf. Chen & Ford 1998; it is also often used in commercial marketing, to inform potential costumers of an hypermedia product that it contains multimedia, not just plain text. The word hypertext is used everywhere else—including here.
Let me suggest that the text in hypertext, as used by Nelson and others, already means more than just plain text, more than just letters. It means document, discourse, sign. It includes pictures, graphics, formulae, even sound and films. In a word: content. This, shall we say, semiotic meaning of text is common in the humanities (an habitual Nelson dweling); recall the idiom subtext of a film, etc. Nelson’s own definition supports this approach: hypertext is ... “textual or pictorial”.5
4Comprising ITS: intelligent tutoring systems. 5Curiously enough, this extension of the meaning of text mirrors the way Otlet, as we shall see, has extended the meaning of document to include all information objects, further than books, e.g. artifacts, archeological findings, models, didactical toys, works of art, etc. Or as, in semiotics, 20 CHAPTER 2. HYPERTEXT
2.3.1.1 Literary vs. technical hypertext
Consider the two distinct classes of text, or discourse:
• literary, poetic, artistic discourse: works of fiction, poetry, etc.
• technical, scientific, utilitarian discourse:6 manuals, thesis, articles, etc.
We note that Nelson often uses the term literature for any kind of text. Here we use the terms literary and technical for the two classes purportedly in opposition.
This aporia relates closely to rhetorical categories. Literary hypertext studies seem to proceed in terms of rhetorical categories, cf. Mancini 2004.
Rosenberg 1999 provides an impressive review of the potential structural complexities of hypertext, particularly of the literary kind.
2.3.2 Items vs. connections
Recall our working definition of hypertext (slightly abridged):
• hypertext is an interface to interconnected items
• the items are of information, textual or pictorial
• the interface lets the user follow any connection
• the interface records the connections followed, and lets the user relive them
Items, connections: cleary the two basic building materials of hypertext. These two materials are—we submit—at a tension, which constitutes a foundational aporia of hypertext.
Theoretically, any large enough item may be transformed—shattered—into, two items plus a connection between them. Conversely, any two items with the proper connection natural language is often taken as the prototype of language, and, accordingly, linguistics as the mother science of semiotics. We also note that the term media in hypermedia denotes something often also called content—which concept is, in a way, the very opposite of media. Probably semiots are more keen on the sensibleness of their terminology than marketeers—as it should. 6Collectivelly known in the learned lusophony as discurso gnosiol´ogico, cf. Belo 1991. 2.3. FORMS OF HYPERTEXT 21 between them may coalesce into a single item. Indeed, this observation is at the basis of the Shattered Documents approach of this thesis.
The Nelsonian concept of transclusion also blurrs the distinction between item and connection, cf. Nelson 2007b.
This aporia is perhaps better understood within an historical perspective. The current times are the age of connections. Little or null attention is given to the nature or construction of items. This focus on connections has been a constant since Bush 1945:
“The process of tying two items together is the important thing.” (Bush 1945)
Incidentally, note that this focus corresponds to the hyper part of the many hyper... words, which would in fact explain their proliferation.
Bush 1945 describes the memex, an hypothetical machine for memory extension, based on a network of microfilms and on an user interface equiped with means to explore this network. Indeed, the memex crucial attribute is, in the author’s own words,
“associative indexing, a provision whereby any item may be caused at will to select immediately and automatically another. This is the essential feature of the memex. The process of tying two items together is the important thing.” (Bush 1945)
Bush 1945 is consistently touted in the current literature (e.g. Mancini 2004) as the seed of hypertext thinking. But we have found descriptions of similar conceptions predating Bush 1945.
Earlier in the 20th century, there was a documentalist movement concerned with using the information and communication technology of the day—radio, x-rays, cinema, microfilm—to improve the global access to global knowledge. Otlet 1934, Otlet 1935 envision the convergence of such technology into an Office of Documentation, or Mundaneum, to form “a mechanical, collective brain... an exodermic appendage to the brain... a substratum of memory... an external mechanism and instrument of the mind” (Otlet 1934, Otlet 1935, apud Rayward 1999).
Note the obstinated use of brain metaphors. Clearly this line of thinking is already inscribed in a tradition of intelect augmentation7 by artificial means.
7Let us use the term introduced by Engelbart 1963, advantageously more general than the memory extension of Bush 1945, assuming that this is what the word memex was an abbreviation of. 22 CHAPTER 2. HYPERTEXT
“Intellectual power, like physical power, can be amplified. Let no one say that it cannot be done, for the gene-patterns do it every time they form a brain that grows up to be something better than the gene-pattern could have specified in detail. What is new is that we can now do it synthetically, consciously, deliberately.” (Ashbi 1956)
Returning to Otlet and the 1930’s: he proposes the organization and transmission of knowledge in a global network. The Mundaneum, conceived as the centre of the network, would be replicated at several levels. There would be a Mundaneum in every country, in every city, and, finally, every person could have their own technologically sophisticated office, called Studium Mundaneum, in which they could access the repertoires. Which would clearly impact the professional, personal, familiar and social life of every individual (Santos 2006, p. 97, quoting Rayward 2003, p.7). Clearly, Otlet envisioned no less than the Internet of today.
Also, the documentalists sustained the Monographic Principle as a basis for the Office of Documentation.
2.3.3 The Monographic Principle
The Monographic Principle takes form in the recording of single pieces of information onto standardized cards, with larger chunks of information recorded on separate sheets. These cards could be managed, copied, combined, in order to search, form, present information. (Otlet 1918, apud Rayward 1994). The approximation to the nodes of hypertext, specially in the Shattered Documents approach, is compelling. As Santos 2006 puts it: “Otlet reformats the document by fragmenting it, and reorganizes content, generating new informational wholes. The informational objects created by the Monographic Principle approximate today’s databases and hypertextual objets.” (p. 90).
Nelson would coin the word hypertext. Otlet created none other than the term: documentation!(Otlet 1918)
The monographic units, the fragments, were reorganized on the basis of classification systems e.g. Otlet’s own Universal Decimal Classification. The units were organized in sequences assigned to a certain classification. The UDC and other classificatory systems allow for multiple classification of the same item, so an item could participate in multiple sequences. So, structurally, these sequences are like the trails of the memex 2.3. FORMS OF HYPERTEXT 23
Bush 1945. Conceptually, however, they are a world apart.
The difference is between positivism (documentalists) and deconstructivism (memex). The documentalists assumed the possibility of a man-made, predefined classification system capable of organizing all knowledge, whereas memex hypertext is created, organized at will by the user, with or without resorting to any predefined classifica- tory system. Indeed Bush 1945 discusses the inadequacy of classification systems for hypertext. So, along with the general passing of positivism, eventually classification systems were abandoned as a means of connecting the items, and replaced with free linking by the usors.
Unfortunately, the baby was thrown with the bath water. Since Bush 1945 the focus has shifted towards the connectionist part of hypertext, and little or null thought has been given to the items connected. More often than not, the items are assumed as given, integral parts of the system: the pages. We have already noted in several parts on this thesis how this assumption lacks theoretic foundation, and how the respective design creates interface faults, for example long pages requiring scrolling, which extra cognitive effort distracts the user from their main quest. Pages must be shatterable. So, our approach of Shattered Documents may be seen as a return to the good part of Otlet’s documentalist theory.
In sum, Otlet created the monographic principle, but no hyperlinking. Bush envisioned hyperlinking, but sans monographic principle. And to this day the monographic principle as been dormant to say the least. Whether on a path of Hegelian dialetics or not, on this thesis we do try to make a small contribution to finally unite the two things—free connections, free items—in a long due synthesis.
2.3.3.1 More words: microtext, macrotext
We have seen how the notion of hypertext is often blurred with the notion of global access to global knowledge. In the current Word Wild Web the two aspects seem inextricable.
A certain branch of (modern) hypertext literature uses the terms microtext, macrotext, to refer to the local vs. global range of hypertext (Keep et al. 1995). The Web is a macrotext. The memex is a microtext.
A single Office of Documentation is local, but Otlet’s envisioned a network of such Offices spanning the World, and each replicating the collection of World’s knowledge. 24 CHAPTER 2. HYPERTEXT
Clearly a world of Internet caf´es.Macrotext.
Macrotext was the focus of another hypertext visionary—seldom recognized as such, albeit a very famous author: H. G. Wells. Aparently since 1902 (apud Rayward 1999), Wells has exposed a vision of a world brain, and a global encyclopedia, created by means of technology—notably the same technology and even the same methods selected by the documentalists (Wells 1938). Aparently Wells and Otlet met in the late 1930’s ou early 1940’s, cf. Rayward 1999.
(At this point, the science fiction connoisseur will probably evoke the Encyclopaedia Galactica or the Hitchhicker’s Guide to The Galaxy. We note in passing that neither of these devices seems to be a macrotext or even a microtext.)
Aparently H. G. Wells has also foretold the screen, or monitor, as we know it today, on his science fiction novels (apud active literature cited above).
2.3.4 Summary of hypertext history
For a summary of hypertext history see table 2.1.
Table 2.1: Summary of hypertext history Early 1900’s Otlet, Wells, envision the cyberfuture Otlet coins the term documentation 1945 Bush designs the Memex 1960 Nelson designs Xanadu 1965 Nelson coins the term hypertext Engelbart creates HyperCards (and the mouse) 1980 Berners-Lee creates HTML, a limited form of hypertext 1990 The Web develops within this initial philosophy 1995 The Web grows enourmously—lost in hyperspace 2000 Web 2.0, social networks—lost in MySpace
2.3.5 The document dogma
Circa 56% of the text on the Web is in the form of traditional documents stored in PDF format. 2.3. FORMS OF HYPERTEXT 25
This reflects a mentality, which we may call the document dogma, that the traditional document is the preferred, or somehow best, format to represent information. Either from the reader or the author point of view—the result is the same.
But it has been shown that hypertext—at least a moderate form thereof—is actually better for online reading than the imitation of paper (cf. Nielsen 1997).
However, traditional documents are hard to decompose into the independent units required for full-fledged hypertext, because traditional documents contain elements of—cross-unit—sequentiality or structure that give meaning to each unit involved.
Also, in the literature—and in designed or constructed systems—the nodes are always given. The various theories, systems, designs, focus on the links, on different ways of connecting the nodes. They never study, model, much less challenge, the internal construction of nodes. This assumption is probably in line with the document dogma mentality.
2.3.6 Information search—the impossible that is done
A search for information is a theoretical conundrum, because it aims at finding what is unknown; and therefore—theoretically—the search cannot be expressed, stated, in the terms of the sought solution. But as we all very well know, search for information happens all the time. This is so because the need for information is real, and must be solved, even if imperfectly. (The difference between theory and practice is that, in theory, there is no difference, but in practice, there is.) Some encompassing works of library science, documentation, and information retrieval, e.g. Foskett 1996, correctly recognize this effort of finding the unkown as the fundamental problem of information retrieval.
This impossibly perfect concept of information search entails the equally impossibly perfect, fixed concept of information need: “a constantly changing, inaccessible phe- nomena present only in the mind of the searching agent; a combination of ideas such as what the target information might look like, where it might be found, or how one might go about tracking it down—with the words look, where, and how used in a most general sense.” (Campbell 2000)
A single query of (classical) information retrieval subsumes a fixed information need. The real, dynamic, changing information need is often realised as a series of queries, with each subsequent one said to be a refinement of the previous. This dynamics is 26 CHAPTER 2. HYPERTEXT also the essence of relevance feedback techniques.
Hypertextual navigation (as well as the ostensive search model of Campbell 2000, which is essentially hypertext), as a form of information search, represents a slight departure from the classical query model. With hypertext, the user is presented, at each time, with predefined paths to follows.
In practice, users of the Web, and of hypertext, alternate between querying and nav- igation, mostly according to the circumstances. The integration of these two distinct search methods in a single interface has been a kind of holy grail for some researchers e.g. Olson & Chi 2003. In this thesis we do not pursue this type of integration. We simply focus on navigation.
2.3.7 Aporias of adaptation
2.3.7.1 Dictatorship of the majority
Of the two main kinds of adaptation—personal, colective—only the latter construes aporias.
Adaptation towards the colective mind brings the problem of dictatorship of the majority.
We assume that the collective brain is nevertheless useful to the individual in a large enough class of situations. There is plenty of (soft) evidence of this fact—the epitome being the very successful ranking of Google results based on PageRank, an algorithm that integrates data from many usors (and therefore is a kind of collective brain).
2.3.7.2 Unwarrant self-reinforcement
The essential design of recommender systems causes the following problem.
In a recommender system, the user is presented with recommendations. It is likely that the user follows the recommendations—or else the recommender system would be useless. But following a recommendation will strengthen the presence of the recommended item itself in the recommendation set. This veritable self reinforcement of the identity of the recommended set, i.e. of its set of items, has the vicious effect of keeping other possibly interesting items out of the set—and hence outside the user’s view. 2.3. FORMS OF HYPERTEXT 27
So far the only patch to this problem is to bypass the recommendation model proper by including random items in the recommendation set. Formally this represents a degradation of the result, and an artificial injection of noise into the system—clearly not a satisfactory situation (and hence an aporia).
(The identification of this problem, as well as the suggested patch, are, to the best of our knowledge, original contributions of this thesis. If I remember correctly, problem and patch were firstly proposed by myself in a meeting of a past European research project (Mladeni´c & Lavraˆc2003). Albeit discussion has ensued, it has remained un- published. However the problem is so evident, that we cannot but wonder how it has been treated by the myriad recommender systems out there in the commercial Web e.g. amazon.com.)
2.3.8 Minor issues
The aporias in this section seem only distantly related, if at all, to our main hypotheses. But they are too related to hypertext in general, or too culturally widespread, or just too interesting, not to be addressed in this thesis.
2.3.8.1 Anchoring aporias
On commonplace hypertext, links are anchored in the text it self, identified by a different typography. This design poses a number of problems.
When the term under anchoring appears multiple times on the same page, as often is the case, the design decision is not clear of which occurrence of the term to select as anchor. The first? All? Some subset? Which one? For example, SWI-Prolog designers chose to select all occurrences, cf. figure 2.6.
An alternate design, which dissolves this aporia, is to represent the links as separate items from the text, perhaps as buttons situated to the right margin of the page, and ranked by order of appearance on the text, as illustrated on figure 2.7.
As another, independent, example of this alternate design for help text, consider the interface depicted on figure 2.8. Note how the help items are strictly contained in the right panel—and not as hyperlinks anchored on the main text (the remainder panels), which would create visual clutter, and incur in the anchoring problems discussed.
Another problem of in-text anchoring is when the anchor is a compound term (i.e. 28 CHAPTER 2. HYPERTEXT
Figure 2.6: SWI-Prolog help for catch/3. Multiple anchors design. (Anchors are set in a green bold face.)
Figure 2.7: Alternative to the design in figure 2.6 2.3. FORMS OF HYPERTEXT 29
Figure 2.8: The web interface of the network router Linksys WRT54G. 30 CHAPTER 2. HYPERTEXT a multiword term) which comprises one or more subterms which are also potential anchors. This occurs extremely often in dictionary or encyclopaedic hypertext e.g. the Wikipedia. In-text anchoring simply cannot solve this problem. Consider, for example, the Dictionary definition of HTML, depicted in the centre of figure 2.9. The definition contains the compound terms Hypertext Markup Language and World Wide Web. These two terms encompass the eight terms entouring the picture, all entries in the Dictionary. However, the in-text anchoring design has made some terms not directly reacheable. The lines connecting the first letter of a word in the definition text represent clickable connections (remember that in Dictionary every word is clickable); the connections are to the indicated dictionary entries. Note that the entries markup language8 and world are disconnected. Note how the designers chose a special behaviour for the word World, namely selecting it as an anchor for the compound World Wide Web—disconnecting world in the process. And, of course, unsignalled different linking behaviours introduce unpredictability in the interface.
Figure 2.9: How compound terms are (mis)treated in Dictionary. markup language
hypertext markup language
World Wide Web web
world wide
Yet another problem, albeit less severe, of in-text, typographical, anchoring is when the anchor is not text at all, but e.g. a picture or an icon. In this case a typographical difference is simply not possible. But this problem only affects the typographical approach to anchoring, and can be solved relatively easily in a number of manners.
8The term markup language is meaning number 3 the entry for markup, requiring scanning, and normally scrolling, on this entry in order to be found; a direct link would avert this cognitive effort. All other terms are main entries. 2.3. FORMS OF HYPERTEXT 31
2.3.8.2 Data deficit
Invariably the access data, or HTTP log, is not rich or acurate enough for the expectations of the adaptation model.
A typical subproblem is the derivation of a session i.e. the identification of the accesses (clicks) that constitute a session by each user. Normally this can only be acomplished by the 30 minute rule—an heuristic rule based on the assumption that if a user is quiet for 30 minutes then their last request terminated a session.9 This rule yields results of varying degree—depending on the context or information domain—but that can never be 100% right.
Another subproblem is the identification of users, for cross-session derivations, user model construction, and of course personalisation. Only registered users can be rightly identified—but user registration impinges somewhat upon the Null User Effort principle.
Another problem is that the Back button serves distinct, even opposite, purposes:10
(1) The user follows a link that appears to be the oracle11; after going there, the user realises that it is not; the user pushes the back button simply to retreat from this false oracle, back to a point known to be of greater interest
(2) Same context as above, but Back button pushed simply out of panic. The Back button as a Panic button. The user is lost. They do not press the button to return to a Emknown place. They simply want to get out of there.
(3) The user has followed a link that appears to be part of the oracle—and it is. Then the user uses the Back button simply to continue exploring the web space from a more confortable place, probably a hub. etc.
Note item pairs (1)–(3) and (2)–(3) are pairs of opposites.
9In an concrete experience this value has been fine-tuned to 28 minutes. 10This problem is distinct from the Back button aporia described in a dedicated section. This problem is about interpreting Back button hits for adaptive purposes. The aporia is about the user expectations towards the interface. 11On this thesis oracle means roughly what the user is looking for. For the specialized meaning of this and other terms on this thesis see Annex A “Glossary” and parts referenced therein. 32 CHAPTER 2. HYPERTEXT
Note that access logs typically do not even contain Back button information. They simply stack the accessed pages. For example, for the session in figure 2.10(1), the log would simply record the sequence A-B-C-B-D-E. This is so in part because HTTP12 was designed as a stateless protocol. So, typical access logs, produced by the web server (e.g. Apache), are too low level for an adaptive system relying on user action information. There is a semantic mismatch. An adaptive system should maintain its own user action log. Several designs are proposed on this thesis.
2.3.8.3 Aporias of renoding
The concept of renoding, introduced in this thesis, carries an aporia of its own: if the page as we knew it ceased to exist, what else becomes the nature of links between pages?
In this thesis, and specifically in the shattered documents approach, this aporia is dealt with by introducing the concept of central noogramicle. Each page has a central noogramicle, which acts as representative of the page for linking across pages. The actual links are between noogramicles. This can be reinterpreted as connection between pages when the noogramicles are central.
2.3.8.4 Landmarks
Already well into the course of our investigation we have realized that Raskin 2000 is probably right in his observation that landmarks are a major help in navigation. Users looking for a certain item A, which they have seem before next to a certain item B, might find it easier to relocate A via B, because, for example, B is in sight (and A is not).
Landmarks and adaptive hypertext are irreducible. Landmarks require a stable, fixed territory which to mark and be inscribed upon. Adaptive hypertext is, by definition, a changing territory.
12HTTP = HyperText Transport Protocol, the communication protocol for WWW pages, cf. W3C. 2.3. FORMS OF HYPERTEXT 33
2.3.8.5 The Back button aporia
The Back button concept contains an aporia of its own, namely an unsolvable tension between its expected behaviour being chronological or not, cf. figures 2.10 and 2.11.
Figure 2.10: The Back button aporia. In certain contexts the user expects a chronological behaviour.
(1) B A C (back)
D E here user wants to go back to C
(2) user presses Back three times, (3) but A is where he gets: expecting to travel thus: B B A C A C
D E D E
Figure 2.11: The Back button aporia. Why the Back button should not behave chronologically.
(1) B A C now user wants to get back to A
(2) user presses Back two times, (3) if Back button behaved chronologically, being delivered to where expected pernicious effect would take place B B A C A C 34 CHAPTER 2. HYPERTEXT
2.3.8.6 Lost in hyperspace
The phrase lost in hyperspace has been echoed in the literature, with two meanings:
(1) lost in the hypertext structure, because of the hypertextual nature
(2) lost in the enourmous sea of information of the Web and modern culture in general
We downplay this purported aporia.
We believe that problem (1) has been solved with the Back Button and akin devices e.g. the Site Map. Golovchinsky 2002 provides a discussion of the Back Button, including advanced variants. The Site Map is commonplace in the Web today. Other navigational aids include: the fish eye, prototyped on Campbell 2000; the ZOOM interface, reported on Raskin 2000; the Map View, commonplace in videogames; the page trail, proposed in the current thesis.
Problem (2), in the Web, is partially solved with search engines e.g. Google. A better metaphor would be: drown in hyperspace. Or, with the advent of the so called Web 2.0, and its social networks, problem (2) becomes more like lost in MySpace—which might happen well before one reaches one million “friends”: according to recent results from psycho-social studies, 150 is the approximate number of direct contacts that a single person is able to manage (Dunbar 1993).13
Problem (2) in general equates the well known information overload problem. We shall observe that, contrarily to a common conception, this problem is not specific of our times, say of TV and Web times, but it has emerged, and has been analysed, and fought, since as early as the begining of the 20th century at the least. It is the very problem addressed by Otlet and the documentalists, as already discussed (sections 2.3.2, 2.3.3, Otlet 1934, Otlet 1935, Rayward 1999).
2.3.8.7 Design by tekkies
The design of web hypertext is extremely technology driven. HTML, JavaScript, Flash, whatnot, set the boundaries of what is done.
13MySpace, FaceBook, LinkedIn, Ecademy, HiFive, Plaxo, OpenSocial, SoundClick, Ning, Indaba... Albeit the social networking phenomenon is here to stay, and it is clearly related to collective adaptiveness in potential, in this thesis we had to stay away from the issue, because it is too large an issue on its own, and clearly the dust has not settled on it yet. 2.3. FORMS OF HYPERTEXT 35
Technologues are not only in charge of designing the underlying technology—much too often they impinge on the area of web design itself, of the use of said technology. Most disastrous designs that plague the web probably stem from this fact.
“The purpose of hypertext was always to make up for the deficiencies of paper. Paper cannot easily show connections, has very limited space, and forces an inflexible rectangular arrangement. Hypertext, the generalization of writing, potentially offers many forms of interconnection and presenta- tion beyond what paper allows. But so far the mechanisms, not the users, have been put in control.” (Nelson & Smith 2007)
Nelson also affirmed that computer interfaces should be designed not by tecnologues— as the Web was—but by filmmakers. He points out a lesson from history that the first films—made by the technologues of the day—were very poor in design or art. Recalling the prototype of first film, La Sortie des usines Lumi`ere (1895), we cannot but agree.
See also: Nelson 1999 and references thereof; Nielsen 1997 and other articles by this columnist on usability; Lebedev 2007 for design insights (and insightful design).
2.3.8.8 The computer as hypertext
One could argue that any clickable item on the computer constitutes hypertext. Menu items, file icons, window controls... the WIMP14 as hypertext. The general pattern of the definitions seen so far certainly apply. The only provision necessary is that the nodes (pages) may be also actions, operations of the computer. Even the Back button requirement is met:
• it is right there on certain WIMP interfaces, notably file navigators like Mac OS’s Finder, Windows Explorer...
• in a limited number of contexts, the WIMP has it under other forms, namely the Escape key (to regress from a submenu or window), or the Undo operation (often bound to the Ctrl-Z key combination).
So there a bit of hypertext in the WIMP after all—which is good.
14WIMP = Windows, Icons, Menus, Poiting device (mouse): the interface quartet on most if not all computers today. 36 CHAPTER 2. HYPERTEXT
Furthermore, there are alternate arrangements to the WIMP, including operating systems based on hypertext (N¨urnberg et al. 1996), and whole computer designs biased towards—or strictly for—information recording and retrieval by the user (Raskin 2000).
More commonly, we shall observe that many of today’s interfaces are indeed realised as hypertext of the web kind, either because they rest upon a web service architecture, or simply for convenience, or because the interfaced object is already a network server or device, e.g. mail servers, network routers (figure 2.8).
2.3.8.9 The logical structure myth
“LaTeX is different from other typesetting systems in that you just have to tell it the logical and semantical structure of a text” (Oetiker et al. 2005, emphasis mine). The same has been said of HTML.
Such assertions are misleading to say the least. In any case they represent an enormous abuse of the concepts of logic and semantics, because Latex and HTML commands are completely document-oriented: chapter, section, heading, paragraph, itemize, unordered list, enumerate, ordered list, table, etc.—all are elements of form. As are the styles of WYSIWYG15 word processors like Microsoft Word.
A truly logical, semantical, abstract, rational, system must be oriented towards rhetor- ical categories or rely on text generation devices. Indeed, so far, only text generation research seems to have produced such logical text devices, e.g. Power et al. 2003. And the art is still confined to the laboratory.
2.4 Structure of documents
2.4.1 Traditional document structure
Traditional documents portray a sequential organization. Long or complex documents also feature organizational devices that are hierarchical, or cross referential, or both.
The sequential nature is either imposed, to a lesser extent, by the physical form of the codex, or, to a greater extent, by the physical form of scroll—which, as we have seen in chapter 2, is a millenia-old book form, resurrected, to our misfortune, in the
15WYSIWYG = What You See Is What You Get. 2.4. STRUCTURE OF DOCUMENTS 37
current computer age. The sequential nature of documents is also associated with the sequential nature of narrative discourse.
Hierarchical devices consist mainly in the hierarchical containment structure of chap- ters and sections. Finally, cross referential devices include: references in one place to another place e.g. “for ... see section X”, “see also section Y”; back of the book indexes e.g. alphabetical index; the Table of Contents.
As for the substance itself of documents, we observe that is consists of figures, formulae, listings, tables and, of course, free text, i.e. text that is situated outside any figure, formula, listing or table. Text is the usual substance of headings and paragraphs.
For the purpose of the present thesis, it is clear that text may be analysed into constituents like headings, paragraphs, sentences—but no further into its smaller, or physical, units like words and characters. In the same vein, figures, formulae, listings and tables are unlikely to be analysed into their constituents. Just to clarify: we do not analyse documents at the level of characters, lines or pixels.
Finally, we observe that the items that constitute a document fall into two categories:
Propositional, or narrative units: figures, formulae, footnotes, listings, paragraphs, sentences, tables. These are units that, all by themselves, form a proposition, i.e. they assert something, or tell a story. These are the units more likely to ultimately satisfy an information need; in Shattered Document terms, these units serve as oracles.
Indicative, or non-narrative units: captions, headings, titles. These elements only indicate, or name, other, normally propositional, parts of the document. They do not make an assertion, or proposition, by themselves. These units are more likely to help in navigation, to guide the search for the oracles, than to serve as oracles themselves.
2.4.2 Standard hypertextualization
The various components of a traditional document are amenable to hypertextualiza- tion in various degrees. Figure 2.12 depicts the standard hypertextual edition of the Ada Reference Manual, or ARM. The representation, albeit stylised, honors the actual data, in the numbering and size of the sections, and the links thereof. Each section is a node, or page—an integral HTML file. 38 CHAPTER 2. HYPERTEXT
The pages are connected by Next and Previous links. Four sequent nodes in this sequence are represented. The links are symbolized by the arrows in figure 2.12, and designed on the interface as buttons located at the top and at the bottom of each page, cf. figure 2.13 which captures the real rendition of the first page in the sequence.
This sequential linkage, with Previous and Next buttons, is ubiquitous on hypertex- tualized large documents. The hierarchical and cross-referential linkage, on the other hand, is found in varied forms.
A Table of Contents (or Site Map) may or may not be present. Naturally, such device should provide direct links to the listed sections of the document. Alternatively, or cummulatively, some or all of the pages or sections may contain a sub-Table-of- Contents, providing such direct access to the their subsections.
An Up or Top button (or both) also may or may not be present in the current page or section, to allow moving upwards in the hierarchy. The Top button may lead to the Table of Contents—probably the most sensible design—or to other high up pages like the document cover, the start of the first top level section, or the start of the current top level section (occasionaly a Top of Section button is included, dedicated to the latter case).
Our example hypertextualised document, the ARM, contains a Table of Contents, and every page includes a button to it (fig. 2.13). Also one click away are: an alphabetical index; a text search facility (only works online); a list of related documents (with links); the cover page (enigmatically called Legal Information). Each of these items is also, like sections, a single node in the hypertext structure.
The text of sections contains cross-references to other sections. These links are to the sections as a whole. Naturally each item of the Table of Contents also links to the respective section as a whole.
Although each page is an entire section, the ARM has the notion of paragraph, properly implemented in the hypertext as a page fragment i.e. an element within an HTML document. The paragraphs are numbered within each section, starting at 1. Whereas the distal content of each Table of Contens item, and of each intext cross-reference, is a whole section, or page, the distal content of each entry in the alphabetical index is a numbered paragraph. 2.4. STRUCTURE OF DOCUMENTS 39
Figure 2.12: Standard hypertextualization of the sequential structure. 1. 1.1 1.1.1 1.1.2
Figure 2.13: Real look of the first page in figure 2.12. 40 CHAPTER 2. HYPERTEXT 2.5 Learning systems
Adaptation, in an adaptive hypertext system, consists in automatically changing the presentation of information and the overall link structure towards the needs of the user (Bra et al. 1999). The benefits over a non adaptive system include:
Reduced authoring effort. “Adaptive Hypermedia Systems (AHS) make it possi- ble to deliver ”personalized” views or versions of a hypermedia document (or hyperdocument for short) without requiring any kind of programming by the author(s).” (Bra et al. 1999)
Improved usability. Hopefully, “information can be found faster or can be better comprehended when adaptive techniques are used; the use of adaptive hyperme- dia (presentation and/or navigation) is beneficial, for comfort or performance.” (Bra et al. 1999)
Brusilovsky & Mill´an2007 call adaptive effect to the effect that each user receives a different view. That term is misleading because adaptation can occur sans a different view per user—but towards the document as one, or towards a collective or the whole of users, as we shall see. Perkowitz & Etzioni 1999 acknowledge that fact and, consistently, call customization to the specific view per user effect. Another preferable word used in the literature at large is personalization.
The research on adaptive hypertext is scattered over a quite large number of areas including such ones as User Modelling,16 Intelligent User Interfaces,17 Adaptive User Interfaces,18 Intelligent Tutoring Systems,19 Human-Computer Interaction,20 Adaptive Hypermedia,21 Learning Systems,22 Hypermedia Operating Systems,23 Cybernetics.24
16Brusilovsky & Mill´an2007, Herder 2003, Perkowitz & Etzioni 1999, Sos- novsky & Brusilovsky 2005.
17Kov´acs & Ueno 2006.
18Paramythis et al. 2001, Perkowitz & Etzioni 2000. 19Czarkowski 2006 and references therein.
20Card et al. 1983, Cockburn et al. 2002, Heylighen 1997, Olson & Chi 2003, Raskin 2000.
21Alves et al. 2004, Bra et al. 1999.
22Ahn et al. 2005, Sosnovsky & Brusilovsky 2005, KnowledgeSea.
23N¨urnberg et al. 1996.
24Ashbi 1956, Bollen 2001, Bollen & Heylighen 1998, Heylighen 1997, Heylighen & Joslyn 2001, Principia Cybernetica Project. 2.5. LEARNING SYSTEMS 41
The latter two areas are probably the most island-like (each on its own). It is revealing that the bibliography of N¨urnberg et al. 1996, a prime representative of the Hypermedia Operating Systems area, is totally disjoint with the bibliographies of the other areas. Regarding the efforts from Cybernetics, e.g. Heylighen 1997, Bollen & Heylighen 1998, Heylighen & Joslyn 2001, Bollen 2001, they are mostly unbeknownst to the others as well—whereas the Cyberneticians themselves seem knowledgeable of those. In this thesis we do not situate ourselves strictly in any Capital Letter Research Area. We would gladly do so regarding Hypertext Science—if such one existed.
In fact, a truly general theory of adaptive hypertext, spanning all areas, does not exist yet—altough the field is fraught with conceptual frameworks—also called reference models, general models, architectures, etc. For example, the Dexter reference model is popular in the Adaptive Hypermedia and Learning Systems areas. Partially dependent on the Dexter reference model, a general theory of modular evaluation has been discussed (Paramythis et al. 2001, Herder 2003, Sosnovsky & Brusilovsky 2005).
2.5.1 Adaptive hypertext techniques
The opposition presentation/navigation discussed on Bra et al. 1999, and on references thereon, is also a popular ontological basis for organizing the world of adaptive hy- pertext models, or techniques. Most items and descriptions below are taken from Bra et al. 1999, but were reorganized, edited, corrected, or extended.
Presentation techniques include:
Conditional inclusion of fragments. Only fragments that are recommended for a user are displayed.
Stretchtext. For each information fragment there is a (short) visible place holder. The system determines which fragments should be stretched (i.e. shown) and which fragments should be shrunk (i.e. only the place holder is shown). This decision only determines the initial presentation of the fragment. The user may stretch or shrink fragments through mouse clicks.
Graying. Fragments that are not recommended for a user are grayed out (instead of excluded or shrunk).
Explanation variants. The same information can be presented in different ways. This can be done within a page or through guidance towards different page 42 CHAPTER 2. HYPERTEXT
variants. (In the latter case the method becomes adaptive navigation support rather than adaptive presentation).
Reordering. The order in which information items are presented may have to be al- tered. For instance, some users may prefer to see an example before a definition, while others prefer it the other way around. On a page, fragments of information are typically sorted from most to least relevant, a method which is best known from information retrieval systems.
Other techniques. Frame-based; natural language generation; dynamic hypertext, combining querying and linking in order to provide personalized link structures; cf. Bodner & Chignell 2000.
Adaptive navigation techniques include the following. Mind that the word link might actually denote—and often does—an anchor, or else entail the existence of an anchor.
Relinking. The target of the link is changed.
Direct guidance. A ”next” or ”continue” (link) button.
Sorting of links. A list of links is sorted and presented from most relevant to least relevant.
Link annotation. Link anchors are presented differently depending on the relevance of the destination.
Link disabling. Inappropriate links are disabled.
Link removal. Recall that this is actually anchor removal. So, naturally—and as Bra et al. 1999 themselves acknowledge—in running text this is a big dont, as it simply ruins the text.
Map adaptation. Some hypermedia systems provide a graphical presentation of (part of) the link structure. Such maps can also be subject to adaptation.
We must note that the technique of relinking is notoriously lacking on Bra et al. 1999. Notoriously but not surprisingly, given the seeming absence of communication between research areas. Relinking is a staple of the Cybernetics approach to adaptive hypertext (Bollen 2000), which results are unbeknownst, apparently, to the Learning Systems and Adaptive Hypermedia dwellings of Bra et al. 1999. 2.5. LEARNING SYSTEMS 43
Bra et al. 1999 also list the technique of link hiding, whereby “links leading to inappro- priate or non-relevant information are hidden” (Bra et al. 1999). We do not understand the difference of this technique with link disabling, or link removal. If link hiding affects only the linking function, then link hiding equates link disabling. If it somehow affects the anchor, then it equates link removal.
Note that this classification of techniques depends on a tripartite classification of hypertext elements into pages, fragments and links (Bra et al. 1999). It is acknowledged that pages are made up of fragments. But the monographic principle is never at work here. Each page is given with its fixed set of fragments. Only the way each fragment (or link) is presented is subject to adaptation.
It is also interesting to analyse the parenthetical remark on the definition of the Explanation variants technique, adapted here for convenience:
“In the case of guidance towards different page variants, the method be- comes adaptive navigation support rather than adaptive presentation.” (Bra et al. 1999)
This shows that the authors acknowledge a possible frailty of their presentation/- navigation dichotomy as a classificatory device. This kind of conceptual classifica- tion attempts—and failures—pervades the literature. Perkowitz & Etzioni 1999 provide another system of classification of adaptive hypertext techniques—also dual, but traversal to the presentantion/navigation system:
“Adaptive sites may use customization or transformation. Customization is modifying a Web site to suit an individual user; the page returned by the site depends on the user model. A site may also perform online customization—customizing the pages served to the user as she browses. In this case, the site takes into account not only the user model, but also the pages just visited in choosing the next page to show the user. In contrast, transformation involves altering the site to make navigation easier for a large set of users. For example, a university Web site may be reorganized to support one “view” for faculty members and a distinct view for students. In addition, certain transformations may seek to improve the site for all visitors.” (Perkowitz & Etzioni 1999)
In our own efforts to either understand or design adaptative hypertext systems, we found ourselves repeatedly thinking in terms of the following concepts; and doing 44 CHAPTER 2. HYPERTEXT so more than in terms of—or else in complementarity to—the classificatory theories above.
The personal—collective opposition, or continuum. Does the adaptive input draw on personal (the user) or collective data (other users)? Of all users? Or of a subset (group) thereof? Which group or groups?
Adaptive input/output. What is the adaptive input specifically, i.e. what data is used to drive the adaptation process? What is the adaptive output specifically, i.e. what is the result of the adaptation process?
Interface. How does the user interface look like? How does it integrate with the adaptive input, output, and process?
Our own approach of Knowledge Atoms, like the KnowledgeSea, looks into the collec- tive side of adaptivity, as input. But, as output, we propose a more extreme kind of adaptivity whereby each page itself is built—at the time of request—from the shattered fragments of the document.
2.5.2 Learning Systems highlights
In our own survey of the variegated landscape of adaptive hypertext research, and particularly of the cluster around Learning Systems, two items have made a singular impression upon us: the overlay model, for its ubiquitousness, clarity, definiteness, simplicity, understandability, usefulness; and the system Knowledge Sea, for being the only learning system that draws on the collective mind, and in a clear way.
Most adaptive hypertext systems, particularly those of the Learning Systems and Intelligent Tutoring Systems denominations, are based on an overlay scheme, or model, whereby the content is overlaid with a measure of, the knowledge of the content, by the user. In other words, each content element is tagged with a measure of its being known by the user. In sum, the content model is overlaid with the user model. (Sosnovsky & Brusilovsky 2005)
Proceeding from this overlay measure, each system then applies an adaptation tech- nique to the presentation of the content.
Knowledge Sea (Ahn et al. 2005, KnowledgeSea) was the first system to exploit the collective mind, for adaptation. The authors call social search to this collective aspect 2.5. LEARNING SYSTEMS 45 of the system (Ahn et al. 2005). The name Knowledge Sea comes, most probably, from the visual aspect of the interface, made of tiles of different shades of watery blue, thus evoking the image of a wavy sea seen from above (figures 2.14, 2.15). Each tile represents a piece of content. The darkness indicates its popularity. Other gauges are available. The person icon denotes the user. Again, depth of blue indicates degree of use. Let the authors’ own word provide the details:
“Knowledge Sea is a Web-based social navigation support system. It organizes Web- based open and closed corpus C language teaching materials including online tutorials and lecture slides. In order to implement this mixed corpus based social navigation, Knowledge Sea uses a knowledge map of the domain—a two-dimensional table con- sisting of 64 cells. It is built by a self-organized map (SOM) algorithm. Semantically related keywords and documents were assigned for each cell. Contents of neighboring cells are semantically related. Background colors of the cells indicate the popularity of the cells. As more users click and visit a cell, the background color of the cell gets darker. When they click a cell, they can see a list of documents and can choose a document from the list. The same logic to represent popularities by color lightness is applied to the representation of documents inside each cell. Each item of the list provides two types of information, traffic and annotations. “Human-figure” icons and colors provide users with popularity information and “thumbs-up” or “thermometer” icon and colors provide users with annotation information. If a document is popular among the group where a user belongs to, the background color of the icon gets darker. The foreground color of the icon gets lighter if the user clicked the document fewer times than other group members. Just like popularity, darker background color of anannotation icon indicates there are a lot of annotations for the document. “Sticky- note,” “thumbs-up,” and “question-mark” icons indicate “General,” “Praise,” and “Question” annotations respectively. A red “thermometer” icon indicates the overall annotations are positive, and a blue icon indicates the overall annotations are negative. Therefore, users can navigate socially by referring to other users’ behavior and opinions by looking up these icons and colors provided by Knowledge Sea.” (Ahn et al. 2005)
Empirical evaluation of Knowledge Sea, reported on Ahn et al. 2005, has yielded an interesting result: the most relevant items were not the most popular, but the most annotated. 46 CHAPTER 2. HYPERTEXT
Figure 2.14: The Knowledge Sea interface (KnowledgeSea).
2.6 Summary
This long chapter presented the definition of hypertext, the terminology used in this thesis, and a rather philosophical offering about the aporias of hypertext. Next we will concretize our ideas into concepts that are scientifically manageable, either formally or in the computer. 2.6. SUMMARY 47
Figure 2.15: Detail of The Knowledge Sea interface (KnowledgeSea). 48 CHAPTER 2. HYPERTEXT Chapter 3
A new model for adaptive hypertext
Research is what I’m doing when I don’t know what I’m doing. Wernher von Braun
We present a new model for adaptive hypertext based on the concept of renoding: the instantaneous construction of pages from a ranked selection of noogramicles (the small parts of documents). We define this model with formal rigour.
3.1 Motivation
3.1.1 Information, not documents
We put forth the shattered document approach, with its postulate of noogramicles making up the documents, as an atomic theory of information needs. Our main premise is that users look for information, not documents. Whether users look for information or documents has been a long standing debate in information science, particularly in library studies (Otlet 1918, Foskett 1996).
In pre-computing times the issue obviously referred to physical, printed documents. With these being the only means of recording written information, the argument could
49 50 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT be made that users look for documents only inasmuch as they look for the information contained in the documents. That is, users know about the physical constraints, and plan their search accordingly. In fact there is ample evidence that this is indeed the case. Most consultation acts result in the selection of a part of a document, or in a combination of parts of one or more documents. The latter activity has even gained professional status and a name: clipping.
The counter-argument would be based on a conception of documents that are perfectly suited for each information need. That is, a document is a knowledge organization device. A well constructed document represents the consummate answer to a specific knowledge need. So the clipping activity is only necessary for non ideal cases, i.e. when the sought knowledge has not been the subject of such a document construction yet. However, the absolute number of non ideal cases is very large. And, specifically, large enough to justify any effort—such as the present thesis—of making it easier for the user to find the right information contained in the imperfect documents.
With the advent of computing, one would expect information to be recorded in a more maleable way than the document. This has in fact happened; two structures have become common: databases, hypertext.
The undeniable need for databases clearly reveals the atomic nature of a vast class of information needs. Virtually all database content is compound of atomic pieces of information, and queries are based on this microstructure. (The need for hypertext is much harder to analyse—we tried our best on the dedicated chapter 2.)
In conclusion, one can safely say that a very large number of information needs can be satisfied with small, atomic, parts of documents. 1
3.1.2 Guidelines for adaptive hypertext
This section is an evolution of a part of Alves & Jorge 2004. It is a set of top level requirements and design guidelines for adaptive hypertext systems: No Documents, No Scroll, No Intrusion,2 Full Control, Clear Labels, Full Incrementality, Best Practices,
1The school of critical thinking (Paul & Elder 2012) acknowledges the existence of problems that are monological. Such problems correspond to atomic information needs. 2Also called Null User Effort occasionally on this thesis or on related software artifacts. Historical reasons. 3.1. MOTIVATION 51
3 Pervasive Adaptivity. This is mainly a result of years of personal experience both as a user and creator of web systems, but these results are also supported by respectable authors, duly cited.
No Documents. Users look for information, not documents. (We are referring here to traditional, long documents: books, articles, etc. Not innovative, short items like cards, noogramicles, which might also be called documents in certain contexts.) Documents are the pre-digital medium of recording and communi- cating information. The document metaphor on the digital world is unadjusted (Coombs et al. 1987). The system must give the sought information right there on the screen, formatted for the screen. The online page is the screen, or the window. “The world of the screen could be anything at all, not just the imitation of paper” (Nelson 1999). See also No Scroll. Furthermore, documents are for reading, but “online users don’t read—they scan” (Nielsen 1997). Clearly here Nielsen is referring to long texts as (bad) containers of oracles. If a user is looking for a certain information, they rather do it on a scannable content, e.g. a short bulleted list, than on a long running text.
No Scroll. A page that requires scrolling is not a page, it is a document—and as such it is banned (see No Documents). Online pages should fit the user’s screen, or the window. Scrollable windows limit the view. Raskin 2000 has shown that scrolling had a impact on the reading effort greater than that of turning or invoking pages. A document requiring scrolling is actually an even more ancient information medium than the modern book, or codex: it is a scroll. So scrolling in the online world is a regression of more than 1000 years, to pre-codex age.
However, the findings of Raskin 2000 are in part derived from the difficulty of positioning the mouse pointer on the scroll bar—and therefore this part has disappeared with the mouse wheel. Also, it should be noted the recent trend of very long web pages in very popular sites such as Wikipedia or Facebook. Their popularity seem to challenge this
3Please keep in mind that the concept of best practices is distinct from that of common practices. Unfortunately, general discourse often confuses the two. 52 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
principle (No Scroll). Although this is an ad populum argument, it should be taken in consideration. So we are not adamant in this principle. We are nevertheless still convinced scrolling is as least as costly as page turning or invoking (by clicking) in the context of document consultation—which is our object of study.
No Intrusion. The user should not be imposed any work other than his normal behaviour. For example, usability questionnaires are banned. Adaptive input should be restricted to the user normal activity. This also applies to authors. Regarding them, the baseline effort shall be that of the normal writing of texts including placement of non-text objects. The general baseline effort is that of non-adaptive systems. The user shall not be imposed any extra effort with respect to this baseline. Exception: Full Control may require extra effort for user levels above normal users and authors, e.g. configurators. Cf. usability studies e.g. Nielsen 1997. Deviation (hypothetical): some users may be willing to provide feedback. Most of this feedback is valid i.e. it is not intentionally boicoting. This set of users is representative of the whole. Therefore the system should provide a way to collect feedback, and use this feedback as valid knowledge.
Full Control. The system must allow the user to control every aspect of the adaptive behaviour of the system. In particular, it must allow the user to turn adaptivity off (and back on), to escape artificial stupidity (Raskin 2000).4 The means to exert this control must be readily available, right there alongside links or main options on the current page.5
Clear Labels. A link must be, at the source, or anchor, clear about its target, or distal content. That is, the interface must be so that users can effortlessly make a reasonably clear idea of the effect of activating the link—before activating it. A clear label approximates what Mancini 2004 calls a “predictable connection” (sec. 1.4.1 of Mancini 2004)
4Recently a concept of scrutability in adaptive hypertext has been proposed (Czarkowski 2006), which is essentially the same as this requirement of Full Control. 5And not buried under multiple layers of obscure menus, as is often the case, unfortunately. Recall Microsoft Excel or Word’s intelligence about capitalizing the first letter, or automatically numbering items—quickly, how do you turn it off? 3.1. MOTIVATION 53
Full Incrementality. Any operation on the system must be doable at any time, without requiring any kind of restart of the system, or stopping any functionality.
Best Practices. The system must be effective (do the job) and efficient (do it well). This is a general requirement of all engineered human interface systems, but we chose to state it explicitly because ever more often web systems are utterly unreliable or slow. Slowness, in particular, is a modern plague. It totally inverts the technology promise of making things easier and faster. Much too often we experience this: a person waiting for the technology, waiting for the computer to respond. Often the system is slow to the point that the corresponding manual procedure would be faster. Also, response latency induces interaction errors: repeated clicking on objects that should have reacted or disappeared, but that unnervingly have not done so, is a common phenomenon, which may cause malfunction.6 250 milliseconds. This should be the maximum response time of the interface to any user action.
According to Auber 2002 (who refers to Ware 2000 for details), 50 ms is the maximum delay between an action of the user and the displayed result on the screen for the user to believe in a causal link. Heer et al. 2003 (who refer to Card et al. 1983) indicate that this time can be relaxed to 100 ms. Raskin 2000 re- laxes further to 250 ms. Nielsen 1993 analyses the issue into the “three important limits”:
0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result. 1.0 second is about the limit for the user’s flow of thought to stay uninterrupted, even though the user will notice the delay. Nor- mally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data. 10 seconds is about the limit for keeping the user’s attention focused on the dialogue. For longer delays, users will want to perform
6Slowness of web systems is often attributed to network latency. We have observed that often the slowness is due to poor engineering of the server or client, including pages too large (XML is too verbose), little or null cache, little or null asynchronous updating, poor database administration (lack of proper indexes, little or null query optimization), etc. In sum, not applying best practices. 54 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect. (Nielsen 1993)
Other best practices include:
• search facility a la Google, preferably deeply integrated with the adaptive model (a fine discussion of the difficulties of integrating different search models is found in Ferr´e & Ridoux 2004) • site map, but only for sites where such a facility makes sense
Pervasive adaptivity. No element of the system should be excluded a priori from adaptivity. Notably, pages should be no less subject to adaptivity than links. That is, relinking—yes; but also: renoding. However, note that this feature is yet to be proven useful, and is still confined to the laboratory—including the current work.
3.2 Model design
3.2.1 The Shattered Documents model
The Shattered Documents approach prescribes that documents be taken apart into their smallest constituents of meaning, or noogramicles. Naturally the noogramicles must connect with each other, in order to create, ultimately, a navigationable network, or hypertext.
We will look firstly—and mostly—at hypertext documents created from traditional documents, like our running example the ARM. We have observed three dimensions in the traditional structure of documents—sequence, hierarchy, cross-reference. We transcribe these dimensions into types of connection in the network.
We have also observed that documents are made up of small narrative and non- narrative units of meaning. Clearly, these are the units that become noogramicles in a Shattered Documents model. Therefore the Shattered Documents model of documents 3.2. MODEL DESIGN 55
Figure 3.1: Model of the same document in figure 2.12 but with the shattered document approach and the two types of connection Next (N) and Child (C). C C
N 1. 1.1 1.1.1 (simile) C NNC CN
N C N C
C N
C N
Figure 3.2: Page made up of document fragments. 1.
contracted views 1.1 (of the 2nd text parag. of sec. 1, heading of sec. 1.1 and 1st text parag. of sec. 1.1, resp.)
is a network structure of two types of node and three types of connection (a di-nodul- tri-nexial model).
Figure 3.1 shows how the same part of the ARM in figure 2.12 is modelled with the shattered document approach and the two types of connection Next and Child. Figure 3.2 shows a page made up of the first five noogramicles in the model; the constituent noogramicles of this page are selected using spreading activation from the first one in a manner detailed later.
We have designed this model with the following desiderata in mind:
• applicable to a vast range of existing documents 56 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
• preservative of the original structure and content of legacy documents
• adequate for processing by the shattered documents approach, including spread- ing activation
• adequate to incremental authoring
To consolidate: a document is represented as a graph, or network datamodel of noogramicles interrelated by directed connections of three types—Next, Child, Refer— , as follows.
Next represents the linear order of paragraphs. Example: from a paragraph to its immediate successor. Note that, by the extended paragraph definition, Next also connects from a section heading to the first classic paragraph of the section, and from the last classic paragraph of a section to the heading of the next section.
Child represents the immediate subordinate relationship between paragraphs. Ex- amples: from a section to each of its subsections; from a paragraph introducing an enumeration (e.g. a bulleted list) to each item of the enumeration; possibly, even from a terminal section (i.e. a section without subsections) to each of its paragraphs
Refer represents other reference relationships. Examples: from a paragraph to a foot- note; from a paragraph to another paragraph or section (e.g. the so-called cross references, and see also references); from a index entry to its target paragraph or section; from a TOC entry to the corresponding section.
In the Child and Refer relationships, a section is represented by its first paragraph, normally a heading.
Original references anchored in sub-paragraph units (e.g. words) are represented as references anchored on the paragraph as a whole.
3.2.2 Adaptive information, and author as first reader
As we are targeting an adaptive system, we need a way to represent the corresponding information. The adaptative part of our model is compound of two main itens: 3.2. MODEL DESIGN 57
Pages. What the reader sees. A page is assembled from a small number of noogram- icles, in a manner detailed later. Naturally the user may navigate to another page. Pages are the adaptive output of shattered documents.
Travels. The navigation steps that usors (authors and readers) make in the document. Each travel is recorded, and used in adapting the construction of pages. Travels are the adaptive input of shattered documents.
The adaptative process integrates the two items, by assembling pages based on travel information. The main idea is to select the noogramicles that are most connected to the current one.
So, we must add the connection type Travel to the trio Next, Child, Refer already explained. Therefore, so far our document model is a network of noogramicles with four types of connection: Child, Next, Refer, Travel.
Finally, we interpret each of Child, Next, Refer, as Travel. This reinterpretation of the traditional document structure connection types Child, Next, Refer allows us to solve the “cold start problem”, and simplifies immenselly the process of spreading activation, as we shall see. This step is justified mainly because, if you look at it, the connections Child, Next, Refer are indeed the travels that the author intended the reader to make in the first place. Next is directly so. Child, Refer are contigently so—they are the paths laid out by the author for the reader to cross, wanting. Or, Child, Refer carry a rhetorical value—which amounts to the same effect (a contigent choice by the reader to follow).
3.2.3 Interface design
Our system is an interface into a large document. The interface unit is the page. Pages are accessed one at a time, normally. The document as a whole is partitioned, shat- tered, into fragments smaller than the page. These fragments are called noogramicles, for small pieces of knowledge.7
Each noogramicle has two renderings, or views: expanded, contracted. The expanded view is the noogramicle itself, normally. The contracted view is a clear label of the
7Noogramicles are also called knowledge atoms, or simply atoms, on other parts of this thesis, specially on software artifacts. Other words that were also, at a time, tried, and possibly used, include fragment, extended paragraph, and even texticle (for small text—this one was quickly dismissed though...) 58 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT noogramicle. Occasionally, the label equates the noogramicle, i.e. their contracted and expanded views are formally identical.
Figure 3.3 exemplifies this design. It represents the adaptive interface to a large document: the InteligˆenciaArtifical em Portugal website.8 On this design expanded noogramicles have a light gray background colour, and are on the top, whereas con- tracted noogramicles have a dark gray background colour, and are positioned below the former group. In fact all items are displayed, from top to bottom, by order of their Level of connectioness to the user request.
The noogramicle on the top is central, and represents the page for certain purposes, explained later. Actually, the values of Level shown correspond to the energy level from the spreading activation algorithm that discovers the noogramicles that are more connected to the central noogramicle (which as a reference Level of 1000). This algorithm is described fully later. On this design, the real estate on a page is divided equally between expanded and contracted noogramicles. The higher-ranking noogramicles are expanded.
Navigation, or travelling, on this design, is effected by recentring on a noogramicle. Normally, recentring is done on a non central noogramicle; but actually recentring on the central noogramicle is possible and may produce a different page, because connection levels may have changed as a consequence of travels made by other usors while this usor was (quiet) on this page.
Recentring is realised either by
• pushing the noogramicle button (the button at the start of the noogramicle labelled with the number and the contracted form of the noogramicle), or by
• selecting the noogramicle (checking the noogramicle checkbox, situated right at the start of the noogramicle) and pushing the Recenter button on the bottom of the page (to recentre on the central noogramicle only this form is available).
Actually recentring may be done on more that one noogramicle simultaneously (by selecting the respective noogramicles and pushing the Recenter button). What recen- tring does is to inkove the spreading activation algorithm with the selected noogram- icles energized, and record travel connections (explained later).
8Artificial Intelligence in Portugal, actually a branch of the website of the APPIA, Associa¸c˜ao Portuguesa Para a InteligˆenciaArtificial (Portuguese association for artificial inteligence), at www. appia.pt 3.2. MODEL DESIGN 59
Figure 3.3: Example design of our own adaptation model.
Buttons Shatter and Coalesce also work on selected noogramicles. These are authoring operations. Shatter splits up each selected noogramicle in two. Coalesce is the inverse operation.
Checkboxes Related are for relevance feedback. The usor may (or may not) check these boxes to indicate that the noogramicle is indeed related to their information need—whether the session has ended or not.
Finally, button End session serves two purposes:
• also relevance feedback, in combination with the Related checks; in particular, a session that has ended with Related checks is deemed successful
• navigation: it cleans up the interface by recentring on a neutral/start/search page.
The main adaptive input of our system consists of the choices, or travels, made by the usors. These travels are memorized in the computer as connections between the 60 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT respective noogramicles. The graph thus formed constitutes a model of the collective mind of usors. This modelling philosophy has been proposed and experimentally validated before in the endeavours surrounding the Principia Cybernetic Project, e.g. Heylighen 1997, Heylighen & Joslyn 2001, Bollen 2001.
This graph model of the collective mind is then automatically explored by means of spreading activation, to find the noogramicles more related to the central one (which has been chosen by the usor). Such noogramicles then form the page—the adaptive output—, in the manner already described (fig. 3.3 and surrounding text).
3.2.4 Detailed design with a network data model
To recap: a shattered document is divided into small constituends called noogramicles, which constitute the vertices of the network. The connections between the vertices are travels—representing both the original structure of the document and the recorded utilization, i.e. navigation, of the document, by its usors. So, a part of the usor model, namely, the past, known, behaviour of the usor, is contained in this database. The upper part of figure 3.4 illustrates this document model, a networked database of noogramicles (also called knowledge atoms, or simply atoms) as vertices, and travels as connections thereof.
Figure 3.5 represents the dynamic aspect of the system: a travel, effected by the user by clicking on an item of the prior page. This action represents a request for recentering on the chosen atom, which is done by serving a new page centred on that atom. Note that the travel is recorded as a new connection added to the database. The cycle then reapeats until the oracle is reached (or the session is abandoned or aborted for some reason).
3.3 Techniques and tools reused
In this section we describe spreading activation, the technique to explore network models, that we have selected for this thesis.9 Spreading activation is either
(1) the propagation of energy amongst the nodes of a network, usually formu-
9This section has its roots in Alves & Jorge 2005 “Minibrain: a generic model of spreading activation in computers, and example specialisations”, but has evolved greatly since then. 3.3. TECHNIQUES AND TOOLS REUSED 61
Figure 3.4: Relating the web page as seen and the graph model underneath.
database of atoms and travels
a page central view (not clickable)
expanded view 2
expanded view 3
contracted view 1 (label) contracted view 2 contracted view 3 contracted view 4 62 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
Figure 3.5: Travelling
new travel
central view (not clickable) central view (not clickable)
expanded view 2 expanded view 2
expanded view 3 new page expanded view 3
contracted view 1 (label) contracted view 1 (label) contracted view 2 contracted view 2 click contracted view 3 contracted view 3 contracted view 4 contracted view 4 3.3. TECHNIQUES AND TOOLS REUSED 63
lated as repeated matrix by vector multiplication (Bollen 2001, Chen & Ng 2004, Crestani & Lee 2000, Dominich 2003, Huang et al. 2004, Rome 2003), or
(2) the specification of a final state of energy distribution amongst the nodes, for- mulated as a set of simultaneous equations (Anderson 1983)
Probably the two formulations are equivalent under certain conditions. However we have not studied this possibility deeply. Although Anderson 1983 seems to be the seminal item on this area, method (1) is much more featured in the literature at large. So we have settled on method (1), which we describe next.
3.3.1 A unified model of spreading activation
We have developed a generic model and software module of spreading activation, and specialisations thereof to support a number of specific models in the literature. The unification thus provided has helped us (and others, cf. Scheir & Lindstaedt 2006) understand spreading activation in general and compare specific models. We also created a new specific model, Waterline10, that reduces the number of parameters of a class of specific models.
The specific spreading activation models studied include leaky capacitor variants (Anderson 1983, Bollen 2001, Rome 2003, Huang et al. 2004), reverberative circles (Dominich 2003), Wa- terline (Alves & Jorge 2005), branch-and-bound (Chen & Ng 2004, apud Huang et al. 2004), Hopfield net (Huang et al. 2004), Contextual Network Graphs (Ceglowski et al. 2003).
3.3.2 A didactical example
We have prepared a toy data set to verify the software, which also serves to take a quick look at how spreading activation works. It is the small network of concepts illustrated in Fig. 3.6. The connections are untyped, undirected, unweighted, conveying only a general notion of association. For example, the connection PET–DOG signifies that dogs are pets, connection MARIUS–LUCKY signifies that these two entities, or NAMEs, are related (most probably Marius owns Lucky, given their species), connection LUCKY–DOG signifies that Lucky is a dog, etc.
10Waterline was initially called Watermark on the cited workshop article. 64 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
Figure 3.6: Toy network, for a general understanding of spreading activation
NAME
MARIUS
LUCKY PET NINA SPECIES
DOG CAT HUMAN CLASS SNAKE
MAMMAL REPTILE
We tested variants of the LCM and Waterline models on this data. The transcript in Fig. 3.7 shows a Waterline session of three activations, or queries. Each query is launched using the following commands (technical details will be defined shortly).
Set NL Set the energy level of node N to L.
Go S Propagate S steps (or until some other stop condition occurs, as may happen with Waterline).
Each result is a list of nodes with non-null Waterline, in descending order of Waterline. The printed data represents the final state. The last query activates MARIUS, PET, and NAME as a representation of “the names of Marius’s pets”. Note how the system correctly high-ranks LUCKY and NINA. Compare with the simpler queries before. 3.3. TECHNIQUES AND TOOLS REUSED 65
Figure 3.7: Instances of spreading activation over the toy network (transcripts of sessions with the Minibrain program).
set marius 1 go 10 Level Waterline MARIUS... 0.000E+00 1.000E+00 HUMAN.... 0.000E+00 2.500E-01 LUCKY.... 0.000E+00 2.500E-01 NINA..... 0.000E+00 2.500E-01 NAME..... 0.000E+00 2.500E-01 Total.... 0.000E+00 2.000E+00 reset set marius 1 set pet 1 go 10 Level Waterline PET...... 0.000E+00 1.000E+00 MARIUS... 0.000E+00 1.000E+00 DOG...... 0.000E+00 5.000E-01 CAT...... 0.000E+00 5.000E-01 HUMAN.... 0.000E+00 2.500E-01 LUCKY.... 0.000E+00 2.500E-01 NINA..... 0.000E+00 2.500E-01 NAME..... 0.000E+00 2.500E-01 Total.... 0.000E+00 4.000E+00 reset set marius 1 set pet 1 set name 1 go 10 Level Waterline MARIUS... 0.000E+00 1.333E+00 NAME..... 0.000E+00 1.250E+00 PET...... 0.000E+00 1.000E+00 LUCKY.... 0.000E+00 5.833E-01 NINA..... 0.000E+00 5.833E-01 DOG...... 0.000E+00 5.000E-01 CAT...... 0.000E+00 5.000E-01 HUMAN.... 0.000E+00 2.500E-01 Total.... -2.220E-16 6.000E+00 66 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
3.3.3 Benefits of spreading activation for information retrieval
Computer models of spreading activation have been used for cognitive studies (Anderson 1983), electric circuit simulation, and information retrieval (Bollen 2001, Crestani 1997, Crestani & Lee 2000, Dominich 2003, Rome 2003, Ceglowski et al. 2003). Naturally, here we focus on the latter field, and specifically on the respective references.
In spreading activation approaches to information retrieval the nodes represent seman- tic entities e.g. documents, terms, queries, users, sessions. The propagation represents a search for the entities that are most related—similar, relevant, associated—to the initially activated ones.
The present study helped us understand spreading activation in general, the differences and commonalities between the specific models in the literature, and we believe that both the model and the module have potential for reuse in other systems. Real or potential advantages that spreading activation offers in relation to classical content retrieval techniques include:
Natural incrementality. The network is the data model. As new data arrives the network is updated immediately. This constitutes learning. The result of spreading activation immediately reflects the new state. Therefore the model is fully incremental of its own nature. — Real advantage.11
Content type independence. “Nodes are not retrieved on the basis of specific, in- dividual node characteristics (e.g. text content of metadata), but based on their connections to other nodes. Retrieval is therefore not limited to a document’s text content and can operate on nodes containing any combination of non-text content such as audio, video, text, digitally formatted text, etc.” (Bollen 2001, p. 247) — Real advantage.
Robustness to missing information. “Activation is propagated in parallel through the network connections, which makes retrieval robust to missing information, i.e. absence of specific node information such as keyterms or labels does not nec- essarily impede retrieval.” (Bollen 2001, p. 247) — Real advantage; Huang et al. 2004
11Incrementalily is a huge issue in theories and systems of information retrieval, machine learning, and artificial intelligence in general. It is the ability to deal with new data coming in for analysis, by integration with the model already in place which was made from the old data. Naturally this requires an adaptation of the model. In recent years the respective area of study has been called data streams (Gaber et al. 2005). 3.3. TECHNIQUES AND TOOLS REUSED 67
shows an example of this, solving the scarcity problem of recommender systems.
Post-coordination. Classic information retrieval is based on precoordination: in- dexes and similarity matrices computed prior to the search (Foskett 1996). The weakness of this approach is that it is fortune-telling: the system designer must choose a finite subset of possible coordination needs. Searches falling outside this set will be inefficient or impossible. Postcoordination solves this problem in theory. In practice, the efficient implementation of postcoordination requires a departure from classical approaches. Spreading activation is one possible such alternative—in our informed intuition one of very high potential.12
Ostensive querying. “The process of retrieval is initiated by patterns of node acti- vation which allows (or even requires) querying-by-example, also referred to as ostensive querying” (Campbell 2000). (Adapted from Bollen 2001, p. 247.) — Potential advantage.
Scalability and incrementability. With classic information retrieval techniques, like Latent Semantic Indexing , the poor scalability of the singular value de- composition algorithm remains an obstacle to indexing very large collections. While techniques have been developed for making incremental updates to a scaled collection, these changes typically cannot exceed a certain threshold without trig- gering a rebuild (Ceglowski et al. 2003). With spreading activation, “for additions, the graph server simply has to parse the new documents, and add additional connections between document nodes and existing term nodes” (idem).
3.3.4 The generic model
Spreading activation is the propagation of energy amongst the nodes of a network. Our generic model, and corresponding software module, consists of means to represent the (non trivially energised part of the) network, set and inspect its state, and advance the propagation, as follows.
Propagation is based on atomic units of change, or steps. Upon each step, each source node in the network transmits a certain amount of energy to each of its targets;
12This pre-coordination problem also exists in the field of relational databases. Which indexes to create and maintain? Here the DBA (database administrator) is the fortune-teller. Advances in database research include overcoming this problem. Cf. for example the network data model developed in this very thesis. 68 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT these elements are defined by the user of the generic model (not the end-user of an application) as the entities Amount/2, Sources/0, Targets/1.13
Amount(X → Y ) A function returning the (possibly null) amount of energy to transmit from node X to node Y at the point of call.
Sources A representation of the (possibly ordered) set of source nodes at the point of call.
Targets(X) A representation of the (possibly ordered) set of targets of node X at the point of call.
Amount/2, Sources/0, Targets/1 are the parameters of the generic model. A con- cretisation of these elements, i.e. their realisation as arguments to the generic model, constitutes a specialisation of the generic model.
The program maintains and computes a number of energy levels, or fields, namely: Level, Gain, Loss, Waterline. Level(X) denotes the current activation level of node X. The other energy fields are auxiliary.
Upon each step, the levels of each node are updated as follows. The prime sign (0) denotes the state after the step. The absence of the sign denotes the state before the step.
Level(X)0 = Level(X) + Gain(X)0 (3.1)
X Gain(X)0 = Amount(Y → X) (3.2) Y ∈Sources(X)
X Loss(X)0 = Amount(Y → X) (3.3) Y ∈Targets(X)
Waterline(X)0 = max(Level(X)0, Waterline(X)) (3.4)
Sources/1 is an auxiliary entity derived from the model parameters Sources/0, Tar- gets/1 in the expected way:
13We use the costumary notation F/n to identify a function-like entity named F with arity n i.e. having n input parameters. 3.3. TECHNIQUES AND TOOLS REUSED 69
Sources(X) = {Y : Y ∈ Sources,X ∈ Targets(Y )} (3.5)
Equations 3.1–3.2 define the effect of a propagation step.A user, or full step, often requires pre- or post-adjustment of some levels, depending on particulars of the specific model and network state.
The module has a minimalist design, to the extent that this does not compromise good software engineering. The parameter entities refer to nodes. Reference entails existence. If a node is invoked as input that has been not referenced before, it is assumed to exist with an activation level of zero. Naturally, existence and level of a node can also be asserted explicitly.
The following entities are also provided by the module to inspect the network state.
Total(EnergyField) The total energy of the field specified (Level, Gain, Loss, Wa- terline). Alternate notation: |EnergyField|.
Range(EnergyField, LowerBound, UpperBound, Order) The possibly ordered set of nodes in the specified energy range. LowerBound, UpperBound are energy values each tagged ˙ or ˚ for closed or open respectively, i.e. as either including or excluding the value respectively. Order is either <, >, or a comma for ascend-
ing, descending, or no order respectively. Occasionally we use an Interval Field notation for Range expressions as follows:
[x < y]EnergyField = Range(EnergyField, x,˙ y,˙ <)
]x < y]EnergyField = Range(EnergyField,˚x, y,˙ <) etc.
Finally, the module provides the procedure Step to advance the propagation, i.e. to perform the state change defined in equations 3.1–3.4; and procedure Assert to energise nodes explicitly (Fig. 3.8). Usage of the module consists in a realisation of the algorithm in Fig. 3.9, in terms of the entities described. 70 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
Figure 3.8: Step algorithm. Assert(x, EnergyField, z) tells the module that the node x has energy z on the specified field; in addition, Assert updates the totals and the Waterline level.
1 for each asserted node X: 1.1 Assert(X, Gain, 0) 1.2 Assert(X, Loss, 0) 2 for each X ∈ Sources: 2.1 for each Y ∈ Targets(X): 2.1.1 Assert(Y, Gain, Gain(Y ) + Amount(X → Y )) 2.1.2 Assert(X, Loss, Loss(X) + Amount(X → Y ))
3 for each X ∈ ]0, +∞[Gain: 3.1 Assert(X, Level, Level(X) + Gain(X))
Figure 3.9: Minibrain usage.
1 define Amount/2, Sources/0, Targets/1 2 assert initial state 3 while not satisfied: 3.1 pre-adjust 3.2 do one Step (Fig. 3.8) 3.3 post-adjust 4 collect results
3.3.5 About the implementation
The software module is called Minibrain. It is implemented in Ada in circa 300 lines of source code, excluding standard libraries. Internally, Minibrain uses Ada.Containers.Sorted Sets for the data structures.14 These structures use main memory, and provide state-of- the-art performance: “If N is the length of a set, then the worst-case time complexity of the Insert, Include, Replace, Delete, Exclude and Find operations that take an
14Actually, because Ada.Containers was due officially later than the time of writing, we used the well known and tried reference implementation AI302 mantained by Matthew Heaney. 3.3. TECHNIQUES AND TOOLS REUSED 71 element parameter should be O((log N)2) or better. The worst-case time complexity of the subprograms that take a cursor parameter should be O(1).” (Ada Reference Manual, A.18.9, 116/2).
All specified entities are represented rather directly. Entities that represent sets are implemented as iterators.
3.3.6 Leaky Capacitor Model (LCM)
The LCM is usually described mathematically, namely with matrix calculus, or linear algebra (Bollen 2001, Rome 2003, Huang et al. 2004).15
For a network of n nodes, a square n × n matrix M represents the propagation factors mij. Here and throughout this section, i = 1 . . . n, j = 1 . . . n.
The activation level of nodes is represented in an n-sized vector A of levels ai.A propagation step consists in updating A as follows (again, the prime indicates the state after the step).
A0 = M × A (3.6)
The matrix M requires some preparation. A propagation factor (often called connec- tion weight in the literature) combines a number of elements:
Connection weight wij The connection weight proper (this is why we don’t use the term for the combined entity). It represents the original, contingent strength, or weight, of the connection from node i to node j (if the connections are directed) or simply between i and j (if not). Let the n × n matrix W represent
these connection weights. Note wii = 0 for all i; more on this later.
Retention factor λ The fraction of energy retained by each node upon each step. Corresponds to the “speed of decay” γ in Huang et al. 2004 as follows: λ = (1−γ). The retention factor only applies to elements in the diagonal of M (see below).
Propagation efficiency α The fraction, of energy of a node, that is propagated, from that node, to each other node, on each step. Same as in Huang et al. 2004.
15Each of these sources have faults of varying severity in the respective descriptions. Hopefully on our paper Alves & Jorge 2005, integrated herein, the LCM model has been presented correctly for the first time in the literature, with the possible exception of Anderson & 1983 . 72 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
Finally, M is defined as follows:
ˆ M = λ × In + α × W (3.7)
( W T if connections are directed Wˆ = (3.8) 0.5W otherwise
The diagonal of M—the connections of each node to itself—is used to represent the retention of energy along time. So for each node i, mii = λ.
If the connections are directed, and the direction of the connection is the direction of the propagation, the matrix must be transposed, in order for the multiplication to have the expected effect.
If the connections are not directed, M is symmetric. In this case, because both sides of the diagonal enter in the multiplication by A, the respective values must be halved, in order for the multiplication to produce the desired result.
The termination condition is simply a fixed number of iterations.
LCM-specialisation of Minibrain
The mathematical definition of LCM can be mapped directly to Minibrain parameters as follows:
SourcesLCM ≡ {1 . . . n} (all nodes) (3.9)
TargetsLCM(i) ≡ {1 . . . n} (ditto; argument ignored) (3.10)
AmountLCM(i → j) = mij × Level(i) (3.11)
However this is simply performing the multiplication M × A, i.e. Minibrain is being used just as a matrix calculator. This might not be the best approach for large and sparse matrices, as is usually the case with real data (for example web navigation data configures a graph with average connection degree in the region of 2 to 3). So an 3.3. TECHNIQUES AND TOOLS REUSED 73 alternate specialisation, more efficient, is one that only asserts the nodes with non-null energy, or that corresponding to non-null connections, or both.
SourcesLCMA ≡ {i : ai > 0} (nodes with non-null activation level) (3.12)
Targets (i) ≡ {j : m > 0} (nodes non-nully connected from i) (3.13) LCMM ij
The specialisation of Amount remains the same. This specialisation can still be seen as a matrix calculator, but this time optimized to sparse instances of M.
Yet another alternative is to use W , not M, in the specialisation of Targets, and effect the retention in a post-adjustment operation.
Targets (i) ≡ {j : w > 0} (nodes non-nully connected from i) (3.14) LCMW ij
Post-adjustment: Level(i) = λ × Level(i), ∀i ∈]0, +∞[Level (3.15)
This specialisation is yet more efficient because it only processes the elements in the diagonal of M that need processing (instead of the whole diagonal). This has impact for large number of nodes, irrespectively of the connection degrees.
3.3.7 Reverberative Circles (RC)
“Activation takes place according to a winner-take-all strategy. The acti- vation is initiated at the query node, and spreads over along the strongest connection thus passing on to another node, and so on. After a finite number of steps the spreading of activation reaches a node already affected (in the worst case it passes through the entire network and eventually gets back to the query node) thus giving rise to a loop called reverberative circle (as a model for short term memory). Those nodes are said to be retrieved which belong to the same reverberative circle, and they are ranked in the order in which they were traversed.”
(Dominich 2003, pages 170–171) 74 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
For the RC-specialisation of Minibrain we employ the trick of equating the level of active nodes with the step count, starting at 1 for the initial query node. This allows us to use Range to both define Sources and observe the result in the prescribed order.
SourcesRC ≡ first of ] + ∞ > 0[Level (node with the highest level) (3.16)
TargetsRC(X) ≡ node to which X is most strongly connected (3.17)
AmountRC(X → Y ) = step count (3.18)
3.3.8 Waterline
We created the Waterline model to use total decay of the network as a termination condition. In the literature it has been noted that “no general methodology has been established to tune the several parameters involved in setting up an efficient spreading activation system” (Bollen 2001, p. 257)
The LCM has two parameters: M, and the number of iterations. Eliminating one would at least ameliorate the situation. The LCM with certain transmission factors, namely positive values less then 1, configures a network of decaying total energy. The Waterline is a memory associated with each node that stores the highest level attained by the node since the start of the propagation. Thus it is possible to let the propagation die its natural death, and still get results: the Waterline values.
3.4 Algorithms
Here we formally define the algorithms used in the shattered document model. The model is organized into three main operations: start page, recentring, learning. For each operation: a main, or nominal algorithm is defined, representing our proposal of shattered documents; alternative, or variant algorithms are also defined, representing alternative approaches for comparison with ours. 3.4. ALGORITHMS 75
3.4.1 Overview
In an shattered document configuration, each page is a ranked assembly of noogramicle views. The highly ranked items are expanded, the lowly ranked are contracted. Figure 3.4 illustrates this design. Normally, the noogramicles that constitute a page are the most connected to the central one, which is the single highly ranked one. They can be selected by any adequate ranking algorithm. We call this operation recentring. Our main approach, which we define here, was to apply spreading activation to the central noogramicle. Possible alternatives include Markov chains, collaborative filtering, association rules, pure chance. Actual alternatives tried, for comparison with the main approach, are Markov chains and chance.
Recentring applies normally for the ensuing pages in a session i.e. to the pages after the first, because normally the central noogramicle is identified by clicking on the respective label on the previous page. For the first, or start page, there is no chosen central noogramicle yet, so an algorithm is required to either
i select a central noogramicle without user input (and then apply recentring)
ii create an otherwise integral page
We were not able to devise a sensible way to choose a single noogramicle for method (i). We have opted for the following instance of (ii): serve the set of the items most connected globally. This page represents the page with more ability to lead to other places, globally. We call this the super page. Again we use spreading activation to construct this page. Like recentring, the start page algorithm could also be done in any of a number of ways. We tried a few, exploratorily. We chose the way defined here because we wanted to experiment with spreading activation, and exploration indicated good results. We also made a configuration with a random start page.
Finally, there is the learning operation: the network of recorded travels is updated with the user input. This is the source for adaptivity. Because each page represents a set of atoms, the user action can be translated into travels between atoms in a number of different ways. The nominal is: central to central. Other alternatives were tried and preterred. Variants retained for comparison are random learning and null learning. 76 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
3.4.2 Formalization
Atoms is the full set of atoms, or noogramicles, of the entire document. In the algorithms, each atom has a memory for its current level of activation, or energy.
Active is the subset of active Atoms i.e. the atoms with a non null activation level.
Propagation is defined with the LCM model of spreading activation described in section 3.3.6; the operation “propagate” signifies one propagation step (equation 3.6). Propagation is along the recorded travels, which are directed pairs (Source Atom, Target Atom). In the algorithms, the energy is propagated along a travel either
• in the forward direction, from source to target of the travel: default case.
• backwards, from target to source of the travel: where specified.
In both algorithms, the propagation parameters α, λ (cf. session 3.3.6) are set to α = 1, λ = 1.
Not Seen Yet is the set of atoms not seen yet in the session to which the recentring operation pertains.
3.4.3 Start page algorithms
The main start page algorithm in defined in figure 3.10. All items in the entire document are energized; a small number of backward spreading activation steps take place; the most energized items represent the most connected ones. This algorithm is implemented by function Super Page in package Kasim2.Activation (body), lines
390-418 (see Appendix B), with nominal parameters Ls = 0.001,Ns = 5.
Figure 3.10: Super Page algorithm Given the required Page size:
1 activate all Atoms with the same amount Ls of energy
2 do Ns times 2.1 propagate back 3 return the Page size most active atoms in Atoms, in order of activation level 3.4. ALGORITHMS 77
3.4.3.1 Variants
The random variant for the start page is trivial: a set of randomly chosen items, randomly ranked, in a uniformely distributed way. Such algorithm is implemented by function Random Page in package Kasim2.Activation (body), lines 420–436.
3.4.4 Recentring algorithms
The main recentring algorithm is defined in figure 3.11. The selected item is activated, and energy is spread; the selected item becomes the central item of the new page; the most energized items constitute the remaining items of the new page. This algorithm is implemented by function Recentre in package Kasim2.Activation (body), lines 202–
284 (see Appendix B), with nominal parameter Lr = 1.
Loop 2 is expected to terminate under normal conditions, namely an adequate docu- ment graph and spreading activation parameters, which are provided in the implemen- tation.16 Step 3 is to ensure that Central comes up top in the resulting page. Recall that this operation is done in the context of a current session. The server is supposed to memorize the pages served (the page trail) and in particular the atoms already seen in this section, which information is required by the algorithm to implement the respective heuristic (Not Seen Yet).
Figure 3.11: Main recentring algorithm Given a Central atom and the required Page size, and with a function Selected = Active ∩ (Not Seen Yet ∪ {Central}):
1 activate Central to level Lr 2 while |Selected|h Page size 2.1 propagate 3 reactivate Central to the highest value possible 4 return the Page size most active atoms in Selected, in order of activation level
16In the implementation, also defensive programming provisions are made to ensure termination, namely the whole invocation is enveloped in a timeout construct. 78 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
3.4.4.1 Variants
The random variant of recentring is trivial and equates the random variant for the start page: a set of randomly chosen items, randomly ranked, in a uniformely dis- tributed way. Such algorithm is implemented by function Random Page in package Kasim2.Activation (body), lines 420–436.
The Markov chains variants use first order Markov chains as defined on Borges & Levene 2008. We reutilize the machinery put in place for spreading activation. We observe that the first order transition probability of Markov chains is proportional to the travel count, so spreading once from the anchor will energize the candidate items adequately. A single propagation step is taken. The page size might be incomplete. The pure Markov chains algorithm is defined in figure 3.12. The Markov chains algorithm incorporating the same Not Seen Yet heuristic as the nominal algorithm (figure 3.11), is defined in figure 3.13. The algorithms are implemented by function Recentre in package Kasim2.Markov (body lines 74–124), and function Recentre in package Kasim2.Markov With Heuristics (body lines 74–148), respectively (see Appendix B).
Figure 3.12: Pure Markov chains recentring algorithm Given a Central atom and the maximum Page size:
1 activate Central to level Lr 2 propagate 3 reactivate Central to the highest value possible 4 return the Page size (or less) most Active atoms, in order of activation level
Figure 3.13: Heuristical Markov chains recentring algorithm Given a Central atom and the maximum Page size, and with a function Selected = Active ∩ (Not Seen Yet ∪ {Central}):
1 activate Central to level Lr 2 propagate 3 reactivate Central to the highest value possible 4 return the Page size (or less) most active atoms in Selected, in order of activation level 3.4. ALGORITHMS 79
3.4.5 Learning algorithms
In our system, learning consists in recording the travels made by the usors. Simple as this concept seems, it poses a problem in a shattered document environment: since the pages are made of atoms, which connections between which atoms should represent a travel between pages? Given that we already have the notion of central atom that is a kind of representative of the page, we intuited that travelling should be recorded as a connection between the central atoms travelled from and to, i.e. as a connection from the central atom of the source page (the page where the user has clicked) to the central atom of the target page (the atom clicked upon). This central to central strategy is our main learning strategy, which effectiveness we have verified against the obvious alternatives. All are defined as follows, and coded as enumeration Update Travels T on package Kasim2 (spec), lines 184–192. The respective logic is implemented in the body of procedure Simulate Session on lines 149–221 of package Kasim2 (body).
Central To Central Increment the travel count from the current central atom to the current choice = next central atom. (Default behaviour.)
Page To Central Increment the travel count from the current entire page (all atoms except choice) to the current choice = next central atom.
All Previous Central To Central Increment the travel count from all previous and current central items to the current choice = next central atom.
All Previous Page To Central Increment the travel count from all previous and current pages, from all items of each page (except the current choice), to the current choice = next central atom.
Randomly Increment the travel count from a randomly chosen atom to a randomly chosen atom in a uniformely distributed way. This variant serves to test the respective random configuration using the same machinery put in place for all configurations.
Central To Page Increment the travel count from the central atom to all items of the next page.
Page To Page Increment travel count from each page item (except choice) to each item of next page. A lot of connections viz. (N-1)*N; normally N=10; 90 connections. 80 CHAPTER 3. A NEW MODEL FOR ADAPTIVE HYPERTEXT
False No increment. No learning. This variant serves to test non-adaptive configura- tions, notably the original structure of the document, using the same machinery put in place for all configurations.
3.5 Summary
In this chapter we have defined the principal algorithms of the shattered document approach—Start Page, Recentre, Update Travels, and the algorithms of spreading activation used to implement them. Next we will define the experimental setup including the simulator. Chapter 4
Experimental methodology
When you can measure what you are speaking about, and express it in numbers, you know something about it. Lord Kelvin
We have defined the experimental methodology described in this chapter, with the objective of evaluating our approach. We have proceeded via simulation i.e. we have designed a user simulator, based on existing findings about hypertext user be- haviour (Ahn et al. 2005, Bollen 2001, Sosnovsky & Brusilovsky 2005, Olson & Chi 2003, West et al. 2009).
The advantages of a simulation methodology over a live user study include the ability to experiment with many different configurations or parameters, and absolute control over all experimental conditions—namely assuring that the conditions that do not pertain to the variants under comparison are exactly the same. The disadvantages of a simulation methodology over a live user study include the fact that the simulation is a simplification of the reality—with the risk of being an over-simplification—and the reliability of the results depends on the proper calibration of the simulator—which can only be approximated and based on theory, in the absence of live data.
Live user experiments are essential for hypertext usabilitity studies. We have used simulation based on existing results from live studies. We wanted indeed to exper- iment with many different configurations of hypertext, and parameters of spreading activation. We hoped to conduct a proper user study too. The construction of the
81 82 CHAPTER 4. EXPERIMENTAL METHODOLOGY simulator and of the adaptation software, and the many experiments done therewith, has turned out to attain an ammount of work such as to preclude the realisation of the user study within the limits of the thesis. It is now hoped for future work (see chapter 6 Conclusions).
4.1 Simulation
To validate our hypotheses we have conducted a number of experiments via simulation. For that, we have designed and implemented a simulator, along with some other necessary programs. The design of the simulator is an interesting contribution on its own. We intend to emulate a set of usors utilizing a large document via an hypertext interface. The whole system is comprised of a large document, the usors of that document, and usor sessions whereby the usors navigate the document to pursue an information need. The concept of session is the crux of the system, as it connects the two models of document and usor. This is done via a certain model of usor behaviour, or navigation, and a transposition of the vague concept of information need to the tangible construct of an oracle (the noogramicle that holds the sought information).
We have designed the simulator of usor sessions—a type of construct sometimes called user model in the literature—, integrating theoretical and experimental re- sults from Ahn et al. 2005, Bollen 2001, Sosnovsky & Brusilovsky 2005, Olson & Chi 2003, West et al. 2009 and others. The main design premise might be called the Smart User Assumption, whereby the user chooses the right link, asymptotically. This assumption is related to Clear Labels (section 3.1.2). Support for this assumption is as follows. For the case when the oracle is only one click away, the label clearly identifies the oracle, and therefore the user selects the item easily.
When the oracle is further away than one click, the intelligence or intuition of the user takes place to select the item most likely to lead to the goal.1 Bollen 2001 has demonstrated that users successfully apply a hill climbing strategy to navigate in a web of words. Olson & Chi 2003 provides additional support for the fact that
1In reality, this is not always true. Usors make errors, or the labels are not always clear enough, or both. A possible refinement to our model could be to associate a probability of the user to choose the best link to follow. Additionally, the simulator would require the possibility for the user to backtrack. However, we are conviced these hypothetical faults or approximations of the simulator do not affect the comparative evaluation. Recall that exactly the same simulator is used for all configurations of adaptive hypertext under experiment. 4.1. SIMULATION 83 users are good sniffers. West et al. 2009 shows that hypertext users “leverage semantic associations based on background knowledge of many common sense facts, and select links according to this knowledge”. And let us recall from chapter 2 how searchers use “a combination of ideas such as what the target information might look like, where it might be found, or how one might go about tracking it down” (Campbell 2000). All these results have been obtained from controlled experiments with real users. They have provided inspiration and support for the simulator.
Another fundamental trait of the simulator is the concretization of the information need of the usor in a session as an oracle. An oracle is a small set of noogramicles, normally just one, holding the information sought by the usor in a session. The simulator supports a statistical model of oracles, based on a power law distribuition of their popularity: a few oracles are very popular, many oracles have low popularity. Actually in the final experiments we have used a uniform distribution of a subset of atoms as the oracle model, as an approximation of the power law distribution of all; exploratory experimentation with both types of distribution has indicated that the former is indeed a good enough approximation of the latter in our system i.e. they showed similar results.
4.1.1 Formalization
We reuse formal entities introduced in the formalization of the main algorithms of shattered documents (section 3.4.2).
The user is simulated by the function Choose (figure 4.1) that, given a page in a session, identifies the item on that page that the user will click on. Because we are using simulation, we know the oracle of each session. From the Smart User Assumption, the user clicks on the item most likely to lead to the oracle. We interpret this likelihood as connectedness in the travel graph, and again use spreading activation to explore this connectedness: the oracle is energized and its energy is back propagated until (at least) one link of the page being seen is activated.
The Choose algorithm in formally defined in figure 4.1. This algorithm is implemented in function Choose of package Kasim2.Activation (body), lines 32–110 (see Appendix
B), with nominal parameters α = 1, λ = 1, LZ = 1.
The loop 2.2 is theoretically guaranteed to terminate, under normal conditions, namely a fully connected graph and no decay (λ ≥ 1): eventually all atoms will receive energy, 84 CHAPTER 4. EXPERIMENTAL METHODOLOGY negating condition 2.2.1 trivially. Nevertheless the costumary defensive programming provisions were included in the implementation.
Figure 4.1: Choose algorithm Given a page of Page Atoms, of which some are Contracted, in a session with oracle Z, and with the function Selected = Active ∩ Page Atoms:
1 if Z ∈ Contracted then 1.1 return Z 2 otherwise
2.1 activate Z to energy level LZ 2.2 while Selected = ∅: 2.2.1 propagate back 2.3 return the most active Selected atom
4.1.1.1 Variants
A random variant of Choose was prepared for verification purposes. The algorithm is trivial: pick a random page item (non-central), in a uniformely distributed way. Such algorithm is implemented by function Random Click in package Kasim2.Comparate.Configurations (body), lines 12–15.
4.2 Experiments
Firstly we have conducted verificatory and exploratory small scale experiments, with artificial documents. Secondly we have realised real life size experiments with a real document, the Ada Reference Manual, or ARM.
The first cycle helped us understand the parameters involved, and provided indicative measures of performance of our approach. These indicative results were encouraging. Figure shows the evolution of session size for ten oracles in an artificial document of 60 atoms. This first cycle of experimentation is fully described on appendix C.2. 4.3. PARAMETER SETTINGS 85
Figure 4.2: Evolution of session size in an exploratory experiment
These preliminary results are a good illustration of the ability of the method we propose to reduce the user effort, here measured as session size. However, to test our hypotheses, we will use a more robust setup, which we describe henceforth on this chapter. We intend to make a comparative evaluation of several approaches to adaptive hypertext. Namely, we intend to compare:
• an adaptive system against one without adaptation
• the shattered documents approach against the (unchanged) traditional structure for documents
• the spreading activation method against another, more standard, method (Markov chains)
4.3 Parameter settings
The experiments consist in the generation of sessions, using the simulator, for different settings, or configurations. The system comprises two models (fig. 4.3). 86 CHAPTER 4. EXPERIMENTAL METHODOLOGY
Figure 4.3: Top level models and respective components of the experimental setup. The subsidiary nodes represent the main components of each model.
Setting an experimental configuration mainly implies defining the parameters of the Adaptatin model. Settings affecting the User Model in non-trivial ways serve mostly as control configurations, like a random user, to verify that our normal User Model, featuring smart user heuristics, is indeed smarter, better than chance.
Components Start, Choose, Recentre, Update Travels are the main subprograms used by the session generator. These subprograms correspond to the operations and algo- rithms already described (chapters 3–4).
In total we have done hundreds of experiments, to explore the universe of possible configurations, to understand the interaction between the parameters. Figure 4.4 and table 4.1 exemplify explored configurations. Finally we have reached the set of configurations that we report fully on Appendix D—and that truly represent the touchstone of the current work—and we have reached the nominal values specified in table 4.1 i.e. the values used by default in the configurations where they apply normally. The quantity of propagation steps in Recentre has no nominal value because ultimately we have adjusted the algorithm to dispense with it i.e. use an independent termination condition cf. figure 3.11. 4.3. PARAMETER SETTINGS 87
Table 4.1: Some scalar parameters of the simulator Parameter values explored nominal value quantity of atoms 30; 100; 1000; 5000 1000 quantity of oracles 1; 5; 10; 50; 100 50 page size (quantity of items on a page) 3; 5; 10 10 quantity of expanded items on a page 1; 2; 3 3 quantity of propagation steps in Super Page 1; 2; 5 5 quantity of propagation steps in Recentre 1; 2; 3; 4... see text λ parameter of spreading activation 0; 0.1; 0.5; 0.8; 1.0 1.0 α parameter of spreading activation 0.1; 0.5; 0.8; 1.0 1.0
Figure 4.4: Partial configuration map of the simulator. Each connection between a component (cf. fig. 4.3) and a subsidiary node represents a parameter.
#
"
! !
88 CHAPTER 4. EXPERIMENTAL METHODOLOGY
Table 4.2: Distribution of nodes per level in the ARM Level Nodes Headings Paragraphs Fanout H-Fanout 1 1 (0.02%) 1 (0.02%) 0 (0.00%) 13.00 13.00 2 13 (0.31%) 13 (0.31%) 0 (0.00%) 10.62 7.70 3 138 (3.30%) 100 (2.40%) 38 (0.91%) 21.10 0.99 4 2110 (50.52%) 99 (2.37%) 2011 (48.16%) 19.34 0.00 5 1914 (45.84%) 0 (0.00%) 1914 (45.84%) 0 0 Total 4176 (100.00%) 213 (5.10%) 3963 (94.90%) 19.60
4.3.1 The document
For a real large document we have used the core part of the Ada Reference Manual, or ARM (i.e. excluding annexes), which consists of 13 chapters with the hierarchical structure depicted in figure 4.5, with five levels and a total of circa 4000 nodes.2
Figure 4.5 shows clearly that most terminal nodes are shared between levels 4 and 5. Table 4.2 precisely describes the complete distribution of nodes per level. A non-root node is either a heading or a paragraph. The Fanout values represent the average fanout for the level. H-Fanout represents the average fanout taking into account Heading nodes only. The Fanout value on the bottom Total line is the ratio Total of Nodes Total of Headings , representing the micro-average fanout of the whole tree. The number of paragraphs and page length of each section are derived from the file size and observation in several browsers of a random sample of sections, reproduced in table 4.3. Namely, the random sample of sections is used to determine the average paragraph size (in characters). This value is then used to compute the number of paragraphs of File size any section in the obvious way: Number of paragraphs = Average paragraph size .
4.3.2 One thousand nodes
We ended up using a slightly reduced version of the ARM, for practical reasons (speed), namely versions of 1000 nodes. Exploration indicated that the results for 1000 nodes and for the whole set of circa 4000 do not differ significantly, that is, the changes in results are linear with the number of atoms.
2We have used the HTML version made publicly available on the internet by the Ada Resources Association. 4.3. PARAMETER SETTINGS 89
Figure 4.5: Hierarchical structure of the ARM (excluding annexes).
Level 1 Level 2 Level 3 Level 4 Level 5 (book) (chapter) (section or paragraph) (subsection or paragraph) (paragraph)
We used two reduced versions, Cut and Trimmed. Cut consists simply of the first 1000 nodes of the whole, in sequential order. This results in a low fanout of 3 for the 90 CHAPTER 4. EXPERIMENTAL METHODOLOGY
Table 4.3: Random sample of sections Sample Nr. Section File size Nr. of paragraphs Page length 1 1.2 5832 11 2 2 2.1 15892 30 4 3 3.7 18538 38 4 4 4.1.4 11547 16 3 5 4.9.1 5517 8 2 6 8.4 8936 17 2 7 8.5.5 5743 7 2 8 9.7.3 5860 6 2 9 11.6 7361 7 2 10 13.5.3 6258 9 2
Table 4.4: Distribution of nodes per level in the Cut ARM Level Nodes Headings Paragraphs Fanout H-Fanout 1 1 (0.10%) 1 (0.10%) 0 (0.00%) 3.00 3.00 2 3 (0.30%) 3 (0.30%) 0 (0.00%) 8.66 7.00 3 26 (2.60%) 21 (2.10%) 5 (0.50%) 22.52 1.48 4 473 (47.30%) 31 (3.10%) 442 (44.20%) 16.03 0.00 5 497 (49.70%) 0 (0.00%) 497 (49.70%) 0 0 Total 1000 (100.00%) 56 (5.60%) 944 (94.40%) 17.85
first level. Trimmed consists in the first 1000 nodes of, the whole trimmed by one level, i.e. the first 1000 nodes that neither belong to level 5 nor are headings at level 4. To clarify: the height of Cut is 5, the height of Trimmed is 4. See tables 4.4 and 4.5 for the detailed structure.
Another interesting document statistics is the average distance from the root: it is a baseline that our adaptative method is supposed to improve upon. We have computed that value for the three legacy document variants:
Average distance from the root... in the Whole ARM = 3.41 in the Cut ARM = 3.46 int the Trimmed ARM = 2.92 When evaluating results of applicable configurations, we will consider these values. 4.4. EVALUATION METHODOLOGY AND MEASURES 91
Table 4.5: Distribution of nodes per level in the Trimmed ARM Level Nodes Headings Paragraphs Fanout H-Fanout 1 1 (0.10%) 1 (0.10%) 0 (0.00%) 6.00 6.00 2 6 (0.60%) 6 (0.60%) 0 (0.00%) 9.50 7.50 3 57 (5.70%) 45 (4.50%) 12 (1.20%) 20.80 0.00 4 936 (93.60%) 0 (0.00%) 936 (93.60%) 0 0 Total 1000 (100.00%) 52 (5.20%) 948 (94.80%) 19.23
4.4 Evaluation methodology and measures
4.4.1 Session size
In the first cycle of exploratory experiments we have used solely the common measure of session size (Bra et al. 1999). Session size is the number of pages visited in a session = number of clicks + 1. It is largely interpreted as a simplistic measure of the effort spent by the user pursuing an information need (cf. Bra et al. 1999 and references therein).
4.4.2 Session cost
In order to make meaningful comparisons across some configurations, we generalized the quality measure, from simple session size, to session cost. This is necessary because, on the limit, a configuration of only one large page containing the entire document has a constant session size of 1—thus unfairly winning the competition even before the start.
So we introduce session cost as a combination of session size with the extra cognitive effort associated with poorer or null navigational aids, longer pages (requiring scroll), and a large quantity of links (requiring more examination). This is in line with the methods described on Raskin 2000.
It sounds natural, practical, reasonable, to extend the unit of session size, the page, to this new unit of session cost. We call it session cost point, or just point. We formalize session cost as the sum of the cognitive effort terms involved. We have identified the extra cognitive factors described below. For each factor, we introduce a formula for its contribution to the cognitive cost. These factors affect the page. Naturally, the session cost is the summation of the page costs. 92 CHAPTER 4. EXPERIMENTAL METHODOLOGY
We quantify cognitive effort in points directly related to the session size component. We establish a basis: for a configuration of no scroll and a fixed, low page size (quantity of page items, or links) of at most ten, we equate session cost with session size, and therefore define a fixed page cost value of 1 point. From this basis we define page cost for non-trivial pages as follows.
Scroll cost. The cognitive cost associated with pages longer than the screen is related to that length (Raskin 2000).
Scroll Cost = max(1, 0.5p/s) (4.1)
where p is the page length, and s the screen size, measured in the same units, e.g. characters. We divide in half because of the reasonable presumption that, on average, only half the long page needs scrolling before an appropriate link or the oracle be found.
Choice cost. If the number n of links exceeds the critical value of seven minus or plus two (Miller 1956), then an extra cognitive effort is imposed upon the user, proportionate to the number of links. From our basis of a fixed cost of one point for ten or less items (close enough to the critical number 7 + 2 = 9), we derive:
Choice Cost = max(1, n/10) (4.2)
We also meter the real time duration of sessions, but only for verification purposes. It is likely that correlations between session size or cost and session duration in real time exist, but, because we are using a simulator, our times are likely to be bound to different complexities than those of real users. We have not investigated this formally.
4.5 Statistics
Each configuration run is a simulation of sessions. Each session pertains to an oracle, i.e. it is a session of a user searching for the oracle. Normally we run a number of sessions ten times the number of oracles. For result reporting and statistical analysis, session results are grouped by oracle. 4.5. STATISTICS 93
Each session is metered for its Duration (or Duratio3), Size and Cost. Duration is used for verification only. Duration is measured in seconds. For comparison we look at measures Size and Cost, and particularly at Cost.
For each configuration, the following report items are compiled. Except for the Outcomes table, an item is compiled for each quality measure Duration, Size, Cost. All these individual, per-configuration tables and charts are contained in Appendix D. Comparative tables and charts are presented in the next chapter. Naturally the values in the comparative items equate the respective values in the individual items.
Outcomes table. Statistics of success and failure of sessions.
Results table. Statistics of the quality measure.
Evolution table. Values of the quality measure along the experiment.
Evolution chart. The Evolution table in graphical form.
Moving average chart. The evolution of quality measure for all sessions distinctly, with a moving average line.
In the Evolution items, the oracle line values are compacted, as follows: the session order is considered for each oracle independently, and the last value is repeated for missing sessions (becase oracles may have slightly different numbers of sessions each). For example, a (toy) configuration with three oracles and the seven sessions with Size result
Session Oracle Size 1 2 5 2 3 7 3 2 4 4 1 2 5 2 3 6 1 1 7 3 6 has the results sequence by oracle as follows
Oracle 1 — 2, 1
Oracle 2 — 5, 4, 3
Oracle 3 — 7, 6
3The latin translate Duratio was used in the source code, because Duration is a reserved identifier in the programming language used; the form Duratio might surface on a few computer generated items e.g. charts. 94 CHAPTER 4. EXPERIMENTAL METHODOLOGY and thus gives the Evolution table
1 2 3 Oracle 1 2 1 1 Oracle 2 5 4 3 Oracle 3 7 6 6 Total 14 11 10 Average 4.67 3.67 3.33 In the Moving Average charts the values are not separated by oracle, and therefore not compacted, nor repeated. They are the exact values in the experiment, and appear in the actual order of sessions in the experiment.
The statistics in the tables are as follows.
4.5.1 Common and bottom line statistics
The tables are divided into two parts of value lines:
• the oracle lines, carrying values for the sessions pertaining to each oracle.
• the bottom lines, with total and central statistics for all oracles (and therefore all sessions).
The following statistics are common to all tables except: Evolution tables do not have Micro avg, because they have no rates.
Oracle i. Introduces the line of values for an oracle group, i.e. for the sequence of sessions pertaining to the orcale. The i is just a sequential number in the table, for reference.
Total. The sum of all values in the column.
Average. The average, or mean, of all values in the column. Therefore, when the values are rates, this is the macro-average (Manning et al. 2009). The macro- average expresses a centrality, an expectation for an individual oracle.
Micro avg. Micro-average (Manning et al. 2009): the rate of averages. The micro- average expresses a centrality, an expectation, for the whole configuration. When the main statistic is not a rate, the micro-average equates the macro-average on the Average line, and the micro-average is not shown. See the sections on statistics for specific tables for the case-by-case definition of the respective micro- averages. 4.5. STATISTICS 95
4.5.2 Statistics in the Outcomes tables
Node id. The node number of the oracle in the tree or sequence of atoms that repre- sents the legacy document. Not really a statistic, but useful for investigating the reason of possible deviant outcomes or results for an oracle, e.g. a Failure rate of 100% or a negative Gain. This id is the same for the oracles in all other configuration tables, but is only recorded here.
Total. The total number of sessions.
Timeout. The number of sessions that exceeded the maximum Duratio allowed. Such sessions are considered to have failed.
Sizeout. The number of sessions that exceeded the maximum Size allowed. Such sessions are considered to have failed.
Costout. The number of sessions that exceeded the maximum Cost allowed. Such sessions are considered to have failed.
Failed. The number of failed sessions:
Failed = Timeout + Sizeout + Costout
Successful. The number of successful sessions i.e. that have not failed:
Successful = Total − Failed
Fail rate. The percentage of sessions that have failed: Failed Fail rate = Total Success rate. The percentage of sessions that have succeeded (not failed): Successful Success rate = Total Micro avg. Micro-averages (see section on bottom line statistics for a general dis- cussion): Average of Failed Micro avg of Fail rate = Average of Total
Average of Succesful Micro avg of Gain = Average of Total 96 CHAPTER 4. EXPERIMENTAL METHODOLOGY
4.5.3 Statistics in the Results tables
Sum. The sum of all values for the measure.
Min. The minimum value attained.
Max. The maximum value attained.
Avg. The average, or mean value (naturally, over all values, not just the Min and the Max, or the First and the Last).
First. The value for the first session.
Last. The value for the last session.
Diff. The difference: Diff = First − Last
Rate. The compression rate: Last Rate = First
Gain. The inverse of compression: the gain, or betterment attained:
1 First Gain = = Rate Last
Micro avg. Micro-averages (see section on bottom line statistics for a general dis- cussion): Average of Last Micro avg of Rate = Average of First
Average of First Micro avg of Gain = Average of Last
4.5.4 Statistics in the Evolution tables
The values in the Evolution tables are simply the values for each individual session. There are no complex statistics here. 4.6. SUMMARY 97
4.5.5 Alternate terms
The concepts of first and last value, for some measure m e.g. cost, are also called initial and final, or m0 and m1, respectively. To clarify: first = initial = m0; last =
final = m1. (Historical reasons.)
4.6 Summary
In this chapter we have defined the experimental setup, including its simulator, mea- sures, and statistics. Next we we present all configurations that are relevant for the tests under research, and present the results of running them in this setup. 98 CHAPTER 4. EXPERIMENTAL METHODOLOGY Chapter 5
Results
No amount of experimentation can ever prove me right; a single experiment can prove me wrong. Albert Einstein
In this chapter we present and discuss our main results. We mostly show aggregate statistics obtained from the large set of experiments conducted. The statistics in the comparative result tables are a transposition of selected statistics of the involved configurations, mapped as follows:
Success rate = Micro avg of Success rate in Outcomes table
Initial size = Average of First in Size Results table
Final size = Average of Last in Size Results table
Initial size Betterment in size = Micro avg of Gain in Size Results table = Final size Initial cost = Average of First in Cost Results table
Final cost = Average of Last in Cost Results table
Initial cost Betterment in cost = Micro avg of Gain in Cost Results table = Final cost
We remind that our statistics of gain represent a betterment of the results, and does not denote numerical increase of the quality measures. In fact, gain statistics correspond to a numerical decrease of the quality measures, because, for these measures, less
99 100 CHAPTER 5. RESULTS
Table 5.1: Main results. Original structure D.9 Shattered document D.8 Initial size 3.48 6.08 Final size 3.48 3.30 Betterment in size 1.00 1.84 Initial cost 10.30 6.08 Final cost 10.30 3.30 Betterment in cost 1.00 1.84 is better. In rigour, as numbers, they are inverse quality measures. So, the gain statistics invert the numerical relation to present a positive value, i.e. a more is better value.
In general, we locate the most important statistics to the right or to the bottom of tables. Ultimately, we look at Cost gain. A configuration with greater Cost gain— all other things being equal—is the winner. But, of course, it is never the case that all other things are equal, hence the need to have the remaining data reported and analysed as well, for a correct interpretation of the results.
5.1 Main results
Table 5.1 shows our main results: the values of the principal measures, across two con- figurations: original structure (non-adaptive); shattered document (adaptive). This comparation intends to test our main, Shattered Document Hypothesis, which states that the navigation effort of a user in a shattered adaptive hypertext document is reduced with respect to the original hypertext document.
We observe that, confirming our expectations, the shattered document indeed fares significantly better than the original structure on both measures of navigation effort, session size, session cost. We have assessed the statistical significance of the difference using a Wilcoxon tests on the sets of 50 final values, one per each oracle. The p- values obtained are in table 5.2. For a significance level of 0.05, only the alternative hypothesis that the variants are different in size is non-significant.
Note that, naturally, for the original structure, the initial and final values are equal, and therefore the betterment is 1.0 (no betterment), because this configuration, by definition, has no adaptation. 5.1. MAIN RESULTS 101
Table 5.2: P-values for the main results. Alternative hypothesis For final size For final cost Orig.Stru.(D.9) 6= Shat.Doc.(D.8) 0.07939 2.2e-16 Orig.Stru.(D.9) > Shat.Doc.(D.8) 0.03969 2.2e-16
Also note that for the shattered document the cost equals the size, because the page size and number of items in a shattered document configuration are always below the thresholds defined for extra cognitive effort, namely one screen, ten items; the cost per page is always the unit.
Throughout this chapter, for precise reference of the configuration under reporting, we identify the section D.x of the complete configuration definition and detailed experimental results on appendix D.
5.1.1 Session size and success rate
The results in table 5.1 are for a success rate of 100%, i.e. for all sessions finding their oracle regardless of their size or cost.1 Looking in the detailed tables (appendix D) we find that some sessions are way too long or costly for a sensible session i.e. it is reasonable to suppose that, for these sessions, a real user would give up prior to reaching the oracle. Therefore, we also tried restricting the maximum session cost in order to simulate this behaviour. Table 5.3 reports the results for a maximum cost set to six, which is close to the theoretical maximum size of five for the original structure (height of the document tree). The following observations in this section continue to refer to table 5.3.
Note the impact on success rate i.e. the percentage of successful sessions: it drops quite a lot for the original structure, but not much for the shattered document.
Note the final size of shattered document being worse than that of the original structure (3.48 > 3.16). Even in the knowledge, as we are, that session size alone is not a realistic measure of navigation effort for sessions with pages that are long or have many choices—as is the case in the original structure—it should be clarified that this apparent win in session size of the original structure is just a statistical side effect: because the average session size and cost values are over all sessions, including failed
1Actually for a maximum cost of 15. We have found out that this number is just large enough for 100% success for this document. 102 CHAPTER 5. RESULTS
Table 5.3: Results for maximum cost = 6. Original structure D.7 Shattered document D.1 Success rate 68% 94% Initial session size (avg.) 3.16 5.62 Final session size (avg.) 3.16 3.48 Betterment in session size 1.00 1.62 Initial session cost (avg.) 9.15 5.62 Final session cost (avg.) 9.15 3.48 Betterment in session cost 1.00 1.62
ones, they are sensitive to variations of the success rate, which in turn is dependent on the settings for the maximum session size and cost. A high maximum will let more sessions pass with high cost values, therefore increasing the average. A low maximum, naturally, will have the inverse effect, i.e. reduce the average. To clarify: failed sessions also count for aggregate statistics, with the value that exceeded the maximum—which accounts for any averages higher than the maximum.
For the supplemental results, we also use this stringent variant of the configurations (maximum session size = 6).
5.1.2 Evolution
It is interesting to analyse the distribution and evolution of session cost along the progression of sessions. Figure 5.1 charts such data for the shattered document. We look particularly at the moving average. Note how adaptation seems to happen relatively quickly, in the first 150–200 sessions; after that passage the variations seem mostly random oscillations around the final average of 3.48
Also of note is the complete disapearance of the bad sessions of cost much higher than four, shortly after that same passage. Remember that cost equals size in this configuration, because pages are short and contain few links. 5.2. TESTING LINK TYPES 103
Figure 5.1: Session cost evolution for the shattered document D.9
Shattered Document Session Cost evolution (all sessions) 12
10
8
6
4
2
0 1 51 101 151 201 251 301 351 401 451
5 Accumulated avg. Moving avg. (window=50)
For completeness we also include a corresponding chart for the original structure, in figure 5.2. It contains only 50 sessions because for this configuration there is no change: the 50 sessions represent the 50 oracles. Remember that in this configuration session cost is the more realistic measure of navigation effort, because pages are long and contain many links.
Figure 5.2: Session costs and sizes for the original structure D.8
Original Structure session cost and size (all sessions) 25
20 20,03 20,03
17,34 16,35 16,2 15,86 15 14,88 13,87 13,87 13,87 13,87 13,87
12,45 12,45 12,45 12,09 12,35 12,09
10,68 10,54 10,52 10,52 10,52 10,52 10 10,09 8,85 8,85 8,85 8,85 8,51 7,98 7,98 7,98 7,98 7,41 7,41 7,57 7,57 6,93 6,93 6,31 6,31 6,01 6,31 6,06 5,75 5,75 5,53 5 5,09 4,91
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
cost size
5.2 Testing link types
The shattered document design merges all type of links (Next, Child, Refer, Travel) into a single type (Travel). This was our usor hypothesis (supplemental). To verify this hypothesis we compared variants of the shattered document configuration with 104 CHAPTER 5. RESULTS
Table 5.4: Varying the link weights Success rate Initial cost Final cost Gain D.1 nominal 94% 5.62 3.48 1.62 D.10 child links only 89% 5.28 3.32 1.59 D.13 strong structural links 92% 5.70 3.68 1.55 D.14 fully connected graph 5% 6.86 6.70 1.02 D.6 random weakly connected graph 98% 4.10 3.40 1.20
different weights for the different types of link. Table 5.4 shows the results. The nominal configuration D1 gives the same weight of 1 to each link of any type. For the detailed parameters see appendix D.
Variants D.10 and D.13 are the main contrasting variants for this verification. They vary greatly the weights of Child and Next links: D.10 removes Next links altogether (null weight); D.13 increases greatly the weight of both Child and Next links. We observe that neither variant improves statistically significantly upon the result of the nominal configuration D.1; the p-values for the alternatives D.1 6= D.10, D.1 < D.10, D.1 > D.10, D.1 < D.13, D.1 > D.13, D.1 < D.13, with respect to either final or initial cost, are all well above the significance level 0.05
These results seems to indicates that initial, structural links have little or null effect on the adaptation. So we tried removing them entirely. Variants D.14 and D.6 have null structure.
Variant D.14 has every noogramicle connected to every other noogramicle. The results are bad. Only 5% of sessions succeed. Looking in the detailed tables, we find that the sessions that succeed are exactly three and they always succeed. So, there is no adaptation.
Variant D.6 has each noogramicle connected to only one other random noogramicle. This was the surprise in this experiment: the results are good. Like for D.10 and D.13, the results in final cost do not differ statistically significantly from the nominal configuration D.1 But the initial cost is significantly better (p-value 2.773e-08, for alternative 6=). This again suggests that an authored initial structure is not required at all for shattered documents. 5.3. TESTING ADAPTATIVE TECHNIQUES 105
Table 5.5: Markov chain results Success rate Initial cost Final cost Gain D.1 nominal 94% 5.62 3.48 1.62 D.15 Markov chains (1st order) 30% 6.34 6.06 1.05 D.16 Markov chains with heuristic 84% 5.84 4.22 1.38
5.3 Testing adaptative techniques
Recall that our nominal shattered document design effects adaptation with a re- centring operation based on spreading activation. This choice is the concretization of the spreading activation hypothesis (supplemental). To test this hypothesis, we compared against a recentring operation based on the standard technique of first order Markov chains, which has been successfully used in recommendation tasks (Borges & Levene 2008). Table 5.5 shows the results.
Configuration D.16 adds to the Markov chains (1st order) D.15 (which performs terribly) the same heuristic rule of the nominal configuration that excludes already seen noogramicles from the new pages. The results are positive, but statistically significantly worse than the nominal D.1, in final cost.
5.4 Summary
In sum, the experimental results seem to confirm our working hypotheses, that shat- tered document reduces the utilization effort, different link types can be merged, and spreading activation adapts better then Markov chains. In the next chapter we shall analise this reading of the results more critically, including a discussion of its limitations. 106 CHAPTER 5. RESULTS Chapter 6
Conclusions
I belong to those who got shattered. David Fonseca
6.1 Main conclusion
In general we conclude positively, namely, that the experimental results verify our main hypothesis that the navigation effort of a user in a shattered adaptive hypertext document is reduced with respect to the original hypertext document. The results also verified our supplements hypotheses of spreading activation and the usor hypothesis (chapter 1).
To get here we have walked a path of instantiating the abstract concepts of the hypotheses into formal entities: noogramicle, oracle, session size, session cost, travel. The assumptions and decisions that we had to make along the way have been supported by previous work done in the area.
6.2 The pros and cons of simulating
The approach of using simulation (instead of real users) was an important debatable methodological decision. We were atracted to this by several factors. We wanted to try many different designs and configurations, particularly in relation to the novel
107 108 CHAPTER 6. CONCLUSIONS technique of spreading activation to explore data graphs. We ended up doing hundreds of experiments. That would have been impossible, in this programme, if we had required real users. Last but not least, we viewed the design and construction of the simulator as a scientific challenge on its own—and hence a contribution of this thesis.
On the other hand, we were always aware of the fact that a final evaluation can only be done with our proposed approach being used with advantage in a real environ- ment. However, despite our efforts, such a test has been deterred by some practical constraints encountered. We have nevertheless started the construction of such a testbed: the Shattered Document Page, a website with standard and shattered version of popular documents; we would then invite people there, and analyse their utilization. We have studied the manuals of Ada, Sicstus Prolog, LATEX, UNIX man pages, and others for this purpose. We have found that shattering real documents for real users was harder that we initially had thought. The model of the ARM that we used in the experiments is basically a quantitative models of headings and paragraphs. Shattering for real requires dealing individually with every real type of noogramicle (paragraph, figure, table, definition list entry, etc.), and dealing individually with every format used (HTML, XHTML, XML, Texi, etc.) As we said, we have made tentative starts down this path, and already collected some results (appendix A, directory KAUS). So we regard the Shattered Documents Page as a future project, to do perhaps with a team, or collaboratively across the internet.
6.3 Paths not taken
Albeit its limitations, this thesis has been the result of an intense search for a general adaptive hypertext approach. As it is probably the case with every thesis, we end the process more interested in thinks we have not done rather than in what we have explored or achieved. Important paths not taken include the experiments with real users, adaptation alternatives to spreading activation and Markov chains, and a thorough calibration and sensitivity analysis of the simulator. However, curiously enough, the aspect of this thesis that most interests me is the path not taken of researching effective means to represent data (and programs), probably resorting to some kind of hypergraphs (Iordanov 2010), most probably explored via spreading activation. We have collected some notes of this direction in appendix C.3
I have arrived at network models of data, and spreading activation, by thinking about the best ways to represent and explore information. My past experience as 6.4. FUTURE WORK 109
an information systems engineer certainly had something to do with it. Having encountered the limitations of the relational model, and of object orientation, and of classic information retrieval, I have tried to see further. I have tried and ditched many schemes until I settled (for the time being anyway) with the data graph explored by energy propagation. Curiously enough, I imagined this scheme before reading anything about spreading activation, either from brain neurobiology (Anderson 1983) or corresponding bio-inspired cybernetics (Heylighen 1997). Only after I have invented the method was I able to effect successful searchs about the subject, and recognize the relevant works. The information retrieval conundrum discussed in section 2.3.6 is not purely theoretical.
6.4 Future work
More to the point of adaptive hypertext than the general avenue of researching graph databases, we envision the following paths of continuation of the current research.
First and foremost, finish the Shattered Document Page, as a means of collecting real utilization data of real documents, or otherwise conduct an user study with real users. The expected positive confirmation of our adaptive model from this methodology would have the extra benefit of independently validating the simulator, which could then be reliably used in similar research.
Also, we would like to compare across more alternatives to our spreading activation method, other than just the Markov chains method tested. Collaborative filtering, association rules, even other spreading activation algorithms, have proven successful in similar applications, and it would be very interesting to test them in our setup. 110 CHAPTER 6. CONCLUSIONS
Another very important path is the improvement of the shattering operation, which this thesis has left in a very crude state to say the least, with only a statistical, quantitative model of the document.1 Possibilities of improvement include a shat- tering approach based on user queries and usage. User concrete queries would help identify popular snippets of the document. Those snippets would become candidate noogramicles. Adaptation would be done as proposed in this work. Another shattering approach would be based on the capture and analysis of user interaction—mouse clicks, mouse moves, scrolling. Regions of interest would be identified by discovered interaction patterns. Such regions would be candidate noogramicles.
Caminante, son tus huellas el camino, y nada m´as; caminante, no hay camino, se hace camino al andar. Al andar se hace camino, y al volver la vista atr´as se ve la senda que nunca se ha de volver a pisar. Caminante, no hay camino, sino estelas en la mar. Ant´onioMachado
1However cf. appendix A, directory KAUS, for last minute results of shattering real documents. References
Ada Reference Manual Ada Reference Manual, ISO/IEC 8652:200y(E) Ed. 3. — Ada Reference Manual ISO/IEC 8652:1995(E) with Technical Corrigendum 1 and Amendment 1 (Draft 16) : Language and Standard Libraries — Copyright c 1992,1993,1994,1995 Intermetrics, Inc. Copyright c 2000 The MITRE Corporation, Inc. Copyright c 2004, 2005 AXE Consultants Copyright c 2004, 2005 Ada-Europe
Ahn et al. 2005 Investigating Users’ Needs and Behavior for Social Search / Jae-wook Ahn ; Peter Brusilovsky ; Rosta Farzan. — In: PIA 2005 Workshop on New Technologies for Personalized Information Access
Aho & Ullman 1992 Foundations of Computer Science / Alfred V. Aho ; Jeffrey D. Ullman. — W. H. Freeman Computer Science Press, 1992. — http://infolab.stanford.edu/ ullman/focs.html
Alves et al. 2004 Extreme Adaptivity / M´arioAmado Alves ; Al´ıpio Jorge ; Jos´ePaulo Leal. — 348–352 p. — In: Adaptive Hypermedia and Adaptive Web-Based Systems: Third International Conference, AH 2004, Eindhoven, The Netherlands, August 23-26, 2004. Proceedings / Paul De Bra ; Wolfgang Nejdl (eds.) — Springer-Verlag, 2004. — (Lecture Notes in Computer Science ; 3137)
Alves & Jorge 2005 Minibrain : a generic model of spreading activation in computers, and example specialisations / M´arioAmado Alves ; Al´ıpioJorge. — 10 p. — In: Relational Machine Learning : ECML-2005 Workshop on sub-symbolic paradigms for learning in structured domains
111 112 CHAPTER 6. CONCLUSIONS
Anderson 1983 A Spreading Activation Theory of Memory / John R. Anderson. — In: Journal of Verbal Learning and Verbal Behavior 22 (1983).
ARM = Ada Reference Manual
Ashbi 1956 An Introduction to Cybernetics / W. Ross Ashby. — Chapman & Hall, London, 1956. Internet (1999): http://pcp.vub.ac.be/books/IntroCyb.pdf
Auber 2002 Using Strahler numbers for real time visual exploration of huge graphs / David Auber. — 56–69 p. — In: International Conference on Computer Vision and Graphics, 2002. — http://tulip.labri.fr/publi tulip/auberICCVG2002.pdf
Baker & Lehoucq 2007 Constrained Eigenvalue Problems / Christopher G. Baker ; Richard B. Lehoucq. — p. 68–72 — In: CSRI Summer Proceedings 2007
Belo 1991 Epistemologia do Sentido / Fernando Belo. — Funda¸c˜aoCalouste Gulbenkian, 1991.
Berglung 2002 Library Communication Among Programmers Worldwide / Erik Berglund. — Link¨opingStudies in Science and Technology, Dissertation No. 758, Department of Computer and Information Science, Link¨opingsuniversitet, Sweden, Link¨oping,2002. — (Doctoral thesis.)
Bodner & Chignell 2000 Dynamic Hypertext: Querying and Linking / Richard C. Bodner ; Mark H. Chignell. — In: ACM Computing Surveys, Symposium on Hypertext and Hypermedia, 2000.
Bollen 2001 A Cognitive Model of Adaptive Web Design and Navigation. A Shared Knowledge Perspective / Johan Bollen. — Doctoral Thesis, Faculteit Psychologie en Opvoedkunde, Vrije Universiteit Brussel, Belgium, June 2001. — xx, 322 p. 6.4. FUTURE WORK 113
Bollen & Heylighen 1998 A system to restructure hypertext networks into valid user models / Johan Bollen ; Francis Heylighen. — In: The New Review of Hypermedia and Multimedia, 1998.
Borges & Levene 2008 Mining Users’ Web Navigation Patterns and Predicting Their Next Step / Jos´eBorges ; Mark Levene. — In: SIT 2008
Bra et al. 1999 Adaptive Hypermedia: From Systems to Framework / Paul De Bra ; Peter Brusilovsky ; Geert-Jan Houben. — 12 p. — In: ACM Computing Surveys 31(4), December 1999.
Bray et al. 2004 Extensible Markup Language (XML) 1.1 — W3C Recommendation 04 February 2004, edited in place 15 April 2004 / Tim Bray ; Jean Paoli ; C. M. Sperberg-McQueen ; Eve Maler ; Fran¸coisYergeau ; John Cowan. — W3C
Breese et al. 1998 Empirical Analysis of predictive Algorithms for Collaborative Filtering / John S. Breese ; David Heckerman ; Carl Kadie. — 10 p. — In: Proc. Fourteenth Conf. Uncertainty in Artificial Intelligence, Madison, WI, July, 1998. — Morgan Kaufmann Publisher.
Brusilovsky & Mill´an2007 User Models for Adaptive Hypermedia and Adaptive Educational Systems / Peter Brusilovsky ; Eva Mill´an.— 3–53 p. — In: The Adaptive Web — Lecture Notes in Computer Science, 2007, Volume 4321
Buckland & Plaunt 1997 Selecting Libraries, Selecting Documents, Selecting Data. / Michael Buckland ; Christian Plaunt — In: Proceedings of the International Symposium on Research, Development & Practice in Digital Libraries, ISDL 97, Nov. 18-21, 1997, University of Library and Information Science, Tsukuba City, Japan. — http://metadata.sims.berkeley.edu/papers/papers bydate.html
Bush 1945 As We May Think / Vannevar Bush. — 106–107 p. — In: The Atlantic Monthly, July 1945 — http://www.theatlantic.com/doc/194507/bush 114 CHAPTER 6. CONCLUSIONS
Campbell 2000 The ostensive model of developing information-needs / Iain Campbell. — PhD thesis, University of Glasgow, Department of Computing Science, September 2000.
Card et al. 1983 The Psychology of Human-Computer Interaction / S. K. Card ; T. P. Moran ; A. Newell. — Lawrence Erlbaum, 1983.
Ceglowski et al. 2003 Semantic Search of Unstructured Data using Contextual Network Graphs / Maciej Ceglowski ; Aaron Coburn ; John Cuadrado. — [2003 most recent date in references]
Chamberlin et al. 2005 XML Query Use Cases. — W3C Working Draft 15 September 2005 / Don Chamberlin ; Peter Fankhauser ; Daniela Florescu ; Massimo Marchiori ; Jonathan Robie. — W3C
Chang et al. 2006 Bigtable: A Distributed Storage System for Structured Data / Fay Chang ; Jeffrey Dean ; Sanjay Ghemawat ; Wilson C. Hsieh ; Deborah A. Wallach ; Mike Burrows ; Tushar Chandra ; Andrew Fikes ; Robert E. Gruber. — 205–218 p. — In: OSDI ’06: 7th USENIX Symposium on Operating Systems Design and Implementation
Chen 1976 The Entity-Relationship Model—Toward a Unified View of Data / Peter Pin-Shan Chen. — 9–36 p. — In: ACM Transactions on Database Systems, Vol. 1, No. 1. March 1976, Pages 9-36.
Chen & Ford 1998 Modelling User Navigation Behaviours in a Hypermedia-Based Learning System; An Individual Differences Approach / Sherry Y. Chen ; Nigel J. Ford. — 67–78 p. — In: Knowledge Organization 25 (1998) No. 3
Chen & Ng 2004 An algorithmic approach to concept exploration in a large knowledge network (automatic thesaurus construction): Symbolic branch-and-bound vs. Connectionist Hopfield net activation / H. Chen ; D. T. Ng. — 348–369 p. — In: J. ASIS 46, 5 6.4. FUTURE WORK 115
Cockburn et al. 2002 Pushing Back: Evaluating a New Behaviour for the Back and Forward Buttons in Web Browsers / Andy Cockburn ; Bruce Mckenzie ; Michael JasonSmith. — 397–414 p. — In: International Journal of Human-Computer Studies, vol. 57 (2002)
Computational Linguistics 1974– Computational Linguistics / Robert Dale (ed.). The Mit Press. Quarterly (March, June, September, December). 160 pp. per issue. Founded 1974. ISSN 0891-2017
Coombs et al. 1987 Markup Systems and the Future of Scholarly Text Processing / James H. Coombs ; Allen H. Renear ; Steven J. DeRose. — 933—947 p. — In: Communications of the ACM, Volume 30, Issue 11 (November 1987). — ISSN 0001-0782
Crestani 1997 Application of Spreading Activation Techniques in Information Retrieval / F. Crestani. — 453–482 p. — In: Artificial Intelligence Review, 1997, vol. 11
Crestani & Lee 2000 Searching the Web by Constrained Spreading Activation / Fabio Crestani ; Puay Leng Lee. — 585-605 p. — In: Information Processing and Management: an International Journal. — Volume 36 ; Issue 4. — July 2000. — (ISSN 0306-4573)
Czarkowski 2006 A Scrutable Adaptive Hypertext / Marek Czarkowski. — PdD thesis, School of Information Technologies at The University of Sydney, March 2006
DeCandia et al. 2007 Dynamo: Amazon’s Highly Available Key-value Store / Giuseppe DeCandia ; Deniz Hastorun ; Madan Jampani ; Gunavardhan Kakulapati ; Avinash Lakshman ; Alex Pilchin ; Swaminathan Sivasubramanian ; Peter Vosshall ; Werner Vogels. — 205–220 p. — In: SOSP’07, October 14–17, 2007, Stevenson, Washington, USA. Copyright 2007 ACM 978-1-59593-591-5/07/0010 116 CHAPTER 6. CONCLUSIONS
Deerwester et al. 1990 Indexing by Latent Semantic Analysis / Scott Deerwester ; Susan T. Dumais ; George W. Furnas ; Thomas K. Landauer. — 391–407 p. In: : Journal of the American Society for Information Science, 41 (1990)
Dictionary Dictionary. — Version 1.0.2 (1.0.2) — Copyright c 2005 Apple Computer, Inc.
Dominich 2000 Formal Foundation of Information Retrieval / Sandor Dominich. — 23rd Annual International Conference on Research and Development in Information Retrieval of the Association for Computing Machinery Special Interest Group for Information Retrieval ACM SIGIR International Workshop Mathematical Formal Methods in Information Retrieval MF/IR ’00. Association for Computing Machinery ACM, University of Athens, Athens, Greece, July 24-28, 2000.
Dominich 2003 Connectionist interaction information retrieval / Sandor Dominich. — 167–193 p. — In: Information Processing and Management 39 (2003). — Elsevier Science Ltd.
Dunbar 1993 Coevolution of neocortical size, group size and language in humans / R. I. M. Dunbar. — 681–735 p. — In: Behavioral and Brain Sciences (1993) 16 — Cambridge University Press, 1993
Engelbart 1963 A Conceptual Framework for the Augmentation of Man’s Intellect / Doug C. Engelbart. — 1–29 p. — In: Vistas in Information Handling 1 — Spartan Books, 1963
Engelbart & English 1968 A research center for augmenting human intellect / D. C. Engelbart ; W. English. — In: AFIPS Conference Proceedings, 1968 Fall Joint Computer Conference, (San Francisco, CA, Dec).
Ferr´e & Ridoux 2004 Introduction to Logical Information Systems / S. Ferr´e; O. Ridoux. — 383–419 p. — In: Inf. Process. Manage. 40, 3 (January 2004). 6.4. FUTURE WORK 117
Foskett 1996 The Subject Approach to Information / A.C. Foskett. — Facet, 1996. — 472 p. — (ISBN 1-85604-048-8).
Gaber et al. 2005 Mining Data Streams: A Review / Mohamed Medhat Gaber ; Arkady Zaslavsky ; Shonali Krishnaswamy. — In: SIGMOD Record, Vol. 34, No. 2, June 2005
Gentner & Nielsen 1996 Gentner, D., and Nielsen, J.: The Anti-Mac interface, Communications of the ACM 39, 8 (August 1996), 70-82. — http://www.useit.com/papers/anti-mac.html, 2009
Glazier 1996 ”Our words were the form we entered” / Loss Pequeno Glazier. — In: Witz, Volume IV, Number Two, Summer 1996 (Witz 4.2)
Goldberg et al. 1992 Using collaborative filtering to weave an information tapestry / David Goldberg ; David Nichols ; Brian M. Oki ; Douglas Terry. — 61–70 p. — In: Communications of the ACM 35, 12 (December 1992)
Goldberg & Robson 1989 Smalltalk-80 : The Language / Adele Goldberg ; David Robson. — Addison-Wesley, 1989. — xii, 585 p.
Golovchinsky 2002 Going Back in Hypertext / Gene Golovchinsky. — 82-83 p. — In: Proceedings of the thirteenth ACM conference on Hypertext and hypermedia : College Park, Maryland, USA — Year of Publication: 2002 — ISBN:1-58113-477-0
Halasz 1988 Reflections on NoteCards: seven issues for the next generation of hypermedia systems / F. G. Halasz. — In: Communications of the ACM, 31, 7, (Jul), 1988.
Heer et al. 2003 AVID : Supporting the creation of scalable, responsive visualizations / J. Heer [...] — Working paper, www.parc.com, 2003. 118 CHAPTER 6. CONCLUSIONS
Herder 2003 Utility-Based Evaluation of Adaptive Systems / Eelco Herder. — 25–30 p. — In: Proceedings of the Second Workshop on Empirical Evaluation of Adaptive Systems, held at the 9th International Conference on User Modeling UM2003 / S. Weibelzahl ; A. Paramythis (Eds.) — Pittsburgh, 2003.
Heylighen 1997 Bootstrapping knowledge representation : from entailment meshes via semantic nets to learning webs / Francis Heylighen. — In: International Journal of Human-Computer Studies, 1997.
Heylighen & Joslyn 2001 Cybernetics and Second-Order Cybernetics / Francis Heylighen ; Cliff Joslyn. In: Encyclopedia of Physical Science & Technology 3rd ed. / R.A. Meyers (ed.) — Academic Press, New York, 2001.
Huang et al. 2004 Applying Associative Retrieval Techniques to Alleviate the Sparsity Problem in Collaborative Filtering / Zan Huang ; Hsinchun Chen ; Daniel Zeng. — 116–142 p. — In: ACM Transactions on Information Systems. — Vol. 22, No. 1, January 2004.
Iordanov 2010 HyperGraphDB : A Generalized Graph Database / Borislav Iordanov. — 12 p. — In: First Internation Workshop on Graph Database (IWGD 2010)
Kapelonis 2006 Moving Away From WIMP Interfaces One Step at a Time / Kostis Kapelonis. — Internet, 2006.
Keep et al. 1995 The Electronic Labyrinth / Christopher Keep ; Tim McLaughlin ; Robin Parmar. — c 1993-2001 Christopher Keep, Tim McLaughlin, Robin Parmar. — “Originally written in 1993 and adapted for the World-Wide Web in November 1995. It was initially hosted at the University of Alberta, and then at the University of Victoria from August 1997 to July 2000.” — http://www2.iath.virginia.edu/elab/elab.html, consulted 2013
KnowledgeSea http://www.sis.pitt.edu/∼paws/system knowledgesea2.htm (moving to http://adapt2.sis.pitt.edu/wiki) — Consulted 2010. 6.4. FUTURE WORK 119
Kov´acs & Ueno 2006 Recommending in Context: A Spreading Activation Model that is Independent of the Type of Recommender System and Its Contents / Alexander I. Kov´acs; Haruki Ueno. — 10 p. — In: Proc. Workshop on Web Personalisation, Recommender Systems and Intelligent User Interfaces, Dublin, June 20, 2006 / Gulden Uchyigit (ed.)
Lawless et al. 2003 Knowledge, Interest, Recall and Navigation: A Look at Hypertext Processing / Kimberly A. Lawless ; Scott W. Brown ; Robert Mills ; Hayley J. Mayall. — 911–934 p. — In: Journal of Literacy Research September 2003 vol. 35 no. 3
Lebedev 2007 Mandership / Artemy Lebedev. — Translated from Russian by Ilya Grigoriev. — http://www.artlebedev.com/mandership, 1997–2007.
Mancini 2004 Towards Cinematic Hypertext: A Theoretical and Empirical Investigation / Clara Mancini. — PhD Thesis, Tech Report kmi-04-6 March 2004, Knowledge Media Institute, The Open University
Manning et al. 2009 An Introduction to Information Retrieval / Christopher D. Manning ; Prabhakar Raghavan ; Hinrich Sch¨utze.— Cambridge University Press, Cambridge, England, Online edition (c) 2009 Cambridge UP — http://nlp.stanford.edu/IR-book/information-retrieval-book.html
Marsh 1997 Paragram as is Hypertext / William Marsh. — In: Witz, Volume V, Number One, Spring 1997 (Witz 5.1)
Marshall et al. 1994 VIKI: spatial hypertext supporting emergent structure / C. C. Marshall ; F. M. Shipman ; J. H. Coombs. — In: Proceedings of ECHT ’94, (Edinburgh, Scotland, Sep), 1994.
Miller 1956 The Magical Number Seven, Plus or Minus Two : Some Limits on Our Capacity for Processing Information / George A. Miller. — In: The Psychological Review, 1956, vol. 63, pp. 81-97 120 CHAPTER 6. CONCLUSIONS
Mladeni´c & Lavraˆc2003 Data Mining and Decision Support for Business Competitiveness: A European Virtual Entreprise : Results of the Sol-Eu-Net Project : January 2000 – March 2003 / Dunja Mladeni´c; Nada Lavraˆc(Eds.) — Ljubljana : DZS, 2003. — xii, 132 p. — ISBN 8634137589
Nelson 1965 A File Structure for the Complex, the Changing and the Indeterminate. / Ted Nelson. — In: Proceedings of the ACM National Conference, 1965.
Nelson 1999 Way Out of the Box / Theodor Holm Nelson. — 99.10.08 — http://ted.hyperland.com
Nelson 2007a Toward a Deep Electronic Literature: The Generalization of Documents and Media / Theodor Holm Nelson. — Project Xanadu and Oxford Internet Institute, 07.04.19 — http://xanadu.com/XanaduSpace/xuGzn.htm
Nelson 2007b TRANSLITERATURETM : A Humanist Format for Re-Usable Documents and Media / Theodor Holm Nelson. — 07.06.17 — http://transliterature.com
Newman 2003 The structure and function of complex networks / M. E. J. Newman — [Technical report?] Departmentof Physics, Universityof Michigan, AnnArbor, MI 48109, U.S.A. and SantaFeInstitute, 1399HydeParkRoad, SantaFe, NM87501, U.S.A., arXiv:cond-mat/0303516 v1 25 Mar 2003
Nielsen 1993 Usability Engineering / Jakob Nielsen. — San Francisco : Morgan Kaufmann, 1993.
Nielsen 1997 Jakob Nielsen’s Alertbox for October 1, 1997 : How Users Read on the Web. — http://www.useit.com/alertbox/9710a.html
N¨urnberg et al. 1996 Hypermedia operating systems: a new paradigm for computing / Peter J. N¨urnberg ; John J. Leggett ; Erich R. Schneider ; John L. Schnase. — 194-202 p. — In: Proceedings of the seventh ACM conference on Hypertex, 1996. 6.4. FUTURE WORK 121
Olson & Chi 2003 ScentTrails: Integrating Browsing and Searching on the Web / Chris Olston ; Ed H. Chi. — In: ACM Transactions on Computer-Human Interaction, 2003
Page et al. 1999 The PageRank Citation Ranking: Bringing Order to the Web / Lawrence Page ; Sergey Brin ; Rajeev Motwani ; Terry Winograd. — Technical Report. Stanford InfoLab. 1999
Paramythis et al. 2001 A modular approach to the evaluation of Adaptive User Interfaces / A. Paramythis ; A. Totter ; C. Stephanidis. — 9–24 p. — In: Empirical Evaluation of Adaptive Systems. Proceedings of workshop at the Eighth International Conference on User Modeling, UM2001 / S. Weibelzahl, D. N. Chin ; G. Weber (Eds.) — Freiburg, 2001.
Perkowitz & Etzioni 1999 Adaptive Web Sites : Conceptual Cluster Mining / Mike Perkowitz ; Oren Etzioni. — In: Proceedings of the 2nd Workshop on Adaptive Systems and User Modeling on the WWW. / Edited by Peter Brusilovsky and Paul De Bra. — http://wwwis.win.tue.nl/asum99/perkowitz.html
Perkowitz & Etzioni 2000 Towards adaptive Web sites: Conceptual framework and case study / Mike Perkowitz ; Oren Etzioni. — 245–275 p. — In: Artificial Intelligence 118 (2000)
Paul & Elder 2012 Critical Thinking: Tools for Taking Charge of Your Learning & Your Life / Richard Paul ; Linda Elder. — 3rd Edition. — — Prentice Hall, 2012.
Power et al. 2003 Document Structure / Richard Power ; Donia Scott ; Nadjet Bouayad-Agha. — 211–260 p. — In: Computational Linguistics, Volume 29, Issue 2 (June 2003)
Preece 1981 A spreading activation network model for information retrieval / Scott Preece. — PhD thesis, CS Dept., Univ. of Illinois, Urbana, IL. 1981. 122 CHAPTER 6. CONCLUSIONS
Principia Cybernetica Project Welcome to Principia Cybernetica Web . . . the website of the Principia Cybernetica Project / Francis Heylighen ; Cliff Joslyn ; Valentin Turchin (editors). — http://pcp.vub.ac.be, 1993 (created), 2006 (modified)
Raskin 2000 The Humane Interface : new directions for designing interactive systems / Jef Raskin. — Addison-Wesley, 2000. — 233 p.
Rayward 1994 Visions of Xanadu: Paul Otlet (1868-1944) and Hypertext / W. Boyd Rayward. — 235–250 p. — In: JASIS 45 (1994)
Rayward 1999 H.G. Wells’s Idea of a World Brain: A Critical Re-Assessment / Boyd Rayward. — 557–579 p. — In: Journal of the American Society for Information Science 50 (May 15, 1999) — working copy at http://people.lis.uiuc.edu/∼wrayward/Wellss Idea of World Brain.htm
Rayward 2003 Knowledge organization and a new world polity: The rise and fall and rise of the ideas of Paul Otlet. / Boyd W. Rayward — 4–15 p. — In: Transnational Associations / Associations Transnationales, 2003 (Issues 1,2)
Reagle 2005 Wikipedia’s heritage: vision, pragmatics, and happenstance / Joseph M. Reagle Jr. — http://reagle.org/joseph/2005/historical/digital-works.html, consulted 2007.
Rome 2003 Eigensearching the World Wide Web : A Survey of Current Web Retrieval Research / Jayson Rome. — Department of Computer Science, The Graduate School and University Center, City University of New York, January 17, 2003.
Rosenberg 1999 A hypertextuality of arbitrary structure: a writer’s point of view / Jim Rosenberg. — In: Proceedings of the First Workshop on Structural Computing : Technical Report AUE-CS-99-04 / Peter J. N¨urnberg, editor. — (C) 1999 Aalborg University Esbjerg, Computer Science Department, 1999. — http://www.cs.aue.auc.dk/publications/ 6.4. FUTURE WORK 123
Saloj¨arvi et al. 2005 Inferring Relevance from Eye Movements: Feature Extraction / Jarkko Saloj¨arvi; Kai Puolam¨aki; Jaana Simola ; Lauri Kovanen ; Ilpo Kojo ; Samuel Kaski. — 23 p. — 3 March 2005 — PASCAL EU Network of Excellence challenge for inferring relevance from eye movements at http://www.cis.hut.fi/eyechallenge2005/
Santos 2006 O ponto de inflex˜aoOtlet: uma vis˜aosobre as origens da documenta¸c˜aoe o processo de constru¸c˜aodo princ´ıpiomonogr´afico/ Paola De Marco Lopes dos Santos — Diss. de Mestrado apresentada `aEscola de Comunica¸c˜oese Artes da Univ. de S. Paulo, 21/03/2006
Scheir & Lindstaedt 2006 A network model approach to document retrieval taking into account domain knowledge / Peter Scheir ; Stefanie N. Lindstaedt. — 154–158 p — In: Lernen, Wissensentdeckung und Adaptivit¨at,2006.
Schneider 2005 Building the Semantic Web Tower from RDF Straw / Peter F. Patel-Schneider. — 6 p. — In: IJCAI 2005
Schwarzelm¨uller2003 Sistemas Hiperm´ıdiaFacilitando a Assimila¸c˜aoda Informa¸c˜ao/ Anna Friedericka Schwarzelm¨uller.— In: Anais do IV CINFORM 2003. Ciˆencias da Informa¸c˜ao:Fronteiras e Tendˆencias.
Sheskin 2000 Handbook of Parametric and Nonparametric Statistical Procedures : Second Edition / David J. Sheskin. — Boca Raton ; London ; New York ; Washington, D.C. : Chapman & Hall/CRC, 2000.
Simsion & Witt 2005 Data modeling essentials / Graeme C. Simsion ; Graham C. Witt. — Amsterdam ; Boston : Morgan Kaufmann Publishers, 2005.
Sosnovsky & Brusilovsky 2005 Layered Evaluation of Topic-Based Adaptation to Student Knowledge / Sergey Sosnovsky ; Peter Brusilovsky. — 47–56 p. — In: Proceedings of Fourth Workshop on the Evaluation of Adaptive Systems at 10th International User Modeling Conference, UM 2005, July 26, 2005 124 CHAPTER 6. CONCLUSIONS
SWI-Prolog SWI-Prolog (Multi-threaded, 32 bits, Version 5.6.64) — Copyright (c) 1990-2008 University of Amsterdam. — www.swi-prolog.org
Turbo C Turbo C. — A C compiler and programming environment from Borland, the 1980’s.
W3C
W3C World Wide Web Consortium — www.w3.org
Ware 2000 Information Visualization : Perception for design / C. Ware. — Morgan Kaufmann, 2000.
Weel 2006 Digital Text and the Gutenberg Heritage : Work in progress / by Adriaan van der Weel. — http://www.let.leidenuniv.nl/wgbw/research/Gut/, consulted 2006.
Weibelzahl 2005 Problems and Pitfalls in Evaluating Adaptive Systems / Stephan Weibelzahl. — 57–64 p. — In: Adaptable and Adaptive Hypermedia Systems / S. Y. Chen ; G. D. Magoulas (Eds.) — Hershey, PA: IRM Press, 2005. — 285–299 p.
Wells 1938
World Brain / H. C. Wells. — Methuen : London, 1938.
West et al. 2009 Wikispeedia: An Online Game for Inferring Semantic Distances between Concepts / Robert West ; Joelle Pineau ; Doina Precup. — 1598–1603 p. — In: Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence (IJCAI-09)
Wiederhold 1985
Database Design / Gio Wiederhold. — McGraw Hill, 1985.
Williams 2002 The Associative Model of Data / Simon Williams. — Second Edition. — Lazy Software, 2002. — *A BIT FAULTY; CONSIDER REMOVING 6.4. FUTURE WORK 125
Witz 1992–1998 Witz : A Journal of Contemporary Poetics / Christopher Reiner (ed.) Summer 1992 - Spring 1998 http://wings.buffalo.edu/epc/ezines/witz/ 126 CHAPTER 6. CONCLUSIONS Appendix A
Electronic archive
DVD-ROM, or card, or zip file on the web, containing:
Directory Thesis Contains file thesis.pdf, which is this dissertation, plus LATEXsources, images and other files used to typeset it.
Directory Kasim1 Source code and other files related to the first cycle of small scale, exploratory experimentation. Includes modules Mneson and Minibrain. This directory may not be on the DVD-ROM, but should be present on the website.
Directory Kasim2 Source code and other files related to the second cycle of large scale, comparate experimentation.
Directory KAUS Knowledge Atoms User Study. Source code and other files related the prepa- ration of a user study and/or the Shattered Documents Page. Sicstus Prolog Manual fully shattered. This directory may not be on the DVD-ROM, but should be present on the website.
File cv.pdf Curriculum Vitae of the candidate. May be only on the web.
The zip file may be found on (or from) the author’s webpage, http://amado-alves.info. May be protected by password xanadu
The disk may be a CD-ROM (instead of a DVD-ROM), or a flash card, or whatever. The zip file on the web may be a directory instead. These things change. Please adjust.
All files (C) 2013 M´arioAmado Alves.
127 128 APPENDIX A. ELECTRONIC ARCHIVE Appendix B
Program listings
Printed listings of selected source code units that represent the main procedures and algorithms of the experiments realised. All written in the language Ada. The compiler used was GNAT. Some compiler-dependencies may exist. The units are listed alphabetically; when a spec and a body exist for the same unit, the spec is listed first. These units depend on many other units developed that are not listed here. All units are contained in the CD-ROM.
B.1 Package Arm05 Model (spec)
1 with Ada. Strings .Unbounded; 2 3 package Arm05 Model i s 4 −− Quantitative model of the ARM 2005. 5 −− Based on the hypertext version that comes with GNAT. 6 −− The model is the Tree T object returned by function Traverse F i l e s . 7 8 pragma Elaborate Body ; 9 use Ada. Strings .Unbounded; 10 function ”+” (X : String) return Unbounded String renames To Unbounded String ; 11 12 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 13 −− MISCELLANEOUS CONSTANTS −− 14 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 15 16 type Systems i s (Linux, Mac, Windows, Windows With Mac Files ) ; 17 18 Arm05 Dirs : array ( Systems ) of Unbounded String := 19 ( Linux => +”/home/maa/gnat/share/doc/gnat/html” , 20 Mac => +”/ Users / marius /SUPERSEDED gnat−2006−powerpc−apple−darwin7.4.1 − bin /”& 21 ”share/doc/gnat/html” , 22 Windows => +”C: \GNAT\2011 d e p r e c a t e d \ share \ doc \ gnat \html” , 23 −− Windows With Mac Files => +”C: \ Users \Marius\Dropbox\PhD\Kasim2\ Arm05 Files Mac ”);
129 130 APPENDIX B. PROGRAM LISTINGS
24 Windows With Mac Files => +”C: \ Users \ Marius \ Google Drive \PhD\Kasim2\ Arm05 Files Mac ” ) ; 25 26 P r e f i x : array ( Systems ) of Unbounded String := 27 ( Linux => +”arm05” , 28 Mac => +”arm05” , 29 Windows => +”RM” , 30 Windows With Mac Files => +”arm05” ) ; 31 32 System : Systems := Mac; 33 34 function Arm05 Dir return S t r i n g ; 35 36 function B u t t o n s S i z e return Natural ; 37 38 B u t t o n s S i z e s : array ( Systems ) of Natural ; 39 −− initialized on elaboration from the actual file sizes 40 41 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 42 −− RANDOM SAMPLE OF SECTIONS FOR GOOD ESTIMATED AVERAGES −− 43 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 44 45 type Section Number T i s array ( P o s i t i v e range <>) of P o s i t i v e ; 46 type Section Number Ptr T is access all Section Number T; 47 function Sect Nr (A : Positive) return Section Number Ptr T; 48 function Sect Nr (A, B : Positive) return Section Number Ptr T; 49 function Sect Nr (A, B, C : Positive) return Section Number Ptr T; 50 51 type Sample T i s record 52 Section Nr : Section Number Ptr T; 53 F i l e Size : Natural; 54 Nr Of Paragraphs : Natural; 55 Page Length : Natural; 56 end record ; 57 58 Sample : array (1 . . 10) of Sample T; 59 60 A v g F i l e Size :Float; 61 Avg Nr Of Paragraphs : Float; 62 Avg Page Length : Float; −− in s c r e e n s 63 Avg Content Size : Float; −− no b u t t o n s 64 Avg Paragraph Size : Float; 65 Av g Sc reen Size : Float; −− in b y t e s 66 67 procedure Adjust Sample ; 68 procedure Compute Averages ; 69 70 −−−−−−−−−−−−−−−−−−−−−− 71 −− TREE UTILITIES −− 72 −−−−−−−−−−−−−−−−−−−−−− 73 74 type Node T i s record 75 Section : Section Number Ptr T; 76 Parent : Natural; −− index of parent in Tree T o b j e c t 77 −− 0 = no parent 78 F i l e Size : Natural; −− in b y t e s ; B.2. PACKAGE ARM05 MODEL (BODY) 131
79 −− 0 if node is a paragraph 80 −− or the book node (node 1) 81 Paragraph : Natural; −− paragraph number (different from ARM 05) 82 −− 0 if node is a section 83 end record ; 84 85 type Tree T i s array ( P o s i t i v e range <>) of Node T; 86 87 function T r a v e r s e F i l e s ( Max Nodes : Natural) return Tree T; 88 −− Index 1 = root. 89 90 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 91 −− MISCELLANEOUS UTILITIES −− 92 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 93 94 function Qt Ancestors 95 ( Tree : Tree T; 96 Node : Positive; 97 Qt : Natural := 0) 98 return Natural ; 99 100 function Is Paragraph (Node : Node T) return Boolean ; 101 102 function Img (X : Section Number T) return S t r i n g ; 103 function Img (X : Node T) return S t r i n g ; 104 end Arm05 Model ; 105 106 −− Some results: 107 −− the old Mac count yields 4185 nodes om the Mac 108 −− the old Mac count yields 4493 nodes on Windows 109 −− the new Mac count yields 4493 nodes on Windows (same as old) 110 −− the Windows count yields 4484 nodes on Windows, avgFanout=7.911, D=4.855 111 −− after adjustment (see body): 4198 nodes, avgFanout=7.777, D=4.853 112 −− new Linux revision on Linux: 4228 nodes, avgFanout=7.792, D=4.853 113 −− Averages : 114 −− File size : 9.61260E+03 115 −− Content Size : 6.70560E+03 116 −− Nr. of paragraphs: 1.49000E+01 117 −− Paragraph size : 4.50040E+02 118 −− Page length : 2.50000E+00 119 −− Screen size : 3.84504E+03 120 −− New version on Mac (new buttons size): 121 −− 4178 nodes , avgFanout=7.767, D=4.852 122 −− The tree has five levels: 123 −− 1 document 124 −− 2 chapter 125 −− 3 section or paragraph 126 −− 4 subsection or paragraph 127 −− 5 paragraph
B.2 Package Arm05 Model (body)
1 with Ada. Characters .Handling; use Ada. Characters .Handling; 2 with Ada. Environment Variables ; 132 APPENDIX B. PROGRAM LISTINGS
3 with Ada . IO Exceptions ; 4 with Ada . Text IO; use ada . t e x t i o ; 5 with Ada. Directories ; 6 with Ada. Strings .Unbounded; 7 8 package body Arm05 Model i s 9 10 function Arm05 Dir return S t r i n g i s 11 begin 12 return To String (Arm05 Dirs (System)); 13 end ; 14 15 function B u t t o n s S i z e return Natural i s 16 begin 17 return B u t t o n s Sizes (System); 18 end ; 19 20 function Sect Nr(A: Positive) return Section Number Ptr T i s 21 begin return new Section Number T ’(1=>A); end ; 22 function Sect Nr(A,B: Positive) return Section Number Ptr T i s 23 begin return new Section Number T ’(1=>A,2=>B); end ; 24 function Sect Nr(A,B,C: Positive) return Section Number Ptr T i s 25 begin return new Section Number T ’(1=>A,2=>B,3=>C); end ; 26 27 procedure Compute Averages i s 28 procedure Add To ( Var : in out Float; Val : Natural) i s 29 begin Var := Var + Float (Val); end ; 30 procedure Adjust (Var : in out Float ) i s 31 begin Var := Var / Float (Sample’Length); end ; 32 begin 33 for I in Sample ’ Range loop 34 Add To ( A v g F i l e Size , Sample(I).File S i z e ) ; 35 Add To ( Avg Page Length , Sample(I).Page Length ) ; 36 Add To ( Avg Nr Of Paragraphs , Sample(I).Nr Of Paragraphs) ; 37 Add To ( Avg Content Size , Sample(I).File S i z e − B u t t o n s S i z e ) ; 38 end loop ; 39 Adjust ( A v g F i l e S i z e ) ; 40 Adjust ( Avg Page Length ) ; 41 Adjust ( Avg Nr Of Paragraphs) ; 42 Adjust ( Avg Content Size ) ; 43 Avg Paragraph Size := Avg Content Size / Avg Nr Of Paragraphs ; 44 Av g Sc reen Si ze := A v g F i l e S i z e / Avg Page Length ; 45 end ; 46 47 function T r a v e r s e F i l e s ( Max Nodes : Natural) return Tree T i s separate ; 48 49 function Img (X : Natural) return S t r i n g i s 50 Z : String := Natural’Image (X); 51 begin 52 return Z (2 .. Z’Last); 53 end ; 54 55 function Img (X : Section Number T) return S t r i n g i s 56 use Ada. Strings .Unbounded; 57 U : Unbounded String ; 58 begin B.2. PACKAGE ARM05 MODEL (BODY) 133
59 for I in X’ Range loop 60 i f I > X’ F i r s t then Append (U, ”.”); end i f ; 61 Append (U, Img (X(I))); 62 end loop ; 63 return To String (U) ; 64 end ; 65 66 function Img (X : Node T) return S t r i n g i s 67 use Ada. Strings .Unbounded; 68 U : Unbounded String := To Unbounded String (Img (X.Section. a l l )); 69 begin 70 i f X. Paragraph > 0 then 71 Append (U, ”(” & Img (X.Paragraph) & ”)”); 72 end i f ; 73 return To String (U) ; 74 end ; 75 76 function File Name (Number : Section Number T) return S t r i n g i s 77 use Ada. Strings .Unbounded; 78 U : Unbounded String := Prefix (System); 79 begin 80 for I in Number ’ Range loop 81 Append (U, ”−” & Img (Number (I))); 82 end loop ; 83 return Ada. Directories .Compose 84 (Containing D i r e c t o r y => Arm05 Dir , 85 Name => To String (U) , 86 Extension => ”html” ) ; 87 end ; 88 89 procedure Adjust Sample i s 90 begin 91 for I in Sample ’ Range loop 92 −−put(sample(i). file s i z e ’ img ) ; 93 Sample (I).File Size := Natural 94 (Ada. Directories .Size 95 ( File Name (Sample (I).Section Nr . a l l ))); 96 −−put(sample(i). file size ’img&”,”); 97 end loop ; 98 end ; 99 100 function Qt Ancestors 101 ( Tree : Tree T; 102 Node : Positive; 103 Qt : Natural := 0) 104 return Natural 105 i s 106 begin 107 i f Tree(Node).Parent = 0 then return Qt ; 108 else 109 return Qt Ancestors(Tree, Tree(Node).Parent, Qt + 1); 110 end i f ; 111 end ; 112 113 function Is Paragraph (Node : Node T) return Boolean i s 114 begin 134 APPENDIX B. PROGRAM LISTINGS
115 return Node. Paragraph > 0 ; 116 end ; 117 118 begin 119 −−put(”init arm05 model...”) ; 120 i f Ada. Environment Variables.Exists (”windir”) then 121 System := Windows With Mac Files ; 122 end i f ; 123 124 for I in Systems loop 125 declare 126 Filename : String := 127 ” arm05 buttons ” & To Lower (Systems’Image (I)) & ”.html”; 128 begin 129 B u t t o n s Sizes (I) := Natural (Ada.Directories.Size (Filename)); 130 exception 131 when Ada . IO Exceptions .Name Error => 132 Ada . Text IO . Put Line (”Arm05 Model warning: file ” & Filename & ” not found ” ) ; 133 end ; 134 end loop ; 135 136 Sample := 137 −− File Nr. Pg. Diff. pg. len. 138 −− size prgr. len. Mac / Windows 139 ( 1 => ( Sect Nr(1,2) , 5832, 11, 2), 140 2 => ( Sect Nr(2,1) , 15892, 30, 4), 141 3 => ( Sect Nr(3,7) , 18538, 38, 4), −− 4 / 5 142 4 => ( Sect Nr(4,1,4) , 11547, 16, 3), 143 5 => ( Sect Nr(4,9,1) , 5517, 8, 2), 144 6 => ( Sect Nr(8,4) , 8936, 17, 2), −− 2 / 3 145 7 => ( Sect Nr(8,5,5) , 5743, 7, 2), 146 8 => ( Sect Nr(9,7,3) , 5860, 6, 2), 147 9 => ( Sect Nr(11,6) , 7361, 7, 2), 148 10 => ( Sect Nr(13,5,3), 6258, 9, 2)); 149 −− Nr. of paragraphs and page length collected manually 150 −− The file sizes above were collected on the Mac; 151 −− better adjust automatically , given that there seems 152 −− to be differences to Windows. 153 −− Procedure Adjust Sample does that. 154 −− And better rerun everything after elaboration , 155 −− because maybe Ada.Directories does not work well during 156 −− elaboration? No, it seems no need. 157 Adjust Sample ; 158 Compute Averages ; 159 −−p u t line(”done.”); 160 end ;
B.3 Procedure Arm05 Model.Get Info (body only)
1 with Ada. Command Line ; 2 with Ada . Text IO; use Ada . Text IO; 3 4 procedure Arm05 Model . G e t I n f o i s B.3. PROCEDURE ARM05 MODEL.GET INFO (BODY ONLY) 135
5 type Real T i s delta 0 . 0 1 range 0 . 0 0 . . 1 000 000 . 0 0 ; 6 function Img (X: Natural) return S t r i n g i s 7 Z : String := Natural’Image (X); 8 begin 9 return Z (2 .. Z’Last); 10 end ; 11 function Img (X: Real T) return S t r i n g i s 12 Z : String := Real T’Image (X); 13 begin 14 return Z (2 .. Z’Last); 15 end ; 16 N : Natural ; 17 begin 18 begin 19 N := Natural ’Value (Ada.Command Line.Argument (1)); 20 exception 21 when others => N := 5000; 22 end ; 23 24 Put Line (”System => ” & Systems’Image (System)); 25 Put Line ( ” Dir => ” & To String (Arm05 Dirs (System))); 26 27 Put Line (”Traversing files for a maximum of” & N’Img & ” nodes...”); 28 declare 29 Tree : Tree T := Traverse F i l e s (N) ; 30 begin 31 Put Line (”Tree size (qt. of nodes) => ” & Tree’Length’img); 32 Put Line (”Quantity of ancestors (qt of ancestors => qt of nodes, paragraphs with such qt of ancestors)”); 33 declare 34 A : Natural ; 35 Qt : array (0 . . 6) of Natural := ( others => 0) ; 36 Qt P: array (0 . . 6) of Natural := ( others => 0) ; 37 function Qt H (I : Integer) return Natural i s 38 begin 39 return Qt ( I ) − Qt P(I); 40 end ; 41 42 Tot : Natural := 0; 43 Tot P : Natural := 0; 44 function Tot H return Natural i s 45 begin 46 return Tot − Tot P; 47 end ; 48 49 procedure Inc (X : in out Natural ) i s begin X := X + 1 ; end ; 50 51 function ImgP (X : Natural) return S t r i n g i s 52 begin 53 return Img (X) & ” (” & Img(Real T ( 1 0 0 . 0 ∗ Float(X) / Float(Tree’Length)) ) & ”%)” ; 54 end ; 55 56 D : Natural ; 57 Avg D : Real T; 58 begin 136 APPENDIX B. PROGRAM LISTINGS
59 for I in Tree ’ Range loop 60 A := Qt Ancestors (Tree, I); 61 Inc (Qt (A)); 62 Inc ( Tot ) ; 63 i f Is Paragraph (Tree (I)) then 64 Inc ( Qt P(A)); 65 Inc ( Tot P); 66 end i f ; 67 end loop ; 68 pragma Assert (Tot = Tree’Length); 69 for I in Qt ’ Range loop 70 Put Line (Img (I) & ” => ” & Img (Qt (I))); 71 end loop ; 72 New Line ; 73 Put Line (”Level & Nodes & Headings & Paragraphs & Fanout & H−Fanout \\”); 74 for I in Qt’First .. Qt’Last − 2 loop 75 Put (Img(I + 1) & ” & ” & 76 ImgP(Qt(I)) & ” & ” & 77 ImgP( Qt H(I))&”&”& 78 ImgP( Qt P(I))&”&”); 79 i f Qt H(I) > 0 then 80 Put 81 ( Img ( Real T(Float(Qt (I + 1)) / Float(Qt H(I)))) 82 &”&”& 83 Img ( Real T ( Float ( Qt H (I + 1)) / Float(Qt H(I))))); 84 else 85 Put (”0 & 0”); 86 end i f ; 87 Put Line ( ” \\”); 88 end loop ; 89 Put Line (”Total & ” & 90 Img (Tot) & ” & ” & 91 ImgP ( Tot H)&”&”& 92 ImgP ( Tot P)&”&”& 93 Img ( Real T(Tot) / Real T ( Tot H))&”&”& 94 ” \\”); 95 96 D := 0 ; 97 for I in Qt ’ Range loop 98 exit when Qt ( I ) = 0 ; 99 D := D + I ∗ Qt ( I ) ; 100 end loop ; 101 Avg D := Real T (D) / Real T ( Tot ) ; 102 Put Line (”Average distance from the root = ” & Img (Avg D)); 103 D := 0 ; 104 for I in Qt ’ Range loop 105 exit when Qt ( I ) = 0 ; 106 D := D + ( I + 1) ∗ Qt ( I ) ; 107 end loop ; 108 Avg D := Real T (D) / Real T ( Tot ) ; 109 Put Line (”Average session size from the root = ” & Img (Avg D)); 110 end ; 111 112 Put Line (”Number of edges ”); 113 declare 114 Qt : Natural := 0; B.4. PACKAGE KASIM2 (SPEC) 137
115 procedure Inc (X : in out Natural ) i s begin X := X + 1 ; end ; 116 begin 117 for I in Tree ’ Range loop 118 i f Tree (I).Parent > 0 then 119 Inc (Qt) ; 120 end i f ; 121 end loop ; 122 Put Line (Img(Qt)); 123 end ; 124 125 New Line ; 126 Put (”Verifying that each paragraph node has a file parent...”); 127 for I in Tree ’ Range loop 128 i f Tree (I).Paragraph > 0 then 129 i f Tree (Tree(I).Parent).File S i z e = 0 then 130 Put Line (”Paragraph node ” & Img(I) & ” does not have file parent.”); 131 end i f ; 132 end i f ; 133 end loop ; 134 Put Line (”done.”); 135 end ; 136 137 Adjust Sample ; 138 139 New Line ; 140 Put Line (”Averages:”); 141 Put Line(” File size :”&Avg F i l e S i z e ’ img ) ; 142 Put Line (” Content Size : ” & Avg Content Size ’ img ) ; 143 Put Line (” Nr. of paragraphs:” & Avg Nr Of Paragraphs ’img) ; 144 Put Line (” Paragraph size :” & Avg Paragraph Size ’ img ) ; 145 Put Line (” Page length :”&Avg Page Length ’img); 146 Put Line (” Screen size :”&Avg Screen Size ’ img ) ; 147 148 New Line ; 149 Put Line (”Buttons sizes:”); 150 for I in Systems loop 151 Put Line (Systems’Image(I) & ”=>” & Natural ’Image(Buttons Sizes(I))); 152 end loop ; 153 154 New Line ; 155 Put Line (”Sample Nr. & Section & Content size & ” & 156 ”Nr. of paragraphs & Page length \\”); 157 for I in Sample ’ Range loop 158 Put Line (Positive’Image (I) & ” & ” & 159 Img (Sample(I).Section Nr . a l l ) & ” &” & 160 Natural ’Image (Sample(I). File S i z e − B u t t o n s S i z e ) & ” &” & 161 Natural ’Image (Sample(I).Nr Of Paragraphs) & ” &” & 162 Natural ’Image (Sample(I).Page Length ) & ” \\”); 163 end loop ; 164 end ;
B.4 Package Kasim2 (spec)
1 with Ada. Calendar ; 138 APPENDIX B. PROGRAM LISTINGS
2 with Ada. Containers . Generic A r r a y S o r t ; 3 with Ada. Containers .Ordered Maps ; 4 with Ada. Containers . Ordered Sets ; 5 with Ada.Containers.Vectors; 6 with Ada. Strings .Unbounded; 7 with Ada . Text IO; 8 with Ada . Wide Text IO; 9 with Ada. Unchecked Deallocation ; 10 11 −−with Large Real Arrays ; 12 −− On some platforms the standard linear algebra library 13 −− based on BLAS/LAPACK seems to crash upon large arrays. 14 −− When that happens we switch to Large Real Arrays (a workaround). 15 with Ada.Numerics. Generic Real Array s ; 16 17 package Kasim2 i s 18 −− Knowledge Atoms Simulator 2. 19 −− Support for large scale experiments. 20 21 use Ada. Calendar ; 22 use Ada. Strings .Unbounded; 23 24 −−−−−−−−−−−−−−−−−−−−−−−−− 25 −− REAL NUMBER TYPES −− 26 −−−−−−−−−−−−−−−−−−−−−−−−− 27 28 type Real T i s digits 6 ; 29 −− General purpose floating point type. 30 −− Theoretically , reducing the number of digits will reduce 31 −− the memory footprint of large matrices; in practice , with GNAT, 32 −− this effect does not seem to take place. 33 34 subtype Fraction T i s Real T range 0 . 0 . . 1 . 0 ; 35 −− Mainly used for probabilities. 36 37 package Real Arrays i s new Ada.Numerics. Generic Real Arrays (Real T); 38 use Real Arrays ; 39 40 type Real Matrix Ptr T is access all Real Matrix ; 41 type Real Vector Ptr T is access all Real Vector ; 42 43 procedure Free i s new Ada. Unchecked Deallocation (Real Matrix , Real Matrix Ptr T); 44 45 package Real IO i s new Ada . Text IO . Float IO ( Real T); 46 package Wide Real IO i s new Ada . Wide Text IO . Float IO ( Real T); 47 48 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 49 −− ATOM (NOOGRAMICLE) AND ORACLE TYPES −− 50 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 51 52 subtype Atom T i s P o s i t i v e ; 53 −− In this software we use the word ”atom” instead of ”noogramicle” 54 −− for the sake of brevity. 55 56 type Atom Array T i s array ( P o s i t i v e range <>) of Atom T; 57 B.4. PACKAGE KASIM2 (SPEC) 139
58 package Atom Vectors i s new Ada.Containers.Vectors (Positive , Atom T); 59 subtype Atom Vector T i s Atom Vectors.Vector; 60 61 package Atom Sets i s new Ada. Containers . Ordered Sets (Atom T); 62 subtype Atom Set T i s Atom Sets . Set ; 63 64 −−−−−−−−−−−−−−−−−− 65 −− PAGE TYPES −− 66 −−−−−−−−−−−−−−−−−− 67 68 type Page Item T i s record 69 Atom : Atom T; 70 Rank : Real T; 71 end record ; 72 73 type Page Item Array T i s array ( P o s i t i v e range <>) of Page Item T; 74 75 package Page Item Vectors i s new Ada.Containers.Vectors (Positive , Page Item T); 76 subtype Page Item Vector T i s Page Item Vectors.Vector; 77 78 function Higher Rank (Left, Right : Page Item T) return Boolean ; 79 80 procedure Sort By Descending Rank i s 81 new Ada. Containers . Generic A r r a y S o r t 82 ( Index Type => P o s i t i v e , 83 Element Type => Page Item T, 84 Array Type => Page Item Array T, 85 ”<” => Higher Rank ) ; 86 87 type Page T i s record 88 Items : Page Item Vector T; 89 Qt Expanded : Natural; 90 Cost : Real T := 1 . 0 ; 91 end record ; 92 93 Empty Page : Page T; 94 95 Undefined Time : constant Time := Time Of (2399, 1, 1, 0.0); 96 Undefined Duration : constant Duration := Duration ’Last; 97 98 type Timed Page T i s record 99 Content : Page T; 100 Requested : Time := Undefined Time ; 101 Served : Time := Undefined Time ; 102 Clicked : Time := Undefined Time ; 103 Choice : Atom T; 104 Final : Boolean := False; 105 end record ; 106 107 Empty Timed Page : Timed Page T; 108 109 type Timed Page Array T i s array ( P o s i t i v e range <>) of Timed Page T; 110 111 package Timed Page Vectors i s 112 new Ada.Containers.Vectors (Positive , Timed Page T); 113 subtype Timed Page Vector T i s Timed Page Vectors.Vector; 140 APPENDIX B. PROGRAM LISTINGS
114 115 type Timed Page Status T i s (Empty, Requested, Served, Clicked , Final , Anomalous); 116 subtype Non Empty Status T i s Timed Page Status T range Served .. Final; 117 function Status (Page : Timed Page T) return Timed Page Status T; 118 119 −−−−−−−−−−−−−−−−−−−−− 120 −− SESSION TYPES −− 121 −−−−−−−−−−−−−−−−−−−−− 122 123 type Session Outcome T i s (Success , Timeout, Sizeout , Costout); 124 125 subtype Success T i s Session Outcome T range Success .. Success; 126 subtype F ai l ur e T i s Session Outcome T range Timeout .. Costout; 127 128 type Session T i s record 129 Oracle : Atom T; 130 Pages : Timed Page Vector T; 131 Computed : Boolean := False; 132 Outcome : Session Outcome T; 133 Duratio : Duration; 134 Size : Natural; 135 Cost : Real T; 136 end record ; 137 138 Empty Session : Session T; 139 140 type Session Array T i s array ( P o s i t i v e range <>) of Session T; 141 N u l l S e s s i o n A r r a y : constant Session Array T (1 . . 0) := ( others => Empty Session ); 142 143 package S e s s i o n V e c t o r s i s new Ada.Containers.Vectors (Positive , Session T); 144 subtype S e s s i o n V e c t o r T i s S e s s i o n Vectors .Vector; 145 146 −−−−−−−−−−−−−−−−−−−− 147 −− RESULT TYPES −− 148 −−−−−−−−−−−−−−−−−−−− 149 150 type Quality Measure T i s (Duratio, Size , Cost); 151 type Aggregate T i s (Sum, Min, Max); 152 type Quality Array T i s array ( Quality Measure T , Aggregate T) of Real T; 153 type Quantity Array T i s array ( Session Outcome T) of Natural ; 154 155 package Index Vectors i s new Ada.Containers.Vectors (Positive , Positive); 156 subtype Index Vector T i s Index Vectors.Vector; 157 158 type Result T is tagged record 159 Oracle : Atom T; 160 Sessions : Index Vector T; 161 Quality : Quality Array T := ( others => ( others => 0 . 0 ) ) ; 162 Quantity : Quantity Array T := ( others => 0) ; 163 end record ; 164 165 package Result Maps i s 166 new Ada. Containers .Ordered Maps 167 ( Key Type => Atom T, 168 Element Type => Result T); B.4. PACKAGE KASIM2 (SPEC) 141
169 subtype Result Map T i s Result Maps .Map; 170 171 −−−−−−−−−−−−−−−−−−−−−−−− 172 −− EXPERIMENT TYPES −− 173 −−−−−−−−−−−−−−−−−−−−−−−− 174 175 type S t a r t F u n c t i o n P t r T i s 176 access function return Page T; 177 178 type Choose Function Ptr T i s 179 access function ( Page : Page T) return Atom T; 180 181 type Recentre Function Ptr T i s 182 access function (Atom : Atom T) return Page T; 183 184 type Update Travels T i s 185 ( False , 186 Central To Central , 187 Randomly , 188 Page To Central , 189 A l l P r e v i o u s C e n t r a l T o C e n t r a l , 190 A ll P re v io us P ag e T o C en t ra l , 191 Central To Page , 192 Page To Page ) ; 193 194 type Experiment T i s record 195 Name : Unbounded String ; −−−−−−−−−−−−− input 196 Travels : Real Matrix Ptr T; −−−−−−−−− input/output 197 Start : Start F u n c t i o n P t r T; −−−−−−−− input 198 Choose : Choose Function Ptr T; −−−−−− input 199 Recentre : Recentre Function Ptr T; −− input 200 Update Travels : Update Travels T; −−− input with default 201 Sessions : Session V e c t o r T; −−−−−−−−− output 202 Oracles : Atom Set T; −−−−−−−−−−−−−−−− output 203 Session : Session T; −−−−−−−−−−−−−−−−− temporary 204 Page : Timed Page T; −−−−−−−−−−−−−−−−− temporary 205 Cost : Real T; −−−−−−−−−−−−−−−−−−−−−−− temporary 206 Choice : Atom T; −−−−−−−−−−−−−−−−−−−−− temporary 207 Seen : Atom Set T; −−−−−−−−−−−−−−−−−−− temporary 208 Page Size : Natural; −−−−−−−−−−−−−−−−− input with default 209 Qt Expanded : Natural; −−−−−−−−−−−−−−− input with default 210 Max Duration : Duration; −−−−−−−−−−−−− input w. default (per sess.) 211 Max Qt Pages : Natural; −−−−−−−−−−−−−− input w. default (per sess.) 212 Max Cost : Real T; −−−−−−−−−−−−−−−−−−− input w. default (per sess.) 213 Colsize : Ada.Text IO . Count ; −−−−−−−−− a u x i l i a r y 214 Col : Ada.Text IO . Count ; −−−−−−−−−−−−− a u x i l i a r y 215 Per Oracle : Result Map T; −−−−−−−−−−− auxiliary (results) 216 Total : Result T; −−−−−−−−−−−−−−−−−−−− auxiliary (results) 217 end record ; 218 219 −−−−−−−−−−−−−−−−−−−−−−−−−− 220 −− CURRENT EXPERIMENT −− 221 −−−−−−−−−−−−−−−−−−−−−−−−−− 222 223 Cur : Experiment T; 224 142 APPENDIX B. PROGRAM LISTINGS
225 procedure Save Cur ; −− to be implemented 226 procedure Load Cur ; −− to be implemented 227 228 −−−−−−−−−−−−−−−−−−−−−−−−−−−−− 229 −− SIMULATOR SUBPROGRAMS −− 230 −−−−−−−−−−−−−−−−−−−−−−−−−−−−− 231 232 procedure Setup Experiment 233 (Name : String; 234 Travels : Real Matrix Ptr T; 235 Start : Start F u n c t i o n P t r T; 236 Choose : Choose Function Ptr T; 237 Recentre : Recentre Function Ptr T; 238 Page Size : Natural := 10; 239 Qt Expanded : Natural := 3; 240 Update Travels : Update Travels T := Central To Central ; 241 Max Duration : Duration := 10.0; 242 Max Qt Pages : Natural := 10; 243 Max Cost : Real T := Real T ’ Last ) ; 244 −− The maxima are per session. 245 246 procedure S i m u l a t e Session (Oracle : Atom T); 247 248 procedure Compute Results ; −− compute Total, Per Oracle 249 250 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 251 −− MISCELLANEOUS UTILITIES −− 252 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 253 254 function Get (S : Session T; Q : Quality Measure T) return Real T; 255 256 function To Vector (X : Page Item Array T) return Page Item Vector T; 257 function To Array (X : Page Item Vector T) return Page Item Array T; 258 259 procedure Inc (X : in out Integer; Y : Integer := 1); 260 procedure Inc (X : in out Real T ; Y : Real T := 1 . 0 ) ; 261 262 procedure Save (Sessions : Session V e c t o r T; Filename : String); 263 procedure Load (Sessions : out S e s s i o n V e c t o r T; Filename : String); 264 265 function I s I n 266 (Atom : Atom T; 267 Items : Page Item Vector T; 268 First, Last : Positive) 269 return Boolean ; 270 −− Atom is in Items (First .. Last), 271 −− if Items were an array. 272 273 function I s I n 274 (Atom : Atom T; 275 Items : Page Item Vector T; 276 First : Positive) 277 return Boolean ; 278 −− Equivalent to Is In (..., First, Items’Last). 279 280 function I s I n B.5. PACKAGE KASIM2 (BODY) 143
281 (Atom : Atom T; 282 Items : Page Item Vector T) 283 return Boolean ; 284 −− Equivalent to Is In (..., Items’First). 285 286 function Count Non Null (X : Real Vector ) return Natural ; 287 288 function To Set (X : Page Item Vector T) return Atom Set T; 289 end ;
B.5 Package Kasim2 (body)
1 with Ada. Calendar .Formatting; 2 with Ada. Characters .Conversions; 3 with Ada.Numerics. Discrete Random ; 4 with Ada. Streams .Stream IO; 5 with Ada . Text IO; 6 with Ada . Wide Text IO; 7 8 package body Kasim2 i s 9 use Ada. Containers ; 10 11 −−−−−−−−−−−−−−−−−−−−−−−− 12 −− SETUP EXPERIMENT −− 13 −−−−−−−−−−−−−−−−−−−−−−−− 14 15 procedure Setup Experiment 16 (Name : String; 17 Travels : Real Matrix Ptr T; 18 Start : Start F u n c t i o n P t r T; 19 Choose : Choose Function Ptr T; 20 Recentre : Recentre Function Ptr T; 21 Page Size : Natural := 10; 22 Qt Expanded : Natural := 3; 23 Update Travels : Update Travels T := Central To Central ; 24 Max Duration : Duration := 10.0; 25 Max Qt Pages : Natural := 10; 26 Max Cost : Real T := Real T ’ Last ) 27 i s 28 use Ada. Strings .Unbounded; 29 begin 30 Cur.Name := To Unbounded String (Name); 31 Cur.Travels := Travels; 32 Cur.Start := Start; 33 Cur.Choose := Choose; 34 Cur.Recentre := Recentre; 35 Cur . Update Travels := Update Travels ; 36 Cur.Sessions.Clear; 37 Cur.Oracles.Clear; 38 Cur . Page Size := Page Size ; 39 Cur . Qt Expanded := Qt Expanded ; 40 Cur . Max Duration := Max Duration ; 41 Cur . Max Qt Pages := Max Qt Pages ; 42 Cur . Max Cost := Max Cost ; 144 APPENDIX B. PROGRAM LISTINGS
43 Cur.Seen.Clear; 44 end ; 45 46 −−−−−−−−−−−−−−−−−−−−−−−− 47 −− SIMULATE SESSION −− 48 −−−−−−−−−−−−−−−−−−−−−−−− 49 50 function Expanded Atoms (Page : Page T) return Atom Array T i s 51 Y : Atom Array T (1 .. Page.Qt Expanded ) ; 52 A : Page Item Array T := To Array (Page.Items); 53 begin 54 for I in Y’ Range loop 55 Y (I) := A (I).Atom; 56 end loop ; 57 return Y; 58 end ; 59 60 function I s In (Atom : Atom T; Atoms : Atom Array T) return Boolean i s 61 begin 62 for I in Atoms ’ Range loop 63 i f Atom = Atoms (I) then return True ; end i f ; 64 end loop ; 65 return Fa ls e ; 66 end ; 67 68 function Random Atom return Atom T i s 69 subtype Atoms T i s P o s i t i v e range 1 .. Cur.Travels ’Length(1); 70 package Random Atoms i s 71 new Ada.Numerics. Discrete Random (Atoms T); 72 Gen : Random Atoms. Generator ; 73 begin 74 Random Atoms.Reset (Gen); 75 return Random Atoms.Random (Gen) ; 76 end ; 77 78 procedure Increment Travels (From : Page T ; To : Atom T) i s 79 procedure Increment (C : Page Item Vectors.Cursor) i s 80 A : Atom T := Page Item Vectors .Element (C).Atom; 81 begin 82 i f A /= To then 83 Inc (Cur.Travels (A, To)); 84 end i f ; 85 end ; 86 begin 87 From.Items. Iterate (Increment ’Unrestricted A c c e s s ) ; 88 end ; 89 90 procedure Increment Travels (From : Atom T ; To : Page T) i s 91 procedure Increment (C : Page Item Vectors.Cursor) i s 92 A : Atom T := Page Item Vectors .Element (C).Atom; 93 begin 94 i f A /= From then 95 Inc (Cur.Travels (From, A)); 96 end i f ; 97 end ; 98 begin B.5. PACKAGE KASIM2 (BODY) 145
99 To.Items. Iterate (Increment ’Unrestricted A c c e s s ) ; 100 end ; 101 102 procedure Increment Travels (From, To : Page T) i s 103 procedure Increment (C : Page Item Vectors.Cursor) i s 104 A : Atom T := Page Item Vectors .Element (C).Atom; 105 begin 106 Increment Travels (A, To); 107 end ; 108 begin 109 From.Items. Iterate (Increment ’Unrestricted A c c e s s ) ; 110 end ; 111 112 procedure S i m u l a t e Session (Oracle : Atom T) 113 i s 114 Start Time : Time ; 115 Prev Page : Timed Page T; 116 begin 117 Start Time := Clock; 118 Cur.Session.Oracle := Oracle; 119 Cur. Session .Pages.Clear; 120 Cur.Seen.Clear; 121 Cur.Cost := 0.0; 122 Cur.Page := Empty Timed Page ; 123 Cur.Page.Requested := Clock; 124 select 125 delay Cur . Max Duration ; 126 Cur. Session .Outcome := Timeout; 127 then abort 128 Cur.Page.Content := Cur.Start. a l l ; 129 Cur.Page.Served := Clock; 130 loop 131 Cur.Cost := Cur.Cost + Cur.Page.Content.Cost; 132 Cur.Seen.Union (To Set (Cur.Page.Content.Items)); 133 i f I s In (Oracle , Expanded Atoms (Cur.Page.Content)) then 134 Cur.Session.Outcome := Success; 135 Cur.Page.Final := True; 136 exit ; 137 e l s i f Natural(Cur. Session .Pages.Length) = Cur.Max Qt Pages then 138 Cur.Session.Outcome := Sizeout; 139 exit ; 140 e l s i f Cur . Cost > Cur . Max Cost then 141 Cur.Session.Outcome := Costout; 142 exit ; 143 else 144 Cur.Choice := Cur.Choose (Cur.Page.Content); 145 Cur.Page.Clicked := Clock; 146 Cur.Page.Choice := Cur.Choice; 147 Cur. Session .Pages.Append (Cur.Page); 148 149 case Cur . Update Travels i s 150 when Central To Central => 151 −− Increment the travel count 152 −− from the current central atom 153 −− to the current choice = next central atom. 154 −− (Default behaviour.) 146 APPENDIX B. PROGRAM LISTINGS
155 Inc (Cur.Travels 156 (Cur.Page.Content.Items. First Element .Atom, 157 Cur.Page.Choice)); 158 159 when Page To Central => 160 −− Increment the travel count 161 −− from the current entire page 162 −− (all atoms except choice) 163 −− to the current choice = next central atom. 164 Increment Travels (Cur.Page.Content , Cur.Page.Choice); 165 166 when A l l P r e v i o u s C e n t r a l T o C e n t r a l => 167 −− Increment the travel count 168 −− from all previous and current central items 169 −− to the current choice = next central atom. 170 declare 171 procedure I n c Travel (C : Timed Page Vectors.Cursor) i s 172 Pg : Timed Page T := Timed Page Vectors.Element (C); 173 begin 174 Inc (Cur.Travels 175 (Pg.Content.Items. First Element .Atom, 176 Cur.Page.Choice)); 177 end ; 178 begin 179 Cur.Session.Pages. Iterate 180 ( Inc Travel ’Unrestricted A c c e s s ) ; 181 end ; 182 183 when A l l P r e v i o u s P a g e T o C e n t r a l => 184 −− Increment the travel count 185 −− from all previous and current pages, 186 −− from all items of each page 187 −− (except the current choice), 188 −− to the current choice = next central atom. 189 declare 190 procedure I n c Travel (C : Timed Page Vectors.Cursor) i s 191 Pg : Timed Page T := Timed Page Vectors.Element (C); 192 begin 193 Increment Travels (Pg.Content, Cur.Page.Choice); 194 end ; 195 begin 196 Cur.Session.Pages. Iterate 197 ( Inc Travel ’Unrestricted A c c e s s ) ; 198 end ; 199 200 when Randomly => 201 Inc (Cur.Travels (Random Atom, Random Atom) ) ; 202 203 when Central To Page => 204 −− Increment travel count from central to 205 −− all items of next page. 206 −− Next page is not known yet (only the central item), 207 −− so here we only memorize the current page and 208 −− we’ll update the travels below after calling Recentre. 209 Prev Page := Cur.Page; 210 B.5. PACKAGE KASIM2 (BODY) 147
211 when Page To Page => 212 −− Increment travel count from each page item 213 −− (except choice) to each item of next page. 214 −− A lot of connections viz. (N−1)∗N; 215 −− normally N=10; 90 connections. 216 −− Just memorize the page for now (see above). 217 Prev Page := Cur.Page; 218 219 when Fa ls e => 220 null ; 221 end case ; 222 223 Cur.Page := Empty Timed Page ; 224 Cur.Page.Requested := Clock; 225 Cur.Page.Content := Cur.Recentre (Cur.Choice); 226 Cur.Page.Served := Clock; 227 228 i f Cur . Update Travels = Central To Page then 229 declare 230 procedure I n c Travel (C : Page Item Vectors.Cursor) i s 231 A : Atom T := Page Item Vectors .Element (C).Atom; 232 begin 233 Inc (Cur.Travels 234 ( Prev Page.Content.Items. First Element.Atom, A)); 235 end ; 236 begin 237 Cur.Page.Content.Items. Iterate 238 ( Inc Travel ’Unrestricted A c c e s s ) ; 239 end ; 240 end i f ; 241 end i f ; 242 end loop ; 243 244 end select ; 245 Cur.Session.Duratio := Clock − Start Time ; 246 Cur. Session .Pages.Append (Cur.Page); 247 Cur.Session.Cost := Cur.Cost; 248 Cur.Session.Size := Natural (Cur.Session.Pages.Length); 249 Cur. Sessions .Append (Cur.Session); 250 Cur.Oracles.Include (Cur.Session.Oracle); 251 end ; 252 253 −−−−−−−−−−−−−−−−−−−−−−− 254 −− COMPUTE RESULTS −− 255 −−−−−−−−−−−−−−−−−−−−−−− 256 257 function Status (Page : Timed Page T) return Timed Page Status T 258 i s 259 function Vet (X : Non Empty Status T) return Timed Page Status T i s 260 begin 261 i f Page.Content = Empty Page then return Anomalous ; end i f ; 262 return X; 263 end ; 264 begin 265 i f Page = Empty Timed Page then return Empty ; 266 e l s i f Page . F i n a l then return Vet (Final); 148 APPENDIX B. PROGRAM LISTINGS
267 e l s i f Page.Clicked /= Undefined Time then return Vet (Clicked); 268 e l s i f Page.Served /= Undefined Time then return Vet (Served); 269 e l s i f Page.Requested /= Undefined Time then return Requested ; 270 else raise Program Error ; 271 end i f ; 272 end ; 273 274 function Get (S : Session T; Q : Quality Measure T) return Real T i s 275 begin 276 case Q i s 277 when Cost => return S . Cost ; 278 when Duratio => return Real T (S.Duratio); 279 when S i z e => return Real T ( S . S i z e ) ; 280 end case ; 281 end ; 282 283 procedure Update Result (R : in out Result T; I : Positive) i s 284 S : Session T := S e s s i o n Vectors.Element (Cur.Sessions , I); 285 begin 286 for Q in Quality Measure T loop 287 Inc (R.Quality (Q, Sum), Get (S, Q)); 288 289 i f Get (S , Q) < R.Quality (Q, Min) then 290 R.Quality (Q, Min) := Get (S, Q); 291 end i f ; 292 293 i f Get (S , Q) > R.Quality (Q, Max) then 294 R.Quality (Q, Max) := Get (S, Q); 295 end i f ; 296 end loop ; 297 Index Vectors.Append (R.Sessions , I); 298 Inc (R.Quantity (S.Outcome)); 299 end ; 300 301 procedure Compute Results i s 302 use Atom Sets ; 303 use S e s s i o n V e c t o r s ; 304 use Result Maps ; 305 R : Result T; 306 procedure P r e p a r e O r a c l e Result (C : Atom Sets.Cursor) i s 307 begin 308 R.Oracle := Element (C); 309 Insert (Cur.Per Oracle , R.Oracle, R); 310 end ; 311 312 procedure Update Results (C : Session Vectors .Cursor) i s 313 I : Positive := Session V e c t o r s . To Index (C) ; 314 S : Session T := S e s s i o n Vectors.Element (C); 315 R : Result T; 316 begin 317 Update Result (Cur.Total, I); 318 R := Result Maps.Element (Cur.Per Oracle , S.Oracle); 319 Update Result (R, I); 320 Replace (Cur.Per Oracle , R.Oracle, R); 321 end ; 322 B.5. PACKAGE KASIM2 (BODY) 149
323 procedure C o l l e c t Oracle (X : Session Vectors .Cursor) i s 324 begin 325 Cur.Oracles.Include (Session Vectors.Element (X).Oracle); 326 end ; 327 begin 328 for Q in Quality Measure T loop 329 R.Quality (Q, Min) := Real T ’ Last ; 330 R.Quality (Q, Max) := Real T ’ F i r s t ; 331 end loop ; 332 333 Cur.Oracles.Clear; 334 Cur.Sessions.Iterate (Collect Oracle ’Unrestricted A c c e s s ) ; 335 −− Define Cur.Oracles; used to be done in Simulate Session piecemeal , 336 −− but better redo it here; one reason is that this way it works with 337 −− loaded, previous simulated , sessions. Leaving the incremental updating 338 −− in Simulate Session, because it is used for other things; for this 339 −− incremental updating the clearing is done in Setup Experiment. 340 341 Cur . Per Oracle.Clear; 342 Cur.Oracles.Iterate (Prepare O r a c l e Result ’Unrestricted A c c e s s ) ; 343 Cur.Sessions.Iterate (Update Results ’ Unrestricted A c c e s s ) ; 344 end ; 345 346 −−−−−−−−−−−−−−−−−−−−−−−−−−−−− 347 −− LOAD, SAVE EXPERIMENT −− 348 −−−−−−−−−−−−−−−−−−−−−−−−−−−−− 349 350 procedure Save Cur i s 351 begin 352 null ; −−tbd 353 end ; 354 355 procedure Load Cur i s 356 begin 357 null ; −−tbd 358 end ; 359 360 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 361 −− MISCELLANEOUS SUBPROGRAMS −− 362 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 363 364 function Higher Rank (Left, Right : Page Item T) return Boolean i s 365 begin 366 return L e f t . Rank > Right .Rank; 367 end ; 368 369 function To Vector (X : Page Item Array T) return Page Item Vector T i s 370 use Page Item Vectors ; 371 Y : Page Item Vector T := To Vector (X’Length); 372 begin 373 for I in X’ Range loop 374 Replace Element (Y, I, X (I)); 375 end loop ; 376 return Y; 377 end ; 378 150 APPENDIX B. PROGRAM LISTINGS
379 function To Array (X : Page Item Vector T) return Page Item Array T i s 380 use Page Item Vectors ; 381 Y : Page Item Array T ( F i r s t Index (X) .. Last Index (X) ) ; 382 begin 383 for I in Y’ Range loop 384 Y (I) := Element (X, I); 385 end loop ; 386 return Y; 387 end ; 388 389 procedure Inc (X : in out Integer; Y : Integer := 1) i s 390 begin 391 X := X + Y; 392 end ; 393 394 procedure Inc (X : in out Real T ; Y : Real T := 1 . 0 ) i s 395 begin 396 X := X + Y; 397 end ; 398 399 procedure Inc (X : in out Real T) i s begin X := X + 1 . 0 ; end ; 400 401 procedure Save (Sessions : Session V e c t o r T; Filename : String) i s 402 use Ada. Streams .Stream IO; 403 F i l e : File Type ; 404 begin 405 Create (File , Out File , Filename); 406 Session Vector T ’Output (Stream (File), Sessions); 407 Close (File); 408 end ; 409 410 procedure Load (Sessions : out S e s s i o n V e c t o r T; Filename : String) i s 411 use Ada. Streams .Stream IO; 412 F i l e : File Type ; 413 begin 414 Open (File, In File , Filename); 415 Sessions := Session Vector T ’Input (Stream (File)); 416 Close (File); 417 end ; 418 419 function I s In (Atom : Atom T; Items : Page Item Vector T) return Boolean i s 420 begin 421 return 422 not Items . Is Empty 423 and then I s In (Atom, Items, Items.First Index , Items.Last Index ) ; 424 end ; 425 426 function I s I n 427 (Atom : Atom T; 428 Items : Page Item Vector T; 429 First : Positive) 430 return Boolean i s 431 begin 432 return 433 not Items . Is Empty 434 and then I s In (Atom, Items, First , Items.Last Index ) ; B.6. PACKAGE KASIM2.ACTIVATION (SPEC) 151
435 end ; 436 437 function I s I n 438 (Atom : Atom T; 439 Items : Page Item Vector T; 440 First, Last : Positive) 441 return Boolean i s 442 begin 443 pragma Assert 444 ( not Items . Is Empty 445 and then F i r s t >= Items.First I n d e x 446 and then Last <= Items.Last Index ) ; 447 for I in First .. Last loop 448 i f Atom = Items.Element(I).Atom then return True ; end i f ; 449 end loop ; 450 return Fa ls e ; 451 end ; 452 453 function Count Non Null (X : Real Vector ) return Natural i s 454 N : Natural := 0; 455 begin 456 for I in X’ Range loop 457 i f X ( I ) /= 0 . 0 then N := N + 1 ; end i f ; 458 end loop ; 459 return N; 460 end ; 461 462 function To Set (X : Page Item Vector T) return Atom Set T i s 463 Z : Atom Set T; 464 procedure Add (C : Page Item Vectors.Cursor) i s 465 begin 466 Z.Include (Page Item Vectors.Element (C).Atom); 467 end ; 468 begin 469 X. Iterate (Add’Unrestricted A c c e s s ) ; 470 return Z; 471 end ; 472 473 474 end ; 475 476 −− gnatmake duration o f l a o p −largs /Developer/SDKs/MacOSX10.4u.sdk/usr/lib/libblas . dylib /Developer/SDKs/MacOSX10.4u.sdk/usr/lib/liblapack . dylib
B.6 Package Kasim2.Activation (spec)
1 package Kasim2. Activation i s 2 −−−−−−−−−−−−−−−−−−−−−−−−−−− 3 −− PRINCIPAL FUNCTIONS −− 4 −−−−−−−−−−−−−−−−−−−−−−−−−−− 5 function Choose (Page : Page T) return Atom T; 6 function Choose Watermark (Page : Page T) return Atom T; 7 function Recentre (Atom : Atom T) return Page T; 8 function Recentre Watermark (Atom : Atom T) return Page T; 152 APPENDIX B. PROGRAM LISTINGS
9 function Super Page return Page T; 10 function Random Page (Atom : Atom T) return Page T; 11 12 −−−−−−−−−−−−−−−−−−−−−−−−−−− 13 −− AUXILIARY FUNCTIONS −− 14 −−−−−−−−−−−−−−−−−−−−−−−−−−− 15 function Most Active 16 (A : Real Vector; Qt : Natural) return Page Item Vector T; 17 end ;
B.7 Package Kasim2.Activation (body)
1 with ada . t e x t i o ; use ada . t e x t i o ; 2 with Ada. Calendar ; use Ada. Calendar ; 3 with Kasim2. Oracle Model ; 4 with Kasim2. Reporting ; 5 6 package body Kasim2. Activation i s 7 8 function Most Active 9 (A : Real Vector; Qt : Natural) return Page Item Vector T 10 i s 11 Items : Page Item Array T(A’ Range); 12 begin 13 for I in A’ Range loop 14 Items (I) := (Atom => I , Rank => A(I)); 15 end loop ; 16 Sort By Descending Rank (Items); 17 return To Vector (Items (1 .. Qt)); 18 end ; 19 20 procedure Update Watermark 21 (Watermark : in out Real Vector; Level : in Real Vector ) i s 22 begin 23 pragma Assert (Watermark’First = Level ’First); 24 pragma Assert (Watermark’Last = Level ’Last); 25 for I in Watermark ’ Range loop 26 i f Level ( I ) > Watermark (I) then 27 Watermark (I) := Level (I); 28 end i f ; 29 end loop ; 30 end ; 31 32 function Choose (Page : Page T) return Atom T i s 33 −− Spread back from the oracle, till a page item is reached. 34 −− Alternative to try: till ∗ a l l ∗ page items are reached. 35 −− Nah. First item reached is the less distant, 36 −− and therefore the obvious choice. 37 −− Also to try: user will not chose a previously seen/chosen item. 38 −− Done by the outer system (Recentre). 39 −− Also: only normal paragraphs for oracles (outer system). 40 −− Yeah, to try. 41 use Real Arrays ; 42 use Page Item Vectors ; B.7. PACKAGE KASIM2.ACTIVATION (BODY) 153
43 W : Real Matrix Ptr T renames Cur.Travels; 44 N : Natural := W’Length; 45 A : Real Vector (1 .. N); 46 lambda : Real T := 1 . 0 ; 47 alpha : Real T := 1 . 0 ; 48 I : Real Matrix := Unit Matrix (N, 1, 1); 49 M : Real Matrix Ptr T; 50 P : Real Vector (1 .. N); 51 X : Positive; 52 begin 53 pragma Assert (Page.Qt Expanded < Natural(Page.Items.Length)); 54 −− Defensive programming, to catch possible bug in the outer 55 −− system . 56 i f I s In (Cur.Session.Oracle , Page.Items, Page.Qt Expanded + 1) then 57 return Cur.Session.Oracle; 58 end i f ; 59 −− If oracle is one of the contracted atoms, that’s the choice. 60 61 −− (We don’t test here for the oracle being one of the expanded 62 −− atoms. The outer system is supposed to detect that as session 63 −− success and never call us.) 64 65 −− So, from this point on SA is required. 66 67 −− We want the page atom that is most strongly connected to the 68 −− oracle globally , as a measure of the likeliness that that 69 −− choice will lead to the oracle quicker/easier. 70 71 −− So we spread from the oracle until we find a page item. 72 73 −− Setup the matrix 74 M := new Real Matrix (1 .. N, 1 .. N); 75 M. a l l := lambda ∗ I + alpha ∗ W. a l l ; 76 −− Energise oracle atom, zero the others 77 A := ( others => 0 . 0 ) ; 78 A (Cur.Session.Oracle) := 1.0; 79 80 −− Prepare P, the page items identity vector, 81 −− to use linear algebra to check if page item reached 82 P := ( others => 0 . 0 ) ; 83 for I in F i r s t Index (Page.Items) .. Last Index (Page.Items) loop 84 P (Element (Page.Items, I).Atom) := 1.0; 85 end loop ; 86 87 −− Propagate till a page item is reached. 88 −− This is the critical part: 89 −− sometimes a page item is never reached, 90 −− which makes this loop until timeout! 91 −− Alternative: propagate till a certain level is 92 −− reached. Sensible level is between 1 (one item) 93 −− to Page Size (all items at 1 each). By exploratory 94 −− experimentation with 200 atoms, found values > 1.0 95 −− worsen results. That is equal to 1 item (the initial 96 −− method ) . 97 while P ∗ A < 1 . 0 loop −− Alt : < Real T (Page.Items.Length) loop 98 A := M. a l l ∗ A; 154 APPENDIX B. PROGRAM LISTINGS
99 end loop ; 100 Free (M) ; 101 −− Find the most energized page item X 102 X := F i r s t Index (Page.Items); 103 for Y in X + 1 . . Last Index (Page.Items) loop 104 i f A (Element (Page.Items , Y).Atom) 105 > A (Element (Page.Items , X).Atom) then X := Y; end i f ; 106 end loop ; 107 108 −− Done . 109 return Element (Page.Items , X).Atom; 110 end ; 111 112 function Choose Watermark (Page : Page T) return Atom T i s 113 −− Like Choose, but with Watermark. 114 use Real Arrays ; 115 use Page Item Vectors ; 116 W : Real Matrix Ptr T renames Cur.Travels; 117 N : Natural := W’Length; 118 A : Real Vector (1 .. N); 119 lambda : Real T := 0 . 8 ; 120 alpha : Real T := 1 . 0 ; 121 I : Real Matrix := Unit Matrix (N, 1, 1); 122 M : Real Matrix Ptr T; 123 P : Real Vector (1 .. N); 124 X : Positive; 125 Watermark : Real Vector (1 .. N); 126 Min Level : Real T := 1 . 0 ; 127 −− See below . 128 −− Watermark requires as much or more 129 −− parameters than the other heuristics 130 −− a f t e r a l l . 131 132 procedure Update A (Value : Real Vector ) i s 133 begin 134 A := Value ; 135 Update Watermark (Watermark, A); 136 end ; 137 138 procedure Update A (Index : Positive; Value : Real T) i s 139 begin 140 A (Index) := Value; 141 Update Watermark (Watermark, A); 142 end ; 143 begin 144 pragma Assert (Page.Qt Expanded < Natural(Page.Items.Length)); 145 −− Defensive programming, to catch possible bug in the outer 146 −− system . 147 148 i f I s In (Cur.Session.Oracle , Page.Items, Page.Qt Expanded + 1) then 149 return Cur.Session.Oracle; 150 end i f ; 151 −− If oracle is one of the contracted atoms, that’s the choice. 152 153 −− (We don’t test here for the oracle being one of the expanded 154 −− atoms. The outer system is supposed to detect that as session B.7. PACKAGE KASIM2.ACTIVATION (BODY) 155
155 −− success and never call us.) 156 157 −− So, from this point on SA is required. 158 159 −− We want the page atom that is most strongly connected to the 160 −− oracle globally , as a measure of the likeliness that that 161 −− choice will lead to the oracle quicker/easier. 162 163 −− So we spread from the oracle until we find a page item. 164 165 −− Setup the matrix 166 M := new Real Matrix (1 .. N, 1 .. N); 167 M. a l l := lambda ∗ I + alpha ∗ W. a l l ; 168 169 −− Energise oracle atom, zero the others 170 A := ( others => 0 . 0 ) ; 171 Watermark := ( others => 0 . 0 ) ; 172 Update A (Cur.Session.Oracle, 1.0); 173 174 −− Prepare P, the page items identity vector, 175 −− to use linear algebra to check if page item reached 176 P := ( others => 0 . 0 ) ; 177 for I in F i r s t Index (Page.Items) .. Last Index (Page.Items) loop 178 P (Element (Page.Items, I).Atom) := 1.0; 179 end loop ; 180 181 −− Propagate till a certain level of energy 182 −− is reached in page items. 183 −− This is the critical part: 184 −− sometimes a page item is never reached, 185 −− which makes this loop until timeout! 186 while P ∗ Watermark < Min Level loop 187 Update A (M. a l l ∗ A); 188 end loop ; 189 Free (M) ; 190 191 −− Find the most energized page item X 192 X := F i r s t Index (Page.Items); 193 for Y in X + 1 . . Last Index (Page.Items) loop 194 i f Watermark (Element (Page.Items , Y).Atom) 195 > Watermark (Element (Page.Items , X).Atom) then X := Y; end i f ; 196 end loop ; 197 198 −− Done . 199 return Element (Page.Items , X).Atom; 200 end ; 201 202 function Recentre (Atom : Atom T) return Page T i s 203 −− Spread forward from the atom until the 204 −− required quantity of atoms is non−nully energized. 205 −− Test only new atoms i.e. atoms not yet seen in the session. 206 use Real Arrays ; 207 use Page Item Vectors ; 208 W : Real Matrix Ptr T renames Cur.Travels; 209 N : Natural := W’Length; 210 A : Real Vector (1 .. N); 156 APPENDIX B. PROGRAM LISTINGS
211 lambda : Real T := 1 . 0 ; 212 alpha : Real T := 1 . 0 ; 213 I : Real Matrix := Unit Matrix (N, 1, 1); 214 M : Real Matrix Ptr T; 215 All Atoms , 216 Not Seen Yet , 217 Not Seen Yet Plus Central , 218 Seen Except Central : Atom Set T; 219 220 function Active return Atom Set T i s 221 Z : Atom Set T; 222 begin 223 Z . Clear ; 224 for I in A’ Range loop 225 i f A(I) > 0 . 0 then 226 Z.Include (I); 227 end i f ; 228 end loop ; 229 return Z; 230 end ; 231 begin 232 −− Prepare atom sets 233 for I in 1 . . N loop 234 All Atoms.Insert (I); 235 end loop ; 236 Not Seen Yet := All Atoms.Difference (Cur.Seen); 237 Not Seen Yet Plus Central := Not Seen Yet ; 238 Not Seen Yet Plus Central.Insert (Atom); 239 Seen Except Central := Cur.Seen; 240 Seen Except Central.Delete (Atom); 241 −− The outer system is supposed to keep Cur.Seen in good order 242 −− viz. containing the central Atom. Otherwise the Insert or 243 −− Delete operations above will raise an exception. We could 244 −− Include and Exclude (which do not raise) but decided to use 245 −− Insert/Delete as an instrument of checking the correctness 246 −− of the outer system. 247 248 −− Setup the matrix 249 M := new Real Matrix (1 .. N, 1 .. N); 250 M. a l l := lambda ∗ I + alpha ∗ Transpose (W. a l l ); 251 252 −− Energise central atom, zero the others 253 A := ( others => 0 . 0 ) ; 254 A (Atom) := 1.0; 255 256 −− Spread . . . 257 for I in 1 . . N loop 258 i f Natural (Atom Sets.Length 259 (Active.Intersection (Not Seen Yet Plus Central ) ) ) 260 >= Cur . Page Size then 261 exit ; 262 end i f ; 263 A := M. a l l ∗ A; 264 end loop ; 265 Free (M) ; 266 B.7. PACKAGE KASIM2.ACTIVATION (BODY) 157
267 −− De−energize seen atoms (except central one) 268 declare 269 procedure Nullify (C :Atom Sets.Cursor) i s 270 begin 271 A ( Atom Sets.Element (C)) := 0.0; 272 end ; 273 begin 274 Seen Except Central.Iterate (Nullify ’Unrestricted A c c e s s ) ; 275 end ; 276 277 −− Make sure the central atom is the most energized 278 A (Atom) := Real T ’ Last ; 279 280 −− Done 281 return ( Items => Most Active (A, Kasim2.Cur.Page Size ) , 282 Qt Expanded => Cur . Qt Expanded , 283 Cost => 1 . 0 ) ; 284 end ; 285 286 function Recentre Watermark (Atom : Atom T) return Page T i s 287 −− Like Recentre, but using watermark (cf. chapter on SA). 288 −− Also, lambda less than 1, because: 289 −− if lambda = 1, then level = watermark. 290 use Real Arrays ; 291 use Page Item Vectors ; 292 W : Real Matrix Ptr T renames Cur.Travels; 293 N : Natural := W’Length; 294 A : Real Vector (1 .. N); 295 lambda : Real T := 0 . 3 ; 296 alpha : Real T := 1 . 0 ; 297 I : Real Matrix := Unit Matrix (N, 1, 1); 298 M : Real Matrix Ptr T; 299 All Atoms , 300 Not Seen Yet , 301 Not Seen Yet Plus Central , 302 Seen Except Central : Atom Set T; 303 Watermark : Real Vector (1 .. N); 304 305 procedure Update A (Value : Real Vector ) i s 306 begin 307 A := Value ; 308 Update Watermark (Watermark, A); 309 end ; 310 311 procedure Update A (Index : Positive; Value : Real T) i s 312 begin 313 A (Index) := Value; 314 Update Watermark (Watermark, A); 315 end ; 316 317 function Active return Atom Set T i s 318 Z : Atom Set T; 319 begin 320 Z . Clear ; 321 for I in A’ Range loop 322 i f A(I) > 0 . 0 then 158 APPENDIX B. PROGRAM LISTINGS
323 Z.Include (I); 324 end i f ; 325 end loop ; 326 return Z; 327 end ; 328 begin 329 −− Prepare atom sets 330 for I in 1 . . N loop 331 All Atoms.Insert (I); 332 end loop ; 333 Not Seen Yet := All Atoms.Difference (Cur.Seen); 334 Not Seen Yet Plus Central := Not Seen Yet ; 335 Not Seen Yet Plus Central.Insert (Atom); 336 Seen Except Central := Cur.Seen; 337 Seen Except Central.Delete (Atom); 338 −− The outer system is supposed to keep Cur.Seen in good order 339 −− viz. containing the central Atom. Otherwise the Insert or 340 −− Delete operations above will raise an exception. We could 341 −− Include and Exclude (which do not raise) but decided to use 342 −− Insert/Delete as an instrument of checking the correctness 343 −− of the outer system. 344 345 −− Setup the matrix 346 M := new Real Matrix (1 .. N, 1 .. N); 347 M. a l l := lambda ∗ I + alpha ∗ Transpose (W. a l l ); 348 349 −− Energise central atom, zero the others 350 A := ( others => 0 . 0 ) ; 351 Watermark := A; 352 Update A (Atom, 1.0); 353 354 −− Spread . . . 355 for I in 1 . . N loop 356 i f Natural (Atom Sets.Length 357 (Active.Intersection (Not Seen Yet Plus Central ) ) ) 358 >= Cur . Page Size then 359 exit ; 360 end i f ; 361 Update A (M. a l l ∗ A); 362 end loop ; 363 Free (M) ; 364 365 −− De−energize seen atoms (except central one) 366 declare 367 procedure Nullify (C :Atom Sets.Cursor) i s 368 begin 369 A ( Atom Sets.Element (C)) := 0.0; 370 Watermark (Atom Sets.Element (C)) := 0.0; 371 end ; 372 begin 373 Seen Except Central.Iterate (Nullify ’Unrestricted A c c e s s ) ; 374 end ; 375 376 −− Make sure the central atom is the most energized 377 Update A (Atom, Real T ’ Last ) ; 378 B.7. PACKAGE KASIM2.ACTIVATION (BODY) 159
379 −− Done 380 return ( Items => Most Active (Watermark, Kasim2.Cur.Page Size ) , 381 Qt Expanded => Cur . Qt Expanded , 382 Cost => 1 . 0 ) ; 383 end ; 384 385 function S t a r t return Page T i s 386 begin 387 return Recentre (1); 388 end ; 389 390 function Super Page return Page T i s 391 −− Activate all atoms (with a small value), 392 −− spread (back) X times, 393 −− return the most active. 394 use Real Arrays ; 395 use Page Item Vectors ; 396 W : Real Matrix Ptr T renames Cur.Travels; 397 N : Natural := W’Length; 398 A : Real Vector (1 .. N) := ( others => 0 . 0 0 1 ) ; 399 lambda : Real T := 1 . 0 ; 400 alpha : Real T := 1 . 0 ; 401 I : Real Matrix := Unit Matrix (N, 1, 1); 402 M : Real Matrix Ptr T; 403 begin 404 −− Setup the matrix. 405 M := new Real Matrix (1 .. N, 1 .. N); 406 M. a l l := lambda ∗ I + alpha ∗ W. a l l ; −−Transpose (W. all); 407 408 −− Spread. (Atoms are already activated.) 409 for I in 1 . . 5 loop −− naively , 1..1 410 A := M. a l l ∗ A; 411 end loop ; 412 Free (M) ; 413 414 −− Done . 415 return ( Items => Most Active (A, Cur.Page Size ) , 416 Qt Expanded => Cur . Qt Expanded , −− 0 on some designs 417 Cost => 1 . 0 ) ; 418 end ; 419 420 function Random Page (Atom : Atom T) return Page T i s 421 package Random Atoms i s 422 new Kasim2. Oracle Model. Uniformly Distributed 423 ( Qt Oracles => Cur.Travels ’Length , 424 Doc Size => Cur.Travels ’Length , 425 Seed => Natural (Seconds (Clock))); 426 function Random Atom return Atom T renames Random Atoms. Random Oracle ; 427 Z : Page Item Vector T; 428 begin 429 for I in 1 .. Cur.Page Size loop 430 Z.Append ((Atom => Random Atom , 431 Rank => Real T ( Cur . Page Size − 1 + 1) ) ) ; 432 end loop ; 433 return ( Items => Z, 434 Qt Expanded => Cur . Qt Expanded , 160 APPENDIX B. PROGRAM LISTINGS
435 Cost => 1 . 0 ) ; 436 end ; 437 end ;
B.8 Package Kasim2.Comparate (spec)
1 −−with Ada.Numerics.Discrete Random ; 2 −−with Ada.Text IO; use Ada.Text IO; 3 −−with Arm05 Model ; 4 −−with Kasim2.Activation; 5 with Kasim2.Documents; 6 −−with Kasim2.Oracle Model ; 7 −−with Kasim2.Reporting; 8 9 package Kasim2. Comparate i s 10 11 −− This unit supports the integration of diverse 12 −− experiments, or configurations , and respective 13 −− reporting in LaTeX format. 14 15 Create Index : Boolean := False; 16 17 type Configuration T; 18 19 type O r a c l e F o r S e s s i o n F u n c t i o n P t r T is access function (I : Positive) return Atom T; 20 21 type Configuration T i s 22 record 23 −− General parameters 24 Name : Unbounded String ; 25 Description : Unbounded String ; 26 Qt Atoms : Natural; 27 Qt Oracles : Natural; 28 Q t Sessions : Natural; 29 Page Size : Natural; 30 Qt Expanded : Natural; 31 32 −− Parameters for simulation 33 Tree : Kasim2.Documents.Tree Ptr T; 34 Travels : Real Matrix Ptr T; 35 Start : Start F u n c t i o n P t r T; 36 Choose : Choose Function Ptr T; 37 Recentre : Recentre Function Ptr T; 38 Update Travels : Update Travels T; 39 O r a c l e F o r Session : Oracle F o r S e s s i o n F u n c t i o n P t r T := null ; 40 −− The subprogram pointer parameters probably cannot 41 −− be saved/loaded significantly ... 42 43 −− Practical limitations to execution 44 Max Duration : Duration := 30.0; 45 Max Qt Pages : Natural := 6; 46 Max Cost : Real T := 6 0 . 0 ; 47 B.9. PACKAGE KASIM2.COMPARATE (BODY) 161
48 −− Parameters for legacy document 49 Next Link Weight : Real T := 1 . 0 ; 50 Child Link Weight Heading Paragraph : Real T := 1 . 0 ; 51 Child Link Weight Heading Heading : Real T := 1 . 0 ; 52 Residual Weight : Real T := 0 . 0 ; 53 end record ; 54 55 type Configuration Array T i s array ( P o s i t i v e range <>) of Configuration T; 56 57 procedure R u n Inconditionally (X : Configuration T); 58 procedure Run (X : Configuration T); 59 procedure Run (Batch : Configuration Array T); 60 procedure Report (Filename : String; Batch : Configuration Array T); 61 −− Each batch Run executes the simulations and stores the results in .dat files. 62 −− Each batch Report reads the .dat files and produces a latex fragment and 63 −− required pictures (.bmp, .pdf, .dot files); the .bmp files are converted to 64 −− .pdf using the ImageMagick command line tool (automatically). 65 −− All generated filenames are prefixed 66 −− kasim2− 67 −− All individual configuration filenames are further affixed 68 −− −configuration −
B.9 Package Kasim2.Comparate (body)
1 with kasim2. structural ; 2 with ada.assertions ; use ada.assertions ; 3 with ada . t e x t i o ; 4 with Ada. Calendar ; 5 with Ada. Calendar .Formatting; 6 with Ada. Characters .Handling; 7 with Ada.Containers. Indefinite O r d e r e d S e t s ; 162 APPENDIX B. PROGRAM LISTINGS
8 with Ada. Directories ; use Ada. Directories ; 9 with Ada. Exceptions ; 10 with Ada. Streams .Stream IO; 11 with Ada . Text IO; 12 with Arm05 Model . Make Graph File ; 13 with Kasim2.Documents; 14 with Execute System Call ; 15 with Kasim2. Oracle Model ; 16 with Kasim2. Reporting ; 17 with Kasim2. Reporting .Latex; 18 19 package body Kasim2. Comparate i s 20 21 function ”<” (L, R : Configuration T) return Boolean i s 22 begin 23 return L . Name < R. Name ; 24 end ; 25 26 package Configuration S e t s i s 27 new Ada.Containers. Indefinite O r d e r e d S e t s 28 ( Element Type => Configuration T); 29 30 Configuration Set : Configuration S e t s . Set ; 31 32 Current Name : Unbounded String ; 33 34 −−−−−−−−−−−−−−−−− 35 −− FILENAMES −− 36 −−−−−−−−−−−−−−−−− 37 38 Annex Filename : String := ”.. \ Thesis \annex−of−d e t a i l e d −results.tex”; 39 Index Filename : String := ”.. \ Thesis \ index−of−configurations.tex”; 40 Edited Filename : String := ”.. \ Thesis \ edited −index−of−configurations.tex”; 41 42 function Filename (X : Configuration T) return S t r i n g i s 43 use Ada. Characters .Handling; 44 begin 45 return ”kasim2−configuration −”& 46 To Lower ( To String (X.Name)); 47 end ; 48 49 function Parameters Filename (X : Configuration T) return S t r i n g i s 50 begin 51 return Filename (X) & ”−parameters.dat”; 52 end ; 53 54 function Results Filename (X : Configuration T) return S t r i n g i s 55 begin 56 return Filename (X) & ”−results .dat”; 57 end ; 58 59 procedure Save (Config : Configuration T) i s 60 use Ada. Streams .Stream IO; 61 F i l e : File Type ; 62 use Ada. Exceptions ; 63 begin B.9. PACKAGE KASIM2.COMPARATE (BODY) 163
64 Create (File , Out File , Parameters Filename (Config)); 65 Configuration T ’Output (Stream (File), Config); 66 Close (File); 67 exception 68 when E: others => 69 Ada . Text IO . Put ( ”EXCEPTION IN SAVE: ” & E x c e p t i o n Information (E)); 70 raise ; 71 end ; 72 73 function Load (Filename : String) return Configuration T i s 74 use Ada. Streams .Stream IO; 75 F i l e : File Type ; 76 Config : Configuration T; 77 use Ada. Exceptions ; 78 begin 79 Open (File, In File , Filename); 80 Config := Configuration T ’Input (Stream (File)); 81 Close (File); 82 return Config ; 83 exception 84 when E: others => 85 Ada . Text IO . Put ( ”EXCEPTION IN LOAD: ” & E x c e p t i o n Information (E)); 86 i f Is Open ( F i l e ) then Close (File); end i f ; 87 raise ; 88 end ; 89 90 −−−−−−−−−−−−−−−−−−−−−−−−− 91 −− RUN CONFIGURATION −− 92 −−−−−−−−−−−−−−−−−−−−−−−−− 93 94 procedure R u n Inconditionally (X : Configuration T) i s 95 package Oracles i s 96 new Kasim2. Oracle Model. Uniformly Distributed 97 ( Doc Size => X. Qt Atoms , 98 Qt Oracles => X. Qt Oracles , 99 Seed => 0) ; 100 function O r a c l e F o r Session (I : Positive) return Atom T i s 101 begin 102 i f X. O r a c l e F o r S e s s i o n /= null then 103 return X. O r a c l e F o r S e s s i o n . a l l (I); 104 else 105 return Oracles .Random Oracle ; 106 end i f ; 107 end ; 108 use type Kasim2.Documents. Tree Ptr T; 109 begin 110 Save (X) ; 111 i f X. Next Link Weight + 112 X. Child Link Weight Heading Paragraph + 113 X. Child Link Weight Heading Heading + 114 X. Residual Weight > 0 . 0 115 and then X. Tree /= null then 116 Arm05 Model . Make Graph File (Filename (X) & ”−doc−tree”, X.Tree. a l l ); 117 −−ada . t e x t i o . p u t line(”x.tree not null”); 118 kasim2.structural.doc := x.tree;−− b i g b i g hack 119 −−ada . t e x t io.put(” to mat, tree len= ”&x.tree ’length ’img); 164 APPENDIX B. PROGRAM LISTINGS
120 −−assert(x.travels /= null ,”x.travels null”); 121 −−ada . t e x t io.put(x.travels ’length ’img); 122 Kasim2.Documents. Tree To Matrix 123 (X. Tree . all , 124 X. Travels , 125 Next Link Weight => X. Next Link Weight , 126 Child Link Weight Heading Paragraph => X. Child Link Weight Heading Paragraph , 127 Child Link Weight Heading Heading => X. Child Link Weight Heading Heading , 128 Residual Weight => X. Residual Weight ) ; 129 end i f ; 130 −−ada . t e x t io.put(” setup ”); 131 −−assert(x.tree /= null , ”x.tree null”); 132 Setup Experiment 133 (Name => To String (X.Name) , 134 Travels => X. Travels , 135 S t a r t => X. Start , 136 Choose => X. Choose , 137 Recentre => X.Recentre , 138 Page Size => X. Page Size , 139 Qt Expanded => X. Qt Expanded , 140 Update Travels => X. Update Travels , 141 Max Qt Pages => X. Max Qt Pages , 142 Max Duration => X. Max Duration , 143 Max Cost => X. Max Cost ) ; 144 −−ada . t e x t io.put(” about tpo loop”); 145 146 for I in 1 . . X. Q t S e s s i o n s loop 147 Ada . Text IO.Put(Integer ’Image (I)); 148 −−ada . t e x t io .put(” ora=”&oracle f o r session(i)’img); 149 Kasim2. Simulate Session (Oracle F o r Session (I)); 150 end loop ; 151 Kasim2.Save (Kasim2.Cur.Sessions , Results Filename (X)); 152 end ; 153 154 function Significant Components (X : Configuration T) return Configuration T i s 155 Z : Configuration T := X; 156 begin 157 Z.Description := Null Unbounded String ; 158 159 Z . S t a r t := null ; 160 Z . Choose := null ; 161 Z.Recentre := null ; 162 Z . O r a c l e F o r S e s s i o n := null ; 163 −− Removing function pointers because they may be comparing wrongly 164 −− i.e. same function different pointer on different runs. 165 166 Z . Tree := null ; 167 Z.Travels := null ; 168 −− Ditto for object pointer. 169 −− This is defeating the purpose because there might have been 170 −− a significant change... 171 −− Doing this now for speed. 172 −− To be corrected. 173 174 return Z; B.9. PACKAGE KASIM2.COMPARATE (BODY) 165
175 end ; 176 177 procedure Run (X : Configuration T) i s 178 use Ada. Directories ; 179 use Ada. Exceptions ; 180 use Ada . Text IO; 181 function Trace (S : String) return Boolean i s 182 begin 183 −−Ada . Text IO.Put (”[” & S & ”]”); 184 return True ; 185 end ; 186 procedure Run Experiment i s 187 begin 188 Put (” simulating” & Natural’Image (X.Qt Sessions) & ” sessions...”); 189 R u n Inconditionally (X); 190 Put Line (” done.”); 191 end ; 192 begin 193 Put (”Processing configuration [” & To String (X.Name) & ”]...”); 194 i f Exists (Results Filename (X)) and then Trace (”results exists”) 195 and then Exists (Parameters Filename (X)) and then Trace (”params exists”) 196 then 197 Put (” results file exists for same params, will try to load and compare:”); 198 begin 199 i f Significant Components (Load (Parameters Filename (X))) = Significant Components (X) then 200 Put Line (”significant params match, no need to re−run . ” ) ; 201 return ; 202 end i f ; 203 exception 204 when others => 205 Put Line (”load failed! Running experiment.”); 206 Run Experiment ; 207 end ; 208 else 209 put(”running ...”); 210 Run Experiment ; 211 end i f ; 212 exception 213 when E: others => 214 Ada . Text IO . Put Line ( ”ERROR IN RUN CONFIG: ” & E x c e p t i o n Information (E)); 215 raise ; 216 end ; 217 218 procedure Run (Batch : Configuration Array T) i s 219 begin 220 Ada . Text IO . Put Line (”Processing batch of” & 221 Natural ’Image (Batch’Length) & ” configurations.”); 222 for I in Batch ’ Range loop 223 Run (Batch (I)); 224 end loop ; 225 end ; 226 227 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 228 −− REPORT COMPARATE EXPERIMENT −− 229 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 166 APPENDIX B. PROGRAM LISTINGS
230 231 type Comparate Field T i s 232 ( Success Rate , S i z e 0 , S i z e 1 , Size Gain , Cost 0 , Cost 1 , Cost Gain ) ; 233 type Comparate Line T i s array ( Comparate Field T) of Real T; 234 type Comparate Table T i s array ( P o s i t i v e range <>) of Comparate Line T; 235 236 package ACH renames Ada. Characters .Handling; 237 use Kasim2. Reporting ; 238 use Ada . Text IO; 239 Tex , Tex Annex , Tex Index : File Type ; 240 241 function Trim (X : String) return S t r i n g i s 242 begin 243 i f X (X’First) = ’ ’ then return X (X’First + 1 .. X’Last); 244 else return X; 245 end i f ; 246 end ; 247 248 function Ref (Label : String) return S t r i n g i s 249 begin 250 return 251 ”\ r e f {” & Latex . To Label(Label) & 252 ”} ( page \ p a g e r e f {” & Latex . To Label(Label) & ” })”; 253 end ; 254 255 −−−−−−−−−−−−−−−−−−−−−−−−−−−− 256 −− REPORT CONFIGURATION −− 257 −−−−−−−−−−−−−−−−−−−−−−−−−−−− 258 259 function Img (X : Comparate Field T) return S t r i n g i s 260 begin 261 case X i s 262 when Success Rate => return ” Success \ Rate ” ; 263 when S i z e 0 => return ”$\mathrm{ S i z e } 0$ ” ; 264 when S i z e 1 => return ”$\mathrm{ S i z e } 1$ ” ; 265 when Size Gain => return ” S i z e \ g a i n ” ; 266 when Cost 0 => return ”$\mathrm{ Cost } 0$ ” ; 267 when Cost 1 => return ”$\mathrm{ Cost } 1$ ” ; 268 when Cost Gain => return ” Cost \ g a i n ” ; 269 when others => return ”Ooops...”; 270 end case ; 271 end ; 272 273 procedure Report (X : Configuration T ; Comp Line : out Comparate Line T) i s 274 −− Outputs a LaTeX subsection with details of the 275 −− configuration , and results including tables and graphs. 276 −− Creates the graph files with BMP and converts to PDF. 277 Id : String := Latex.To Label ( To String (X.Name)); 278 function Ref (Prefix : String) return S t r i n g i s 279 Z : String := Latex.To Label (Prefix); 280 begin 281 return ”\ r e f {” & Z & Id & ”} ( page \ p a g e r e f {” & Z & Id & ” })”; 282 end ; 283 −−−−−−−−−−−−−−−−−− 284 −− REPORT QUALITY 285 function Quality Name (Q : Quality Measure T) return S t r i n g i s B.9. PACKAGE KASIM2.COMPARATE (BODY) 167
286 Z : String := Latex.To Name (Quality Measure T ’Image (Q)); 287 begin 288 i f Z = ”Duratio” then return ”Duration”; 289 else return Z; 290 end i f ; 291 end ; 292 procedure Report (Q : Quality Measure T) i s 293 Q Name : String := Quality Name (Q) ; 294 Res Tbl : Result Table T := Make Result Table (Q) ; 295 Evo Tbl : Evolution Table T := Make Evolution Table (Q) ; 296 begin 297 i f Q = S i z e then 298 Comp Line ( S i z e 0 ) := Res Tbl (Ind (Res Tbl, Average), First); 299 Comp Line ( S i z e 1 ) := Res Tbl (Ind (Res Tbl, Average), Last); 300 end i f ; 301 i f Q = Cost then 302 Comp Line ( Cost 0 ) := Res Tbl (Ind (Res Tbl, Average), First); 303 Comp Line ( Cost 1 ) := Res Tbl (Ind (Res Tbl, Average), Last); 304 end i f ; 305 −−Put Line ( Tex , ”\ clearpage”); 306 Put Line ( Tex , 307 ”\ s u b s e c t i o n {”&Q Name & 308 ” Results of configuration ” & To String (X.Name) & 309 ”\ l a b e l { sectionresults” & Latex.To Label (Q Name) & Id & ”}}”); 310 New Line ( Tex ) ; 311 −− Put Line ( Tex , 312 −− ”See table ” & Ref (”tableresults” & Q Name) & 313 −− ” for the final results of ” &Q Name & 314 −− ”, table ” & Ref (”tableevolution” & Q Name) & 315 −− ” f o r a l l ” & Q Name & ” values in the experiment” & 316 −− ”, and figures ” & Ref (”figurecompact” & Q Name) & 317 −− ” and ” & Ref (”figuremovingavg” & Q Name) & 318 −− ” for charts of the evolution of ” &Q Name & 319 −− ” in the experiment.”); 320 Put Line (Tex, Latex.Tabular (Res Tbl , Q) ) ; 321 Put Line ( Tex , 322 ”\ s u b s e c t i o n {”&Q Name & 323 ” Evolution of configuration ” & To String (X.Name) & 324 ”\ l a b e l { sectionevolution” & Latex.To Label (Q Name) & Id & ”}}”); 325 Put Line (Tex, Latex.Tabular (Evo Tbl , Q) ) ; 326 Put Line ( Tex , 327 ”\ c e n t e r l i n e {\ includegraphics [width=0.75\ textwidth ] { ”& 328 Latex . To Filename (To String (X.Name) & ”−”&Q Name) & 329 ”−compact−chart . pdf }}”); 330 Put Line ( Tex , 331 ”\ c e n t e r l i n e {\ includegraphics [width=0.75\ textwidth ] { ”& 332 Latex . To Filename (To String (X.Name) & ”−”&Q Name) & 333 ”−moving−avg−chart . pdf }}”); 334 335 −− draw charts if required 336 i f Exists (Results Filename (X)) 337 and then Exists (Parameters Filename (X)) 338 and then Significant Components (Load (Parameters Filename (X))) 339 = Significant Components (X) 340 and then E x i s t s 168 APPENDIX B. PROGRAM LISTINGS
341 ( Latex . To Filename (To String (X.Name) & ”−”&Q Name) & ”−compact−chart . pdf ” ) 342 then 343 Ada . Text IO . Put Line (” chart exists for same params, no need to re−draw . ”); 344 else 345 Draw Compact Evolution Chart (Q) ; 346 Draw Moving Average Chart (Q) ; 347 end i f ; 348 end ; 349 −−−−−−−−−−−−−−−−−− 350 function Param (Name, Value : String) return S t r i n g i s 351 begin 352 return Name & ”&=&” & Value & ”\\” & EOL; 353 end ; 354 begin 355 Kasim2.Load (Cur.Sessions , Results Filename (X)); 356 Kasim2.Cur.Name := X.Name; 357 Kasim2. Compute Results ; 358 Put Line ( Tex , ”\ clearpage”); 359 Put Line ( Tex , 360 ”\ s e c t i o n { Configuration ” & To String (X.Name) & ”\ l a b e l {”& 361 ”section” & Id & ”}}”); 362 New Line ( Tex ) ; 363 Put Line (Tex, To String (X.Description) & EOL); 364 365 New Line ( Tex ) ; 366 Put Line ( Tex , 367 ”Configuration parameters:” & EOL & 368 EOL & 369 ”\ begin { t a b u l a r } { | l c l | } ” & EOL & 370 ”\ hline” & EOL & 371 Param (”Quantity of atoms”, Img (X.Qt Atoms ) ) & 372 Param (”Quantity of oracles”, Img (X.Qt Oracles ) ) & 373 Param (”Quantity of sessions”, Img (X.Qt Sessions)) & 374 Param (”Page size (number of items)”, Img (X.Page Size ) ) & 375 Param (”Quantity of expanded items”, Img (X.Qt Expanded)) & 376 Param (”Update travel matrix”, 377 Latex . To Name (Update Travels T ’Image (X.Update Travels))) & 378 Param (”Maximum duration of session”, Img (Real T (X. Max Duration))) & 379 Param (”Maximum session size”, Img (X.Max Qt Pages ) ) & 380 Param (”Maximum session cost”, Img (X.Max Cost ) ) & 381 Param (”Weight of Next links”, Img (X.Next Link Weight ) ) & 382 Param (”Weight of heading−paragraph Child links”, 383 Img (X.Child Link Weight Heading Heading ) ) & 384 Param (”Weight of heading−heading Child links”, 385 Img (X.Child Link Weight Heading Heading ) ) & 386 Param (”Residual weight”, Img (X.Residual Weight ) ) & 387 ”\ hline” & EOL & 388 ”\end{ t a b u l a r }” & EOL) ; 389 390 declare 391 Outcome Tbl : Outcome Table T := Make Outcome Table ; 392 S i z e T b l : Result Table T := Make Result Table (Size); 393 Cost Tbl : Result Table T := Make Result Table (Cost); 394 Comp Line : Comparate Line T; B.9. PACKAGE KASIM2.COMPARATE (BODY) 169
395 begin 396 Comp Line (Success Rate ) := 397 Outcome Tbl (Ind (Outcome Tbl , Micro Avg), Success Rate ) ; 398 Comp Line ( S i z e 0 ) := S i z e Tbl (Ind (Size Tbl , Average), First); 399 Comp Line ( S i z e 1 ) := S i z e Tbl (Ind (Size Tbl , Average), Last); 400 Comp Line ( Cost 0 ) := Cost Tbl (Ind (Cost Tbl, Average), First); 401 Comp Line ( Cost 1 ) := Cost Tbl (Ind (Cost Tbl, Average), Last); 402 Put Line ( Tex , 403 EOL & 404 ”\ d e f \ imagetop#1{\vtop {\ n u l l \hbox{#1}}}” & EOL & 405 EOL & 406 ”\ begin { t a b u l a r }{ l l }” & EOL & 407 ”Principal results: & Cost evolution: \\ ” & EOL & 408 ”\ imagetop {”& 409 ”\ begin { t a b u l a r } { | l c l | } ” & EOL & 410 ”\ hline” & EOL & 411 Param (Img (Success Rate), Img (Comp Line (Success Rate ) ) ) & 412 Param (Img (Size 0), Img (Comp Line ( S i z e 0 ) ) ) & 413 Param (Img (Size 1), Img (Comp Line ( S i z e 1 ) ) ) & 414 Param (Img (Size Gain), Img (Comp Line ( S i z e 0 ) / Comp Line ( S i z e 1 ) ) ) & 415 Param (Img (Cost 0), Img (Comp Line ( Cost 0 ) ) ) & 416 Param (Img (Cost 1), Img (Comp Line ( Cost 1 ) ) ) & 417 Param (Img (Cost Gain), Img (Comp Line ( Cost 0 ) / Comp Line ( Cost 1 ) ) ) & 418 ”\ hline” & EOL & 419 ”\end{ t a b u l a r }}” & EOL & 420 ”&” & EOL & 421 ”\ imagetop {”& 422 ”\ includegraphics [width=9.5cm] { ”& 423 Latex . To Filename (To String (X.Name) & ”−c o s t ” ) & 424 ”−moving−avg−chart . pdf }}\\ ” & EOL & 425 ”\end{ t a b u l a r }” & EOL) ; 426 end ; 427 New Line ( Tex ) ; 428 Put Line ( Tex , 429 ”See ensuing subsections for more result statistics and detailed evolution data.” & EOL & 430 ”\ vfill” & EOL); 431 −− Put Line (Tex, ”See sections ” & Ref (”tableoutcomes”) & ” for oracle details and session outcomes (success or failure). See sections ” & Ref (” sectionduratio”) & ”, ” & Ref (”sectionsize”) & ”, ” & Ref (”sectioncost”) & ” for tables and figures depicting the results of each quality measure Duratio, Size, Cost, respectively.”); 432 New Line ( Tex ) ; 433 declare 434 Outcome Tbl : Outcome Table T := Make Outcome Table ; 435 begin 436 Comp Line (Success Rate ) := 437 Outcome Tbl (Ind (Outcome Tbl , Micro Avg), Success Rate ) ; 438 Put Line ( Tex , 439 ”\ s u b s e c t i o n {Outcomes of configuration ” & To String (X.Name) & 440 ”\ l a b e l { sectionoutcomes” & Latex.To Label ( To String (X.Name)) & 441 ”}}” & EOL & 442 Latex.Tabular (Outcome Tbl ) ) ; 443 for Q in Quality Measure T loop 170 APPENDIX B. PROGRAM LISTINGS
444 Report (Q) ; 445 end loop ; 446 end ; 447 end ; 448 449 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 450 −− REPORT CONFIGURATION, VERSION 2 −− 451 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 452 453 function C o n f i g S e c t i o n I d I n Chapter (Batchname : String; X : Configuration T) return S t r i n g i s 454 begin 455 return Latex . To Label (”sectioninchapter” & Batchname & To String (X.Name)); 456 end ; 457 458 function C o n f i g S e c t i o n I d I n Chapter (X : Configuration T) return S t r i n g i s 459 begin 460 return C o n f i g S e c t i o n I d I n C h a p t e r ( To String (Current Name ) , X) ; 461 end ; 462 463 function C o n f i g S e c t i o n I d I n Annex (X : Configuration T; Suffix : String := ””) return S t r i n g i s 464 begin 465 return Latex . To Label (”sectioninannex” & To String (X.Name) & Suffix); 466 end ; 467 468 function Obs Label (X : Configuration T) return S t r i n g i s 469 begin 470 return ”\ obs” & Latex.To Label (Config S e c t i o n I d I n Chapter (X)); 471 end ; 472 473 procedure Report2 (Batchname : String; X : Configuration T ; Comp Line : out Comparate Line T) i s 474 −− Outputs LaTeX files . 475 −− Creates the graph files with BMP and converts to PDF. 476 477 −−−−−−−−−−−−−−−−−− 478 −− REPORT QUALITY 479 function Quality Name (Q : Quality Measure T) return S t r i n g i s 480 Z : String := Latex.To Name (Quality Measure T ’Image (Q)); 481 begin 482 i f Z = ”Duratio” then return ”Duration”; 483 else return Z; 484 end i f ; 485 end ; 486 procedure Report (Q : Quality Measure T) i s 487 Q Name : String := Quality Name (Q) ; 488 Res Tbl : Result Table T := Make Result Table (Q) ; 489 Evo Tbl : Evolution Table T := Make Evolution Table (Q) ; 490 begin 491 i f Q = S i z e then 492 Comp Line ( S i z e 0 ) := Res Tbl (Ind (Res Tbl, Average), First); 493 Comp Line ( S i z e 1 ) := Res Tbl (Ind (Res Tbl, Average), Last); 494 end i f ; 495 i f Q = Cost then 496 Comp Line ( Cost 0 ) := Res Tbl (Ind (Res Tbl, Average), First); B.9. PACKAGE KASIM2.COMPARATE (BODY) 171
497 Comp Line ( Cost 1 ) := Res Tbl (Ind (Res Tbl, Average), Last); 498 end i f ; 499 −−Put Line ( Tex , ”\ clearpage”); 500 Put Line ( Tex Annex , 501 ”\ s u b s e c t i o n {”&Q Name & 502 ” Results of configuration ” & To String (X.Name) & ”}”); 503 New Line ( Tex Annex ) ; 504 Put Line ( Tex Annex, Latex.Tabular (Res Tbl , Q) ) ; 505 Put Line ( Tex Annex , 506 ”\ s u b s e c t i o n {”&Q Name & 507 ” Evolution of configuration ” & To String (X.Name) & ”}”); 508 Put Line ( Tex Annex, Latex.Tabular (Evo Tbl , Q) ) ; 509 declare 510 Prefix : String := Latex.To Filename (To String (X.Name) & ”−”&Q Name) ; 511 begin 512 i f Prefix = ”structural −−trimmed−t re e −s i z e ” then 513 Put Line ( Tex Annex , 514 ”(Due to an yet unsolved bug in the chart generation software, ” & 515 ” the compact and moving average charts for this table are not included. ” & 516 ” Note that the values are trivial , making the charts not stricly r e q u i r e d ” & 517 ” for interpretation. Probably it is because the values are trivial ”& 518 ” that the chart generation bug manifests itself.)”); 519 else 520 Put Line ( Tex Annex , 521 ”\ c e n t e r l i n e {\ includegraphics [width=0.75\ textwidth ] { ”& 522 P r e f i x & ”−compact−chart . pdf }}”); 523 Put Line ( Tex Annex , 524 ”\ c e n t e r l i n e {\ includegraphics [width=0.75\ textwidth ] { ”& 525 P r e f i x & ”−moving−avg−chart . pdf }}”); 526 end i f ; 527 end ; 528 529 −− draw charts if required 530 i f Exists (Results Filename (X)) 531 and then Exists (Parameters Filename (X)) 532 and then Significant Components (Load (Parameters Filename (X))) 533 = Significant Components (X) 534 and then E x i s t s 535 ( Latex . To Filename (To String (X.Name) & ”−”&Q Name) & ”−compact−chart . pdf ” ) 536 then 537 Ada . Text IO . Put Line (” chart exists for same params, no need to re−draw . ”); 538 else 539 Draw Compact Evolution Chart (Q) ; 540 Draw Moving Average Chart (Q) ; 541 end i f ; 542 end ; 543 −−−−−−−−−−−−−−−−−− 544 function Param (Name, Value : String) return S t r i n g i s 545 begin 546 return Name & ”&=&” & Value & ”\\” & EOL; 547 end ; 172 APPENDIX B. PROGRAM LISTINGS
548 549 procedure Put Dashboard (Tex : File Type ) i s 550 begin 551 Put Line ( Tex , 552 ”Configuration parameters:” & EOL & 553 EOL & 554 ”\ begin { t a b u l a r } { | l c l | } ” & EOL & 555 ”\ hline” & EOL & 556 Param (”Quantity of atoms”, Img (X.Qt Atoms ) ) & 557 Param (”Quantity of oracles”, Img (X.Qt Oracles ) ) & 558 Param (”Quantity of sessions”, Img (X.Qt Sessions)) & 559 Param (”Page size (number of items)”, Img (X.Page Size ) ) & 560 Param (”Quantity of expanded items”, Img (X.Qt Expanded)) & 561 Param (”Update travel matrix strategy”, 562 Latex . To Name (Update Travels T ’Image (X.Update Travels))) & 563 Param (”Maximum duration of session”, Img (Real T (X. Max Duration))) & 564 Param (”Maximum session size”, Img (X.Max Qt Pages ) ) & 565 Param (”Maximum session cost”, Img (X.Max Cost ) ) & 566 Param (”Weight of Next links”, Img (X.Next Link Weight ) ) & 567 Param (”Weight of heading−to−paragraph Child links”, 568 Img (X.Child Link Weight Heading Heading ) ) & 569 Param (”Weight of heading−to−heading Child links”, 570 Img (X.Child Link Weight Heading Heading ) ) & 571 Param (”Residual weight (all links)”, 572 Img (X.Residual Weight ) ) & 573 ”\ hline” & EOL & 574 ”\end{ t a b u l a r }” & EOL) ; 575 576 declare 577 Outcome Tbl : Outcome Table T := Make Outcome Table ; 578 S i z e T b l : Result Table T := Make Result Table (Size); 579 Cost Tbl : Result Table T := Make Result Table (Cost); 580 Comp Line : Comparate Line T; 581 begin 582 Comp Line (Success Rate ) := 583 Outcome Tbl (Ind (Outcome Tbl , Micro Avg), Success Rate ) ; 584 Comp Line ( S i z e 0 ) := S i z e Tbl (Ind (Size Tbl , Average), First); 585 Comp Line ( S i z e 1 ) := S i z e Tbl (Ind (Size Tbl , Average), Last); 586 Comp Line ( Cost 0 ) := Cost Tbl (Ind (Cost Tbl, Average), First); 587 Comp Line ( Cost 1 ) := Cost Tbl (Ind (Cost Tbl, Average), Last); 588 Put Line ( Tex , 589 EOL & 590 ”\ d e f \ imagetop#1{\vtop {\ n u l l \hbox{#1}}}” & EOL & 591 EOL & 592 ”\ begin { t a b u l a r }{ l l }” & EOL & 593 ”Principal results: & Cost evolution: \\ ” & EOL & 594 ”\ imagetop {”& 595 ”\ begin { t a b u l a r } { | l c l | } ” & EOL & 596 ”\ hline” & EOL & 597 Param (Img (Success Rate), Img (Comp Line (Success Rate ) ) ) & 598 Param (Img (Size 0), Img (Comp Line ( S i z e 0 ) ) ) & 599 Param (Img (Size 1), Img (Comp Line ( S i z e 1 ) ) ) & 600 Param (Img (Size Gain), Img (Comp Line ( S i z e 0 ) / Comp Line ( S i z e 1 )))& 601 Param (Img (Cost 0), Img (Comp Line ( Cost 0 ) ) ) & 602 Param (Img (Cost 1), Img (Comp Line ( Cost 1 ) ) ) & B.9. PACKAGE KASIM2.COMPARATE (BODY) 173
603 Param (Img (Cost Gain), Img (Comp Line ( Cost 0 ) / Comp Line ( Cost 1 )))& 604 ”\ hline” & EOL & 605 ”\end{ t a b u l a r }}” & EOL & 606 ”&” & EOL & 607 ”\ imagetop {”& 608 ”\ includegraphics [width=9.5cm] { ”& 609 Latex . To Filename (To String (X.Name) & ”−c o s t ” ) & 610 ”−moving−avg−chart . pdf }}\\ ” & EOL & 611 ”\end{ t a b u l a r }” & EOL) ; 612 end ; 613 end Put Dashboard ; 614 begin 615 Kasim2.Load (Cur.Sessions , Results Filename (X)); 616 Kasim2.Cur.Name := X.Name; 617 Kasim2. Compute Results ; 618 619 declare 620 Outcome Tbl : Outcome Table T := Make Outcome Table ; 621 S i z e T b l : Result Table T := Make Result Table (Size); 622 Cost Tbl : Result Table T := Make Result Table (Cost); 623 begin 624 Comp Line (Success Rate ) := 625 Outcome Tbl (Ind (Outcome Tbl , Micro Avg), Success Rate ) ; 626 Comp Line ( S i z e 0 ) := S i z e Tbl (Ind (Size Tbl , Average), First); 627 Comp Line ( S i z e 1 ) := S i z e Tbl (Ind (Size Tbl , Average), Last); 628 Comp Line ( Cost 0 ) := Cost Tbl (Ind (Cost Tbl, Average), First); 629 Comp Line ( Cost 1 ) := Cost Tbl (Ind (Cost Tbl, Average), Last); 630 end ; 631 632 Put Line ( Tex , ”\ clearpage”); 633 Put Line ( Tex , 634 ”\ s e c t i o n { Configuration ” & To String (X.Name) & ”\ l a b e l {”& 635 C o n f i g S e c t i o n I d I n Chapter (Batchname, X) & ”}}”); 636 New Line ( Tex ) ; 637 Put Line (Tex, To String (X.Description) & EOL); 638 New Line ( Tex ) ; 639 Put Dashboard (Tex); 640 New Line ( Tex ) ; 641 Put Line ( Tex , 642 ”See ” & Ref (Config S e c t i o n I d I n A n n e x (X) ) & 643 ” for complete result statistics and detailed evolution data.” & EOL & 644 ”\ vfill” & EOL); 645 New Line ( Tex ) ; 646 647 Put Line ( Tex , 648 ”\ Beginobservations” & EOL & 649 Obs Label (X) & EOL & 650 ”\ Endobservations” & EOL); 651 652 i f not Configuration Set.Contains (X) then 653 i f Configuration S e t . Is Empty then 654 Create (File => Tex Annex, Name => Annex Filename , Mode => Append File ) ; 655 end i f ; 656 i f not Is Open ( Tex Annex ) then 657 Open ( F i l e => Tex Annex, Name => Annex Filename , Mode => Append File ) ; 174 APPENDIX B. PROGRAM LISTINGS
658 end i f ; 659 Configuration Set.Insert (X); 660 declare 661 Outcome Tbl : Outcome Table T := Make Outcome Table ; 662 begin 663 Put Line ( Tex Annex , 664 ”\ s e c t i o n {Complete results of configuration ” & To String (X.Name) & 665 ”\ l a b e l {” & C o n f i g S e c t i o n I d I n A n n e x (X) & ”}}” & EOL) ; 666 667 Put Line ( Tex Annex , To String (X.Description) & EOL); 668 669 Put Line ( Tex Annex , ”\ s u b s e c t i o n {Dashboard}” & EOL) ; 670 Put Dashboard (Tex Annex ) ; 671 New Line ( Tex Annex ) ; 672 Put Line ( Tex Annex , 673 ”\ s u b s e c t i o n {Outcomes of configuration ” & To String (X.Name) & ”}”& EOL & 674 Latex.Tabular (Outcome Tbl ) ) ; 675 for Q in Quality Measure T loop 676 Report (Q) ; 677 end loop ; 678 end ; 679 Close ( Tex Annex ) ; 680 end i f ; 681 682 i f Create Index then 683 Put Line ( Tex Index , 684 ”% Configuration ” & To String (X.Name) & ” in batch ” & Batchname & EOL & 685 ”\newcommand{” & Obs Label (X) & ”}{” & EOL & 686 ”(Observations, to be written.)” & EOL & 687 ”}”); 688 New Line ( Tex Index ) ; 689 end i f ; 690 end ; 691 692 procedure Report2a (Batchname : String; X : Configuration T ; Comp Line : out Comparate Line T) i s 693 −− Outputs LaTeX files . 694 −− Creates the graph files with BMP and converts to PDF. 695 696 −−−−−−−−−−−−−−−−−− 697 −− REPORT QUALITY 698 function Quality Name (Q : Quality Measure T) return S t r i n g i s 699 Z : String := Latex.To Name (Quality Measure T ’Image (Q)); 700 begin 701 i f Z = ”Duratio” then return ”Duration”; 702 else return Z; 703 end i f ; 704 end ; 705 procedure Report (Q : Quality Measure T) i s 706 Q Name : String := Quality Name (Q) ; 707 Res Tbl : Result Table T := Make Result Table (Q) ; 708 Evo Tbl : Evolution Table T := Make Evolution Table (Q) ; 709 begin 710 i f Q = S i z e then B.9. PACKAGE KASIM2.COMPARATE (BODY) 175
711 Comp Line ( S i z e 0 ) := Res Tbl (Ind (Res Tbl, Average), First); 712 Comp Line ( S i z e 1 ) := Res Tbl (Ind (Res Tbl, Average), Last); 713 end i f ; 714 i f Q = Cost then 715 Comp Line ( Cost 0 ) := Res Tbl (Ind (Res Tbl, Average), First); 716 Comp Line ( Cost 1 ) := Res Tbl (Ind (Res Tbl, Average), Last); 717 end i f ; 718 −−Put Line ( Tex , ”\ clearpage”); 719 Put Line ( Tex Annex , 720 ”\ s u b s e c t i o n ∗{”&Q Name & 721 ” Results of configuration ” & To String (X.Name) & ”}”); 722 New Line ( Tex Annex ) ; 723 Put Line ( Tex Annex, Latex.Tabular (Res Tbl , Q) ) ; 724 Put Line ( Tex Annex , 725 ”\ s u b s e c t i o n ∗{”&Q Name & 726 ” Evolution of configuration ” & To String (X.Name) & ”}”); 727 Put Line ( Tex Annex, Latex.Tabular (Evo Tbl , Q) ) ; 728 declare 729 Prefix : String := Latex.To Filename (To String (X.Name) & ”−”&Q Name) ; 730 begin 731 i f Prefix = ”structural −−trimmed−t re e −s i z e ” then 732 Put Line ( Tex Annex , 733 ”(Due to an yet unsolved bug in the chart generation software, ” & 734 ” the compact and moving average charts for this table are not included. ” & 735 ” Note that the values are trivial , making the charts not stricly r e q u i r e d ” & 736 ” for interpretation. Probably it is because the values are trivial ”& 737 ” that the chart generation bug manifests itself.)”); 738 else 739 Put Line ( Tex Annex , 740 ”\ c e n t e r l i n e {\ includegraphics [width=0.75\ textwidth ] { ”& 741 P r e f i x & ”−compact−chart . pdf }}”); 742 Put Line ( Tex Annex , 743 ”\ c e n t e r l i n e {\ includegraphics [width=0.75\ textwidth ] { ”& 744 P r e f i x & ”−moving−avg−chart . pdf }}”); 745 end i f ; 746 end ; 747 748 −− draw charts if required 749 i f Exists (Results Filename (X)) 750 and then Exists (Parameters Filename (X)) 751 and then Significant Components (Load (Parameters Filename (X))) 752 = Significant Components (X) 753 and then E x i s t s 754 ( Latex . To Filename (To String (X.Name) & ”−”&Q Name) & ”−compact−chart . pdf ” ) 755 then 756 Ada . Text IO . Put Line (” chart exists for same params, no need to re−draw . ”); 757 else 758 Draw Compact Evolution Chart (Q) ; 759 Draw Moving Average Chart (Q) ; 760 end i f ; 761 end ; 176 APPENDIX B. PROGRAM LISTINGS
762 −−−−−−−−−−−−−−−−−− 763 function Param (Name, Value : String) return S t r i n g i s 764 begin 765 return Name & ”&=&” & Value & ”\\” & EOL; 766 end ; 767 768 procedure Put Dashboard (Tex : File Type ) i s 769 begin 770 Put Line ( Tex , 771 ”Configuration parameters:” & EOL & 772 EOL & 773 ”\ begin { t a b u l a r } { | l c l | } ” & EOL & 774 ”\ hline” & EOL & 775 Param (”Quantity of atoms”, Img (X.Qt Atoms ) ) & 776 Param (”Quantity of oracles”, Img (X.Qt Oracles ) ) & 777 Param (”Quantity of sessions”, Img (X.Qt Sessions)) & 778 Param (”Page size (number of items)”, Img (X.Page Size ) ) & 779 Param (”Quantity of expanded items”, Img (X.Qt Expanded)) & 780 Param (”Update travel matrix strategy”, 781 Latex . To Name (Update Travels T ’Image (X.Update Travels))) & 782 Param (”Maximum duration of session”, Img (Real T (X. Max Duration))) & 783 Param (”Maximum session size”, Img (X.Max Qt Pages ) ) & 784 Param (”Maximum session cost”, Img (X.Max Cost ) ) & 785 Param (”Weight of Next links”, Img (X.Next Link Weight ) ) & 786 Param (”Weight of heading−to−paragraph Child links”, 787 Img (X.Child Link Weight Heading Heading ) ) & 788 Param (”Weight of heading−to−heading Child links”, 789 Img (X.Child Link Weight Heading Heading ) ) & 790 Param (”Residual weight (all links)”, 791 Img (X.Residual Weight ) ) & 792 ”\ hline” & EOL & 793 ”\end{ t a b u l a r }” & EOL) ; 794 795 declare 796 Outcome Tbl : Outcome Table T := Make Outcome Table ; 797 S i z e T b l : Result Table T := Make Result Table (Size); 798 Cost Tbl : Result Table T := Make Result Table (Cost); 799 Comp Line : Comparate Line T; 800 begin 801 Comp Line (Success Rate ) := 802 Outcome Tbl (Ind (Outcome Tbl , Micro Avg), Success Rate ) ; 803 Comp Line ( S i z e 0 ) := S i z e Tbl (Ind (Size Tbl , Average), First); 804 Comp Line ( S i z e 1 ) := S i z e Tbl (Ind (Size Tbl , Average), Last); 805 Comp Line ( Cost 0 ) := Cost Tbl (Ind (Cost Tbl, Average), First); 806 Comp Line ( Cost 1 ) := Cost Tbl (Ind (Cost Tbl, Average), Last); 807 Put Line ( Tex , 808 EOL & 809 ”\ d e f \ imagetop#1{\vtop {\ n u l l \hbox{#1}}}” & EOL & 810 EOL & 811 ”\ begin { t a b u l a r }{ l l }” & EOL & 812 ”Principal results: & Cost evolution: \\ ” & EOL & 813 ”\ imagetop {”& 814 ”\ begin { t a b u l a r } { | l c l | } ” & EOL & 815 ”\ hline” & EOL & 816 Param (Img (Success Rate), Img (Comp Line (Success Rate ) ) ) & 817 Param (Img (Size 0), Img (Comp Line ( S i z e 0 ) ) ) & B.9. PACKAGE KASIM2.COMPARATE (BODY) 177
818 Param (Img (Size 1), Img (Comp Line ( S i z e 1 ) ) ) & 819 Param (Img (Size Gain), Img (Comp Line ( S i z e 0 ) / Comp Line ( S i z e 1 )))& 820 Param (Img (Cost 0), Img (Comp Line ( Cost 0 ) ) ) & 821 Param (Img (Cost 1), Img (Comp Line ( Cost 1 ) ) ) & 822 Param (Img (Cost Gain), Img (Comp Line ( Cost 0 ) / Comp Line ( Cost 1 )))& 823 ”\ hline” & EOL & 824 ”\end{ t a b u l a r }}” & EOL & 825 ”&” & EOL & 826 ”\ imagetop {”& 827 ”\ includegraphics [width=9.5cm] { ”& 828 Latex . To Filename (To String (X.Name) & ”−c o s t ” ) & 829 ”−moving−avg−chart . pdf }}\\ ” & EOL & 830 ”\end{ t a b u l a r }” & EOL) ; 831 end ; 832 end Put Dashboard ; 833 begin 834 Kasim2.Load (Cur.Sessions , Results Filename (X)); 835 Kasim2.Cur.Name := X.Name; 836 Kasim2. Compute Results ; 837 838 declare 839 Outcome Tbl : Outcome Table T := Make Outcome Table ; 840 S i z e T b l : Result Table T := Make Result Table (Size); 841 Cost Tbl : Result Table T := Make Result Table (Cost); 842 begin 843 Comp Line (Success Rate ) := 844 Outcome Tbl (Ind (Outcome Tbl , Micro Avg), Success Rate ) ; 845 Comp Line ( S i z e 0 ) := S i z e Tbl (Ind (Size Tbl , Average), First); 846 Comp Line ( S i z e 1 ) := S i z e Tbl (Ind (Size Tbl , Average), Last); 847 Comp Line ( Cost 0 ) := Cost Tbl (Ind (Cost Tbl, Average), First); 848 Comp Line ( Cost 1 ) := Cost Tbl (Ind (Cost Tbl, Average), Last); 849 end ; 850 851 Put Line ( Tex , ”\ clearpage”); 852 Put Line ( Tex , 853 ”\ s e c t i o n { Configuration ” & To String (X.Name) & ”\ l a b e l {”& 854 C o n f i g S e c t i o n I d I n Chapter (Batchname, X) & ”}}”); 855 New Line ( Tex ) ; 856 Put Line (Tex, To String (X.Description) & EOL); 857 New Line ( Tex ) ; 858 Put Dashboard (Tex); 859 New Line ( Tex ) ; 860 Put Line ( Tex , 861 ”See ” & Ref (Config S e c t i o n I d I n A n n e x (X) ) & 862 ” for complete result statistics and detailed evolution data.” & EOL & 863 ”\ vfill” & EOL); 864 New Line ( Tex ) ; 865 866 Put Line ( Tex , 867 ”\ Beginobservations” & EOL & 868 Obs Label (X) & EOL & 869 ”\ Endobservations” & EOL); 870 871 i f not Configuration Set.Contains (X) then 178 APPENDIX B. PROGRAM LISTINGS
872 i f Configuration S e t . Is Empty then 873 Create (File => Tex Annex, Name => Annex Filename , Mode => Append File ) ; 874 end i f ; 875 i f not Is Open ( Tex Annex ) then 876 Open ( F i l e => Tex Annex, Name => Annex Filename , Mode => Append File ) ; 877 end i f ; 878 Configuration Set.Insert (X); 879 declare 880 Outcome Tbl : Outcome Table T := Make Outcome Table ; 881 begin 882 Put Line ( Tex Annex , 883 ”\ s e c t i o n {Complete results of configuration ” & To String (X.Name) & 884 ”\ l a b e l {” & C o n f i g S e c t i o n I d I n A n n e x (X) & ”}}” & EOL) ; 885 886 Put Line ( Tex Annex , To String (X.Description) & EOL); 887 888 −−Put Line ( Tex Annex , ”\ s u b s e c t i o n ∗{ Dashboard }” & EOL) ; 889 Put Dashboard (Tex Annex ) ; 890 New Line ( Tex Annex ) ; 891 Put Line ( Tex Annex , 892 ”\ s u b s e c t i o n ∗{Outcomes of configuration ” & To String (X.Name) & ”}”& EOL & 893 Latex.Tabular (Outcome Tbl ) ) ; 894 for Q in Quality Measure T loop 895 Report (Q) ; 896 end loop ; 897 end ; 898 Close ( Tex Annex ) ; 899 end i f ; 900 901 i f Create Index then 902 Put Line ( Tex Index , 903 ”% Configuration ” & To String (X.Name) & ” in batch ” & Batchname & EOL & 904 ”\newcommand{” & Obs Label (X) & ”}{” & EOL & 905 ”(Observations, to be written.)” & EOL & 906 ”}”); 907 New Line ( Tex Index ) ; 908 end i f ; 909 end ; 910 911 −−−−−−−−−−−−−−−−−−−− 912 −− REPORT BATCH −− 913 −−−−−−−−−−−−−−−−−−−− 914 915 function Tabular (Batch : Configuration Array T; Tbl : Comparate Table T) 916 return S t r i n g i s 917 U : Unbounded String ; 918 Tot : array ( Comparate Field T) of Real T := ( others => 0 . 0 ) ; 919 Min : array ( Comparate Field T) of Positive := ( others => 1) ; 920 Max : array ( Comparate Field T) of Positive := ( others => 1) ; 921 begin 922 Append (U, 923 ”\ footnotesize” & EOL & 924 ”\ begin { c e n t e r }” & EOL & 925 ”\ begin { adjustbox }{ width={\textwidth } ,keepaspectratio}%” & EOL & B.9. PACKAGE KASIM2.COMPARATE (BODY) 179
926 ”\ begin { t a b u l a r }{ l p{5cm}” & Tbl (1)’Length ∗ ” r ” & ”}” & EOL & 927 ”\ hline” & EOL & 928 ”&Configuration”) ; 929 for I in Comparate Field T loop 930 Append (U, ” & ” & Img (I)); 931 end loop ; 932 Append (U, 933 ” \\” & EOL & 934 ”\ hline” & EOL); 935 936 −− find maxs to bold them 937 for F i e l d in Comparate Field T loop 938 for Config in 2 .. Tbl’Length loop 939 i f Tbl (Config) (Field) < Tbl (Min (Field)) (Field) then 940 Min (Field) := Config; 941 end i f ; 942 i f Tbl (Config) (Field) > Tbl (Max (Field)) (Field) then 943 Max (Field) := Config; 944 end i f ; 945 end loop ; 946 end loop ; 947 948 for I in Tbl ’ Range( 1 ) loop 949 Append (U, 950 ”\ scriptsize” & 951 ”\ r u l e [ −1.5\ baselineskip] { 0 pt }{3\ baselineskip }”& 952 ”\ r e f {” & C o n f i g S e c t i o n I d I n Chapter (Batch (I)) & ”}&”& 953 ”\ s c r i p t s i z e {” & Latex . To Name ( To String (Batch (I).Name)) & ”}” & EOL) ; 954 for J in Comparate Field T loop 955 Append (U, ”&\ footnotesize {”); 956 i f ( ( J = S i z e 1 or J = Cost 1 ) and Min ( J ) = I ) 957 or ( ( J = Size Gain or J = Cost Gain ) and Max ( J ) = I ) 958 then 959 Append (U, ”$\ bigstar$”); 960 end i f ; 961 i f J /= Success Rate and (Max ( J ) = I or Min ( J ) = I ) then 962 Append (U, ”\Em{” & Trim (Img (Tbl (I) (J))) & ”}”); 963 else 964 Append (U, Trim (Img (Tbl (I) (J)))); 965 end i f ; 966 Append (U, ”}”); 967 end loop ; 968 Append (U, ” \\” & EOL) ; 969 end loop ; 970 Append (U, ”\ hline” & EOL); 971 972 for I in Tbl ’ Range ( 1 ) loop 973 for J in Comparate Field T loop 974 Tot (J) := Tot (J) + Tbl (I) (J); 975 end loop ; 976 end loop ; 977 978 Append (U, ”&Total”); 979 for I in Comparate Field T loop 980 Append (U, ”&” & Img (Tot (I))); 981 end loop ; 180 APPENDIX B. PROGRAM LISTINGS
982 Append (U, ”\\” & EOL) ; 983 984 Append (U, ”&Average”); 985 for I in Comparate Field T loop 986 Append (U, ”&” & Img (Tot (I) / Real T (Tbl’Length))); 987 end loop ; 988 Append (U, ”\\” & EOL) ; 989 990 Append (U, 991 ”\ hline” & EOL & 992 ”\end{ t a b u l a r }” & EOL & 993 ”\end{ adjustbox }” & EOL & 994 ”\end{ c e n t e r }” & EOL & 995 ”\ normalsize”); 996 return To String (U) ; 997 end ; 998 999 function Tabular3 (Batch : Configuration Array T; Tbl : Comparate Table T) 1000 −− Like Tabular, but for Report3. 1001 return S t r i n g i s 1002 U : Unbounded String ; 1003 Tot : array ( Comparate Field T) of Real T := ( others => 0 . 0 ) ; 1004 Min : array ( Comparate Field T) of Positive := ( others => 1) ; 1005 Max : array ( Comparate Field T) of Positive := ( others => 1) ; 1006 begin 1007 Append (U, 1008 ”\ footnotesize” & EOL & 1009 ”\ begin { c e n t e r }” & EOL & 1010 −−”\ begin { a d j u s t b o x }{ width={\ t e x t w i d t h } ,keepaspectratio }%” & EOL & 1011 ”\ begin { t a b u l a r }{p{5cm}” & Tbl (1)’Length ∗ ” r ” & ”}” & EOL & 1012 ”\ hline” & EOL & 1013 ”Configuration”); 1014 for I in Comparate Field T loop 1015 Append (U, ” & ” & Img (I)); 1016 end loop ; 1017 Append (U, 1018 ” \\” & EOL & 1019 ”\ hline” & EOL); 1020 1021 −− find maxs to bold them 1022 for F i e l d in Comparate Field T loop 1023 for Config in 2 .. Tbl’Length loop 1024 i f Tbl (Config) (Field) < Tbl (Min (Field)) (Field) then 1025 Min (Field) := Config; 1026 end i f ; 1027 i f Tbl (Config) (Field) > Tbl (Max (Field)) (Field) then 1028 Max (Field) := Config; 1029 end i f ; 1030 end loop ; 1031 end loop ; 1032 1033 for I in Tbl ’ Range( 1 ) loop 1034 Append (U, 1035 ”\ scriptsize” & 1036 ”\ r u l e [ −1.5\ baselineskip] { 0 pt }{3\ baselineskip }”& 1037 −−”\ r e f {” & C o n f i g S e c t i o n I d I n Chapter (Batch (I)) & ”}&”& B.9. PACKAGE KASIM2.COMPARATE (BODY) 181
1038 ”\ s c r i p t s i z e {” & Latex . To Name ( To String (Batch (I).Name)) & ”}” & EOL) ; 1039 for J in Comparate Field T loop 1040 Append (U, ”&\ footnotesize {”); 1041 i f ( ( J = S i z e 1 or J = Cost 1 ) and Min ( J ) = I ) 1042 or ( ( J = Size Gain or J = Cost Gain ) and Max ( J ) = I ) 1043 then 1044 Append (U, ”$\ bigstar$”); 1045 end i f ; 1046 i f J /= Success Rate and (Max ( J ) = I or Min ( J ) = I ) then 1047 Append (U, ”\Em{” & Trim (Img (Tbl (I) (J))) & ”}”); 1048 else 1049 Append (U, Trim (Img (Tbl (I) (J)))); 1050 end i f ; 1051 Append (U, ”}”); 1052 end loop ; 1053 Append (U, ” \\” & EOL) ; 1054 end loop ; 1055 Append (U, ”\ hline” & EOL); 1056 1057 for I in Tbl ’ Range ( 1 ) loop 1058 for J in Comparate Field T loop 1059 Tot (J) := Tot (J) + Tbl (I) (J); 1060 end loop ; 1061 end loop ; 1062 1063 Append (U, ”Total”); 1064 for I in Comparate Field T loop 1065 Append (U, ”&” & Img (Tot (I))); 1066 end loop ; 1067 Append (U, ”\\” & EOL) ; 1068 1069 Append (U, ”Average”); 1070 for I in Comparate Field T loop 1071 Append (U, ”&” & Img (Tot (I) / Real T (Tbl’Length))); 1072 end loop ; 1073 Append (U, ”\\” & EOL) ; 1074 1075 Append (U, 1076 ”\ hline” & EOL & 1077 ”\end{ t a b u l a r }” & EOL & 1078 −−”\ end{ a d j u s t b o x }” & EOL & 1079 ”\end{ c e n t e r }” & EOL & 1080 ”\ normalsize”); 1081 return To String (U) ; 1082 end ; 1083 1084 procedure Report (Filename : String; Batch : Configuration Array T) i s 1085 −− Creates a LaTeX file for the batch, 1086 −− with subsections for each configuration. 1087 1088 Comp Tbl : Comparate Table T ( Batch ’ Range); 1089 1090 function Ref (Label : String) return S t r i n g i s 1091 begin 1092 return 1093 ”\ r e f {” & Latex . To Label(Label) & 182 APPENDIX B. PROGRAM LISTINGS
1094 ”} ( page \ p a g e r e f {” & Latex . To Label(Label) & ” })”; 1095 end ; 1096 1097 procedure P u t Description Item (X : Configuration T) i s 1098 Id : String := Latex.To Label ( To String (X.Name)); 1099 begin 1100 Put Line ( Tex , 1101 ”\ item [ \ r e f { section” & Id & ”} ” & To String (X.Name) & ”.] ” & 1102 To String (X.Description) & 1103 ” ( page \ p a g e r e f { section” & Id & ” })”); 1104 end ; 1105 1106 procedure P u t Description Item (I : Positive) i s 1107 begin 1108 P u t Description Item (Batch (I)); 1109 end ; 1110 1111 Subsect : String := Latex.To Label (”subsectionbatch” & Filename); 1112 begin 1113 Create (Tex, Out File , ”kasim2−batch−” & Filename & ”.tex”); 1114 Put Line (Tex, ”This section describes a comparate experiment of the following configurations:”); 1115 New Line ( Tex ) ; 1116 Put Line ( Tex , ”\ begin { description }”); 1117 for I in Batch ’ Range loop 1118 P u t Description I t e m ( I ) ; 1119 end loop ; 1120 Put Line ( Tex , 1121 ”\ item [ \ r e f {” & Subsect & 1122 ”} Comparate results.] Integrative report of all items above (page \ p a g e r e f {” & 1123 Subsect & ” }).”); 1124 Put Line ( Tex , ”\end{ description }”); 1125 Put Line (Tex, ”See the indicated sections for a detailed description , and experimental results , of each configuration individually , and section ” & Ref(Subsect) & ” for a synthetic , comparative, summary report of the whole experiment, covering the entire set of configurations.”); 1126 New Line ( Tex ) ; 1127 for I in Batch ’ Range loop 1128 Put Line (”Reporting config ” & Img (I) & ” ” & 1129 To String (Batch (I).Name)); 1130 Report (Batch (I), Comp Tbl ( I ) ) ; 1131 end loop ; 1132 1133 Put Line ( Tex , ”\ s e c t i o n {\ l a b e l {” & Subsect & ”}Comparate results }”); 1134 for I in Comp Tbl ’ Range ( 1 ) loop 1135 Comp Tbl (I) (Size Gain) := Comp Tbl (I) (Size 0 ) / Comp Tbl (I) (Size 1 ) ; 1136 Comp Tbl (I) (Cost Gain) := Comp Tbl (I) (Cost 0 ) / Comp Tbl (I) (Cost 1 ) ; 1137 end loop ; 1138 1139 Put Line ( Tex , 1140 ”The following table shows the principal results for each configuration of this batch. The \Em{minimum} and \Em{maximum} values in each column except Success \ r a t e are \Em{ emphasized } . The \Em{ b e s t } final and gain scores in their columns are $\ bigstar$starred.” & EOL & 1141 EOL & B.9. PACKAGE KASIM2.COMPARATE (BODY) 183
1142 Tabular (Batch, Comp Tbl ) ) ; 1143 Close (Tex); 1144 end ; 1145 1146 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 1147 −− REPORT BATCH, VERSION 2 −− 1148 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 1149 1150 procedure Report2 (Filename : String; Batch : Configuration Array T) i s 1151 −− Creates LaTeX files for the batch 1152 1153 Comp Tbl : Comparate Table T ( Batch ’ Range); 1154 1155 function C o n f i g S e c t i o n I d I n Chapter (X : Configuration T) return S t r i n g i s 1156 begin 1157 return C o n f i g S e c t i o n I d I n Chapter (Filename, X); 1158 end ; 1159 1160 procedure P u t Description Item (X : Configuration T) i s 1161 Id : String := Config S e c t i o n I d I n Chapter (X); 1162 begin 1163 Put Line ( Tex , 1164 ”\ item [ \ r e f {” & Id & ”} ” & To String (X.Name) & ”.] ” & 1165 To String (X.Description)); 1166 end ; 1167 1168 procedure P u t Description Item (I : Positive) i s 1169 begin 1170 P u t Description Item (Batch (I)); 1171 end ; 1172 1173 Subsect : String := Latex.To Label (”subsectionbatch” & Filename); 1174 −− id of integrative subsection of batch 1175 begin 1176 Current Name := To Unbounded String (Filename); 1177 Put Line (”Reporting batch ” & Filename); 1178 Create (Tex, Out File , ”kasim2−batch−” & Filename & ”.tex”); 1179 Put Line (Tex, ”This section describes a comparate experiment of the following configurations:”); 1180 New Line ( Tex ) ; 1181 Put Line ( Tex , ”\ begin { description }”); 1182 for I in Batch ’ Range loop 1183 P u t Description I t e m ( I ) ; 1184 end loop ; 1185 Put Line ( Tex , 1186 ”\ item [ \ r e f {” & Subsect & 1187 ”} Comparate results.] Integrative report of all items above (page \ p a g e r e f {” & 1188 Subsect & ” }).”); 1189 Put Line ( Tex , ”\end{ description }”); 1190 Put Line ( Tex , 1191 ”See the indicated sections for a description of each configuration individually , ” & 1192 ” and sections starting ” & Ref(Subsect) & 1193 ” for a synthetic , comparative, summary report of the whole experiment, ” & 1194 ” covering the entire set of configurations.”); 184 APPENDIX B. PROGRAM LISTINGS
1195 New Line ( Tex ) ; 1196 for I in Batch ’ Range loop 1197 Put Line (”Reporting config ” & Img (I) & ” ” & 1198 To String (Batch (I).Name)); 1199 Report2 (Filename, Batch (I), Comp Tbl ( I ) ) ; 1200 end loop ; 1201 1202 Put Line ( Tex , ”\ s e c t i o n {\ l a b e l {” & Subsect & ”}Comparate results }”); 1203 for I in Comp Tbl ’ Range ( 1 ) loop 1204 Comp Tbl (I) (Size Gain) := Comp Tbl (I) (Size 0 ) / Comp Tbl (I) (Size 1 ) ; 1205 Comp Tbl (I) (Cost Gain) := Comp Tbl (I) (Cost 0 ) / Comp Tbl (I) (Cost 1 ) ; 1206 end loop ; 1207 1208 Put Line ( Tex , 1209 ”The following table shows the principal results for each configuration of this batch. The \Em{minimum} and \Em{maximum} values in each column except Success \ r a t e are \Em{ emphasized } . The \Em{ b e s t } final and gain scores in their columns are $\ bigstar$starred.” & EOL & 1210 EOL & 1211 Tabular (Batch, Comp Tbl ) ) ; 1212 Close (Tex); 1213 end ; 1214 1215 procedure Report2a (Filename : String; Batch : Configuration Array T) i s 1216 −− Creates LaTeX files for the batch 1217 1218 Comp Tbl : Comparate Table T ( Batch ’ Range); 1219 1220 function C o n f i g S e c t i o n I d I n Chapter (X : Configuration T) return S t r i n g i s 1221 begin 1222 return C o n f i g S e c t i o n I d I n Chapter (Filename, X); 1223 end ; 1224 1225 procedure P u t Description Item (X : Configuration T) i s 1226 Id : String := Config S e c t i o n I d I n Chapter (X); 1227 begin 1228 Put Line ( Tex , 1229 ”\ item [ \ r e f {” & Id & ”} ” & To String (X.Name) & ”.] ” & 1230 To String (X.Description)); 1231 end ; 1232 1233 procedure P u t Description Item (I : Positive) i s 1234 begin 1235 P u t Description Item (Batch (I)); 1236 end ; 1237 1238 Subsect : String := Latex.To Label (”subsectionbatch” & Filename); 1239 −− id of integrative subsection of batch 1240 begin 1241 Current Name := To Unbounded String (Filename); 1242 Put Line (”Reporting batch ” & Filename); 1243 Create (Tex, Out File , ”kasim2−batch−” & Filename & ”.tex”); 1244 Put Line (Tex, ”This section describes a comparate experiment of the following configurations:”); 1245 New Line ( Tex ) ; 1246 Put Line ( Tex , ”\ begin { description }”); B.9. PACKAGE KASIM2.COMPARATE (BODY) 185
1247 for I in Batch ’ Range loop 1248 P u t Description I t e m ( I ) ; 1249 end loop ; 1250 Put Line ( Tex , 1251 ”\ item [ \ r e f {” & Subsect & 1252 ”} Comparate results.] Integrative report of all items above (page \ p a g e r e f {” & 1253 Subsect & ” }).”); 1254 Put Line ( Tex , ”\end{ description }”); 1255 Put Line ( Tex , 1256 ”See the indicated sections for a description of each configuration individually , ” & 1257 ” and sections starting ” & Ref(Subsect) & 1258 ” for a synthetic , comparative, summary report of the whole experiment, ” & 1259 ” covering the entire set of configurations.”); 1260 New Line ( Tex ) ; 1261 for I in Batch ’ Range loop 1262 Put Line (”Reporting config ” & Img (I) & ” ” & 1263 To String (Batch (I).Name)); 1264 Report2a (Filename, Batch (I), Comp Tbl ( I ) ) ; 1265 end loop ; 1266 1267 Put Line ( Tex , ”\ s e c t i o n {\ l a b e l {” & Subsect & ”}Comparate results }”); 1268 for I in Comp Tbl ’ Range ( 1 ) loop 1269 Comp Tbl (I) (Size Gain) := Comp Tbl (I) (Size 0 ) / Comp Tbl (I) (Size 1 ) ; 1270 Comp Tbl (I) (Cost Gain) := Comp Tbl (I) (Cost 0 ) / Comp Tbl (I) (Cost 1 ) ; 1271 end loop ; 1272 1273 Put Line ( Tex , 1274 ”The following table shows the principal results for each configuration of this batch. The \Em{minimum} and \Em{maximum} values in each column except Success \ r a t e are \Em{ emphasized } . The \Em{ b e s t } final and gain scores in their columns are $\ bigstar$starred.” & EOL & 1275 EOL & 1276 Tabular (Batch, Comp Tbl ) ) ; 1277 Close (Tex); 1278 end ; 1279 1280 procedure Report3 (Filename : String; Batch : Configuration Array T) i s separate ; 1281 1282 procedure Report4 (Filename : String; Batch : Configuration Array T) i s separate ; 1283 1284 use Ada. Calendar ; 1285 use Ada. Calendar .Formatting; 1286 begin 1287 i f Create Index then 1288 i f Exists (Index Filename ) then 1289 Rename (Index Filename , Index Filename & ”.backup” & 1290 Latex . Keep Chars 1291 (Image (Date => Clock, Include T i m e F r a c t i o n => True ) , 1292 ”0123456789”)) ; 1293 end i f ; 1294 Create (File => Tex Index, Name => Index Filename , Mode => Out File ) ; 1295 end i f ; 1296 end ; 186 APPENDIX B. PROGRAM LISTINGS B.10 Procedure Kasim2.Comparate.Experiment (body only)
1 with ada . t e x t i o ; use ada . text IO; 2 with Ada.Numerics. Discrete Random ; 3 with Arm05 Model ; 4 with Kasim2. Activation ; 5 with Kasim2.Comparate. Configurations ; 6 with Kasim2.Documents; 7 with Kasim2.Markov; 8 with Kasim2. Markov With Heuristics ; 9 with Kasim2. Oracle Model ; 10 with Kasim2. Reporting ; 11 with Kasim2. Structural ; 12 13 procedure Kasim2.Comparate. Experiment i s 14 use Kasim2.Comparate. Configurations ; 15 16 procedure Main i s 17 begin 18 Kasim2. Reporting .Draw Charts := Draw Charts ; 19 20 i f Run Experiments then 21 Prepare Random Document ; 22 Run (Shattered Random Document ) ; 23 24 Kasim2.Structural.Doc := Subtree ’Unrestricted A c c e s s ; 25 Run (Against Chance (1 .. Against Chance ’Last − 1) & 26 A g a i n s t Structural & Against Markov ) ; 27 28 Prepare Trimmed Document ; 29 Run ( Batch Trimmed ) ; 30 end i f ; 31 32 Report2 (”chance”, Against Chance ) ; 33 Report2 (”structural”, Against Structural); 34 Report2 (”markov”, Against Markov & Batch Trimmed ) ; 35 end Main ; 36 37 task type Subtask T; 38 for Subtask T ’ S t o r a g e S i z e use 200 000 000 ; 39 task body Subtask T i s 40 begin 41 Main ; 42 end ; 43 begin 44 declare 45 Subtask : Subtask T; 46 begin 47 null ; 48 end ; 49 end ; B.11. PACKAGE KASIM2.MARKOV (SPEC) 187 B.11 Package Kasim2.Markov (spec)
1 package Kasim2 .Markov i s 2 3 −− This package provides adaptive operations 4 −− Start , Choose, Recentre 5 −− based on Markov chains (1st order). 6 7 −−−−−−−−−−−−−−−−−−−−−−−−−−− 8 −− PRINCIPAL FUNCTIONS −− 9 −−−−−−−−−−−−−−−−−−−−−−−−−−− 10 function S t a r t return Page T; 11 function Choose (Page : Page T) return Atom T; 12 function Recentre (Atom : Atom T) return Page T; 13 14 −−−−−−−−−−−−−−−−−−−−−−−−−−− 15 −− AUXILIARY FUNCTIONS −− 16 −−−−−−−−−−−−−−−−−−−−−−−−−−− 17 function Most Active 18 (A : Real Vector; Qt : Natural) return Page Item Vector T; 19 end ;
B.12 Package Kasim2.Markov (body)
1 with Kasim2. Activation ; 2 3 package body Kasim2 .Markov i s 4 −− This package provides adaptive operations 5 −− Start , Choose, Recentre 6 −− based on Markov chains (1st order). 7 −− 8 −− ”Consider the collection 9 −− of sessions and the corresponding first −order model given in Figure 1. Each web page 10 −− corresponds to a state in the model. In addition, there is an artificial start state (S) and an 11 −− artificial final state (F) appended to every session. ∗∗∗The f i r s t −order model is incrementally 12 −− built by processing each sequence of page requests.∗∗∗ A transition probability is estimated 13 −− by the ratio of the number of times the transition was traversed to the number of times 14 −− the anchor state was visited. Next to a link, the first number gives the number o f times 15 −− the link was traversed and the number in parentheses gives its estimated probability. 16 −− The model accuracy can be assessed by comparing a transition probability with the 17 −− corresponding higher −order probability estimated by the n−gram frequency counts. For 18 −− example, according to the input data the conditional probability of going to state A3 after 19 −− following link (A1 , A2 ) is given by the number of times users followed (A1 , A2 , A3 ) 20 −− dividedbythenumberoftimesusersfollowed(A1,A2), that is3/6=0.5. Thefirst − 188 APPENDIX B. PROGRAM LISTINGS
21 −− ordermodelisnotaccuratesincep(A2,A3)=0.38. 22 −− Session freq. 23 −− A1, A2, A3 3 24 −− A1, A2, A4 3 25 −− A5, A2, A3 1 26 −− A5, A2, A4 4 27 −− A6, A2, A3 2 28 −− A6, A2, A4 3 29 −− Figure1. Acollectionofusernavigationsessionsandthecorrespondingfirst −ordermodel” 30 −− (Borges & Levene 2008) 31 −− File ”Borges SIT 2008 . pdf ” 32 33 −−−−−−−−−−−−− 34 −− START −− 35 −−−−−−−−−−−−− 36 37 function S t a r t return Page T i s 38 Cnt : array (Cur.Travels ’ Range( 1 ) ) of Natural := ( others => 0) ; 39 procedure Update Cnt (C : Session Vectors .Cursor) i s 40 begin 41 Inc (Cnt (Session Vectors.Element (C).Pages.First Element .Content.Items. F i r s t Element .Atom)) ; 42 end ; 43 Max : Atom T := 1 ; 44 begin 45 −− Here we need the atom that has served more times 46 −− as central of the start page. Then recentre on it. 47 −− At the start of experiment (no sessions), we just 48 −− serve atom 1. 49 Cur.Sessions.Iterate (Update Cnt’ Unrestricted A c c e s s ) ; 50 for I in 2 .. Cnt’Last loop 51 i f Cnt ( I ) > Cnt (Max) then 52 Max := I ; 53 end i f ; 54 end loop ; 55 return Recentre (Max); 56 end ; 57 58 −−−−−−−−−−−−−− 59 −− CHOOSE −− 60 −−−−−−−−−−−−−− 61 62 function Choose (Page : Page T) return Atom T i s 63 −− This is a user simulation function, not adaptation. 64 −− We are not using Markov for simulation. 65 −− Just reuse the common function (see Kasim2.Activation). 66 begin 67 return Kasim2. Activation .Choose (Page); 68 end ; 69 70 −−−−−−−−−−−−−−−− 71 −− RECENTRE −− 72 −−−−−−−−−−−−−−−− 73 74 function Recentre (Atom : Atom T) return Page T i s 75 −− The first order transition probability is B.12. PACKAGE KASIM2.MARKOV (BODY) 189
76 −− proportional to the travel count. 77 −− So, spreading once from the ”anchor” will 78 −− energize the candidate items accordingly. 79 −− So, this is similar to Kasim2.Activation 80 −− but with a single propagation step. 81 −− Other important differences include: 82 −− − the page size might be incomplete 83 −− − seen atoms are not avoided 84 use Real Arrays ; 85 use Page Item Vectors ; 86 W : Real Matrix Ptr T renames Cur.Travels; 87 N : Natural := W’Length; 88 A : Real Vector (1 .. N); 89 lambda : Real T := 1 . 0 ; 90 alpha : Real T := 1 . 0 ; 91 I : Real Matrix := Unit Matrix (N, 1, 1); 92 M : Real Matrix Ptr T; 93 Cnt : Natural; 94 95 function Active Count return Natural i s 96 N : Natural := 0; 97 begin 98 for I in A’ Range loop 99 i f A(I) > 0 . 0 then Inc (N) ; end i f ; 100 end loop ; 101 return N; 102 end ; 103 begin 104 −− Setup the matrix 105 M := new Real Matrix (1 .. N, 1 .. N); 106 M. a l l := lambda ∗ I + alpha ∗ Transpose (W. a l l ); 107 108 −− Energise central atom, zero the others 109 A := ( others => 0 . 0 ) ; 110 A (Atom) := 1.0; 111 112 −− Spread once 113 A := M. a l l ∗ A; 114 Free (M) ; 115 116 −− Make sure the central atom is the most energized 117 A (Atom) := Real T ’ Last ; 118 119 −− Done 120 Cnt := Active Count ; 121 return ( Items => Most Active (A, Natural ’Min (Cnt, Cur.Page Size ) ) , 122 Qt Expanded => Natural ’Min (Cnt, Cur.Qt Expanded ) , 123 Cost => 1 . 0 ) ; 124 end ; 125 126 −−−−−−−−−−−−−−−−−−−−−−−−−−− 127 −− AUXILIARY FUNCTIONS −− 128 −−−−−−−−−−−−−−−−−−−−−−−−−−− 129 130 function Most Active 131 (A : Real Vector; Qt : Natural) return Page Item Vector T 190 APPENDIX B. PROGRAM LISTINGS
132 i s 133 Items : Page Item Array T(A’ Range); 134 begin 135 for I in A’ Range loop 136 Items (I) := (Atom => I , Rank => A(I)); 137 end loop ; 138 Sort By Descending Rank (Items); 139 return To Vector (Items (1 .. Qt)); 140 end ; 141 end ;
B.13 Package Kasim2.Markov With Heuristics (body)
1 with Kasim2. Activation ; 2 3 package body Kasim2. Markov With Heuristics i s 4 −− This package provides adaptive operations 5 −− Start , Choose, Recentre 6 −− based on Markov chains (1st order). 7 −− 8 −− ”Consider the collection 9 −− of sessions and the corresponding first −order model given in Figure 1. Each web page 10 −− corresponds to a state in the model. In addition, there is an artificial start state (S) and an 11 −− artificial final state (F) appended to every session. ∗∗∗The f i r s t −order model is incrementally 12 −− built by processing each sequence of page requests.∗∗∗ A transition probability is estimated 13 −− by the ratio of the number of times the transition was traversed to the number of times 14 −− the anchor state was visited. Next to a link, the first number gives the number o f times 15 −− the link was traversed and the number in parentheses gives its estimated probability. 16 −− The model accuracy can be assessed by comparing a transition probability with the 17 −− corresponding higher −order probability estimated by the n−gram frequency counts. For 18 −− example, according to the input data the conditional probability of going to state A3 after 19 −− following link (A1 , A2 ) is given by the number of times users followed (A1 , A2 , A3 ) 20 −− dividedbythenumberoftimesusersfollowed(A1,A2), that is3/6=0.5. Thefirst − 21 −− ordermodelisnotaccuratesincep(A2,A3)=0.38. 22 −− Session freq. 23 −− A1, A2, A3 3 24 −− A1, A2, A4 3 25 −− A5, A2, A3 1 26 −− A5, A2, A4 4 27 −− A6, A2, A3 2 28 −− A6, A2, A4 3 29 −− Figure1. Acollectionofusernavigationsessionsandthecorrespondingfirst −ordermodel” 30 −− (Borges & Levene 2008) 31 −− File ”Borges SIT 2008 . pdf ” B.13. PACKAGE KASIM2.MARKOV WITH HEURISTICS (BODY) 191
32 33 −−−−−−−−−−−−− 34 −− START −− 35 −−−−−−−−−−−−− 36 37 function S t a r t return Page T i s 38 Cnt : array (Cur.Travels ’ Range( 1 ) ) of Natural := ( others => 0) ; 39 procedure Update Cnt (C : Session Vectors .Cursor) i s 40 begin 41 Inc (Cnt (Session Vectors.Element (C).Pages.First Element .Content.Items. F i r s t Element .Atom)) ; 42 end ; 43 Max : Atom T := 1 ; 44 begin 45 −− Here we need the atom that has served more times 46 −− as central of the start page. Then recentre on it. 47 −− At the start of experiment (no sessions), we just 48 −− serve atom 1. 49 Cur.Sessions.Iterate (Update Cnt’ Unrestricted A c c e s s ) ; 50 for I in 2 .. Cnt’Last loop 51 i f Cnt ( I ) > Cnt (Max) then 52 Max := I ; 53 end i f ; 54 end loop ; 55 return Recentre (Max); 56 end ; 57 58 −−−−−−−−−−−−−− 59 −− CHOOSE −− 60 −−−−−−−−−−−−−− 61 62 function Choose (Page : Page T) return Atom T i s 63 −− This is a user simulation function, not adaptation. 64 −− We are not using Markov for simulation. 65 −− Just reuse the common function (see Kasim2.Activation). 66 begin 67 return Kasim2. Activation .Choose (Page); 68 end ; 69 70 −−−−−−−−−−−−−−−− 71 −− RECENTRE −− 72 −−−−−−−−−−−−−−−− 73 74 function Recentre (Atom : Atom T) return Page T i s 75 −− The first order transition probability is 76 −− proportional to the travel count. 77 −− So, spreading once from the ”anchor” will 78 −− energize the candidate items accordingly. 79 −− So, this is similar to Kasim2.Activation 80 −− but with a single propagation step. 81 −− In this ”with heuristics” variant we include 82 −− the same heuristics of ”activation” concerning 83 −− seen atoms . 84 use Real Arrays ; 85 use Page Item Vectors ; 86 W : Real Matrix Ptr T renames Cur.Travels; 192 APPENDIX B. PROGRAM LISTINGS
87 N : Natural := W’Length; 88 A : Real Vector (1 .. N); 89 lambda : Real T := 1 . 0 ; 90 alpha : Real T := 1 . 0 ; 91 I : Real Matrix := Unit Matrix (N, 1, 1); 92 M : Real Matrix Ptr T; 93 Cnt : Natural; 94 All Atoms , 95 Not Seen Yet , 96 Not Seen Yet Plus Central , 97 Seen Except Central : Atom Set T; 98 99 function Active Count return Natural i s 100 N : Natural := 0; 101 begin 102 for I in A’ Range loop 103 i f A(I) > 0 . 0 then Inc (N) ; end i f ; 104 end loop ; 105 return N; 106 end ; 107 begin 108 −− Prepare atom sets 109 for I in 1 . . N loop 110 All Atoms.Insert (I); 111 end loop ; 112 Not Seen Yet := All Atoms.Difference (Cur.Seen); 113 Not Seen Yet Plus Central := Not Seen Yet ; 114 Not Seen Yet Plus Central.Include (Atom); 115 Seen Except Central := Cur.Seen; 116 Seen Except Central.Exclude (Atom); 117 −− ”The outer system is supposed to keep Cur.Seen in good order 118 −− viz. containing the central Atom. Otherwise the Insert or 119 −− Delete operations above will raise an exception. We could 120 −− Include and Exclude (which do not raise) but decided to use 121 −− Insert/Delete as an instrument of checking the correctness 122 −− of the outer system.” (Kasim2.Activation) 123 −− For Markov this does not seem to hold. 124 −− So I did change to Include, Exclude, as a simple, harmless, 125 −− s o l u t i o n . 126 127 −− Setup the matrix 128 M := new Real Matrix (1 .. N, 1 .. N); 129 M. a l l := lambda ∗ I + alpha ∗ Transpose (W. a l l ); 130 131 −− Energise central atom, zero the others 132 A := ( others => 0 . 0 ) ; 133 A (Atom) := 1.0; 134 135 −− Spread once 136 A := M. a l l ∗ A; 137 Free (M) ; 138 139 −− Apply ”not seen” heuristic: 140 −− Remove seen atoms (by de−energizing). 141 −− Only remove while count larger than 142 −− required quantity of items for page. B.14. PACKAGE KASIM2.MARKOV WITH HEURISTICS (BODY) 193
143 for I in A’ Range loop 144 exit when Active Count <= Cur . Page Size ; 145 i f I /= Atom and then Cur.Seen.Contains (I) then 146 A ( I ) := 0 . 0 ; 147 end i f ; 148 end loop ; 149 150 −− Make sure the central atom is the most energized 151 A (Atom) := Real T ’ Last ; 152 153 −− Done 154 Cnt := Active Count ; 155 return ( Items => Most Active (A, Natural ’Min (Cnt, Cur.Page Size ) ) , 156 Qt Expanded => Natural ’Min (Cnt, Cur.Qt Expanded ) , 157 Cost => 1 . 0 ) ; 158 end ; 159 160 −−−−−−−−−−−−−−−−−−−−−−−−−−− 161 −− AUXILIARY FUNCTIONS −− 162 −−−−−−−−−−−−−−−−−−−−−−−−−−− 163 164 function Most Active 165 (A : Real Vector; Qt : Natural) return Page Item Vector T 166 i s 167 Items : Page Item Array T(A’ Range); 168 begin 169 for I in A’ Range loop 170 Items (I) := (Atom => I , Rank => A(I)); 171 end loop ; 172 Sort By Descending Rank (Items); 173 return To Vector (Items (1 .. Qt)); 174 end ; 175 end ;
B.14 Package Kasim2.Markov With Heuristics (body)
1 with Kasim2. Activation ; 2 3 package body Kasim2. Markov With Heuristics i s 4 −− This package provides adaptive operations 5 −− Start , Choose, Recentre 6 −− based on Markov chains (1st order). 7 −− 8 −− ”Consider the collection 9 −− of sessions and the corresponding first −order model given in Figure 1. Each web page 10 −− corresponds to a state in the model. In addition, there is an artificial start state (S) and an 11 −− artificial final state (F) appended to every session. ∗∗∗The f i r s t −order model is incrementally 12 −− built by processing each sequence of page requests.∗∗∗ A transition probability is estimated 13 −− by the ratio of the number of times the transition was traversed to the number of times 194 APPENDIX B. PROGRAM LISTINGS
14 −− the anchor state was visited. Next to a link, the first number gives the number o f times 15 −− the link was traversed and the number in parentheses gives its estimated probability. 16 −− The model accuracy can be assessed by comparing a transition probability with the 17 −− corresponding higher −order probability estimated by the n−gram frequency counts. For 18 −− example, according to the input data the conditional probability of going to state A3 after 19 −− following link (A1 , A2 ) is given by the number of times users followed (A1 , A2 , A3 ) 20 −− dividedbythenumberoftimesusersfollowed(A1,A2), that is3/6=0.5. Thefirst − 21 −− ordermodelisnotaccuratesincep(A2,A3)=0.38. 22 −− Session freq. 23 −− A1, A2, A3 3 24 −− A1, A2, A4 3 25 −− A5, A2, A3 1 26 −− A5, A2, A4 4 27 −− A6, A2, A3 2 28 −− A6, A2, A4 3 29 −− Figure1. Acollectionofusernavigationsessionsandthecorrespondingfirst −ordermodel” 30 −− (Borges & Levene 2008) 31 −− File ”Borges SIT 2008 . pdf ” 32 33 −−−−−−−−−−−−− 34 −− START −− 35 −−−−−−−−−−−−− 36 37 function S t a r t return Page T i s 38 Cnt : array (Cur.Travels ’ Range( 1 ) ) of Natural := ( others => 0) ; 39 procedure Update Cnt (C : Session Vectors .Cursor) i s 40 begin 41 Inc (Cnt (Session Vectors.Element (C).Pages.First Element .Content.Items. F i r s t Element .Atom)) ; 42 end ; 43 Max : Atom T := 1 ; 44 begin 45 −− Here we need the atom that has served more times 46 −− as central of the start page. Then recentre on it. 47 −− At the start of experiment (no sessions), we just 48 −− serve atom 1. 49 Cur.Sessions.Iterate (Update Cnt’ Unrestricted A c c e s s ) ; 50 for I in 2 .. Cnt’Last loop 51 i f Cnt ( I ) > Cnt (Max) then 52 Max := I ; 53 end i f ; 54 end loop ; 55 return Recentre (Max); 56 end ; 57 58 −−−−−−−−−−−−−− 59 −− CHOOSE −− 60 −−−−−−−−−−−−−− 61 62 function Choose (Page : Page T) return Atom T i s 63 −− This is a user simulation function, not adaptation. B.14. PACKAGE KASIM2.MARKOV WITH HEURISTICS (BODY) 195
64 −− We are not using Markov for simulation. 65 −− Just reuse the common function (see Kasim2.Activation). 66 begin 67 return Kasim2. Activation .Choose (Page); 68 end ; 69 70 −−−−−−−−−−−−−−−− 71 −− RECENTRE −− 72 −−−−−−−−−−−−−−−− 73 74 function Recentre (Atom : Atom T) return Page T i s 75 −− The first order transition probability is 76 −− proportional to the travel count. 77 −− So, spreading once from the ”anchor” will 78 −− energize the candidate items accordingly. 79 −− So, this is similar to Kasim2.Activation 80 −− but with a single propagation step. 81 −− In this ”with heuristics” variant we include 82 −− the same heuristics of ”activation” concerning 83 −− seen atoms . 84 use Real Arrays ; 85 use Page Item Vectors ; 86 W : Real Matrix Ptr T renames Cur.Travels; 87 N : Natural := W’Length; 88 A : Real Vector (1 .. N); 89 lambda : Real T := 1 . 0 ; 90 alpha : Real T := 1 . 0 ; 91 I : Real Matrix := Unit Matrix (N, 1, 1); 92 M : Real Matrix Ptr T; 93 Cnt : Natural; 94 All Atoms , 95 Not Seen Yet , 96 Not Seen Yet Plus Central , 97 Seen Except Central : Atom Set T; 98 99 function Active Count return Natural i s 100 N : Natural := 0; 101 begin 102 for I in A’ Range loop 103 i f A(I) > 0 . 0 then Inc (N) ; end i f ; 104 end loop ; 105 return N; 106 end ; 107 begin 108 −− Prepare atom sets 109 for I in 1 . . N loop 110 All Atoms.Insert (I); 111 end loop ; 112 Not Seen Yet := All Atoms.Difference (Cur.Seen); 113 Not Seen Yet Plus Central := Not Seen Yet ; 114 Not Seen Yet Plus Central.Include (Atom); 115 Seen Except Central := Cur.Seen; 116 Seen Except Central.Exclude (Atom); 117 −− ”The outer system is supposed to keep Cur.Seen in good order 118 −− viz. containing the central Atom. Otherwise the Insert or 119 −− Delete operations above will raise an exception. We could 196 APPENDIX B. PROGRAM LISTINGS
120 −− Include and Exclude (which do not raise) but decided to use 121 −− Insert/Delete as an instrument of checking the correctness 122 −− of the outer system.” (Kasim2.Activation) 123 −− For Markov this does not seem to hold. 124 −− So I did change to Include, Exclude, as a simple, harmless, 125 −− s o l u t i o n . 126 127 −− Setup the matrix 128 M := new Real Matrix (1 .. N, 1 .. N); 129 M. a l l := lambda ∗ I + alpha ∗ Transpose (W. a l l ); 130 131 −− Energise central atom, zero the others 132 A := ( others => 0 . 0 ) ; 133 A (Atom) := 1.0; 134 135 −− Spread once 136 A := M. a l l ∗ A; 137 Free (M) ; 138 139 −− Apply ”not seen” heuristic: 140 −− Remove seen atoms (by de−energizing). 141 −− Only remove while count larger than 142 −− required quantity of items for page. 143 for I in A’ Range loop 144 exit when Active Count <= Cur . Page Size ; 145 i f I /= Atom and then Cur.Seen.Contains (I) then 146 A ( I ) := 0 . 0 ; 147 end i f ; 148 end loop ; 149 150 −− Make sure the central atom is the most energized 151 A (Atom) := Real T ’ Last ; 152 153 −− Done 154 Cnt := Active Count ; 155 return ( Items => Most Active (A, Natural ’Min (Cnt, Cur.Page Size ) ) , 156 Qt Expanded => Natural ’Min (Cnt, Cur.Qt Expanded ) , 157 Cost => 1 . 0 ) ; 158 end ; 159 160 −−−−−−−−−−−−−−−−−−−−−−−−−−− 161 −− AUXILIARY FUNCTIONS −− 162 −−−−−−−−−−−−−−−−−−−−−−−−−−− 163 164 function Most Active 165 (A : Real Vector; Qt : Natural) return Page Item Vector T 166 i s 167 Items : Page Item Array T(A’ Range); 168 begin 169 for I in A’ Range loop 170 Items (I) := (Atom => I , Rank => A(I)); 171 end loop ; 172 Sort By Descending Rank (Items); 173 return To Vector (Items (1 .. Qt)); 174 end ; 175 end ; B.15. PACKAGE KASIM2.STRUCTURAL (SPEC) 197 B.15 Package Kasim2.Structural (spec)
1 with Arm05 Model ; 2 3 package Kasim2. Structural i s 4 5 −− This package provides operations 6 −− Start , Choose, Recentre 7 −− based on the original hierarchical structure of the document. 8 9 −−Subtree : aliased Arm05 Model . Tree T := Arm05 Model. Traverse Files (1000); 10 Doc : access Arm05 Model . Tree T;−− := Subtree ’Access; 11 12 function S t a r t return Page T; 13 function Choose (Page : Page T) return Atom T; 14 function Recentre (Atom : Atom T) return Page T; 15 end ;
B.16 Package Kasim2.Structural (body)
1 with ada . t e x t i o ; use ada . t e x t i o ; 2 with ada.assertions ; use ada.assertions ; 3 with Kasim2. Activation ; 4 5 package body Kasim2. Structural i s 6 7 −− This package provides adaptive operations 8 −− Start , Choose, Recentre 9 −− based on the original hierarchical structure of the document. 10 11 −− This structural view of the document serves entire sections , 12 −− when they are a leaf of the structure, or else the section 13 −− heading, any preambular paragraphs (in expanded form) and 14 −− the section index. 15 16 −− Paragraphs are always expanded. 17 18 −− There is no adaptation. (The outer system is supposed to 19 −− set Kasim2.Cur.Update Travels to False.) 20 21 −− Doc should point to the document represented in Cur.Travels 22 23 −− Cur . Page Size is used as the unit of cognitive effort for 24 −− number of page items. 25 26 27 28 −−−−−−−−−−−−− 29 −− START −− 30 −−−−−−−−−−−−− 31 32 function S t a r t return Page T i s 33 begin 34 −− Atom 1 is the top of the book 198 APPENDIX B. PROGRAM LISTINGS
35 −− i.e. the book as a whole, 36 −− connected to all chapters. 37 return Recentre (1); 38 end ; 39 40 −−−−−−−−−−−−−− 41 −− CHOOSE −− 42 −−−−−−−−−−−−−− 43 44 function Choose (Page : Page T) return Atom T i s 45 −− This is a user simulation function, not adaptation. 46 −− Just reuse the common function (see Kasim2.Activation). 47 −− (Yes, the Kasim2 architecture needs refactoring.) 48 begin 49 −−put(” xxx ”); 50 −−put ( atom t ’image(kasim2. activation .choose(page))); 51 return Kasim2. Activation .Choose (Page); 52 exception when others => put(” !!! ”); raise ; 53 end ; 54 55 −−−−−−−−−−−−−−−− 56 −− RECENTRE −− 57 −−−−−−−−−−−−−−−− 58 59 function Recentre (Atom : Atom T) return Page T i s 60 P : Page T; 61 Not Used : Real T := 0 . 0 ; 62 use Arm05 Model ; 63 64 function F i l e S i z e return Float i s 65 begin 66 i f Atom = 1 then return A v g F i l e S i z e ; 67 else return Float (Doc (Atom).File S i z e ) ; 68 end i f ; 69 end ; 70 71 function Choice Cost return Real T i s 72 begin 73 return 74 Real T’Max (1.0, Real T(P.Items.Length) / Real T ( Cur . Page Size ) ) ; 75 end ; 76 77 function S c r o l l C o s t return Real T i s 78 begin 79 return Real T ( 0 . 5 ∗ ( F i l e S i z e − Float (Buttons S i z e ) ) / Av g Sc reen Si ze ) ; 80 end ; 81 82 begin 83 P.Items.Clear; 84 P.Items.Append ((Atom => Atom , Rank => Not Used ) ) ; 85 P. Qt Expanded := 1; 86 for I in Doc ’ Range loop 87 i f Doc (I).Parent = Atom then 88 P.Items.Append ((Atom => I , Rank => Not Used ) ) ; 89 i f Is Paragraph (Doc (I)) then 90 Inc (P. Qt Expanded ) ; B.16. PACKAGE KASIM2.STRUCTURAL (BODY) 199
91 end i f ; 92 end i f ; 93 end loop ; 94 P.Cost := Choice Cost + Scroll C o s t ; 95 −−put(” cost=”&P.Cost ’img); 96 return P; 97 end ; 98 99 end ; 200 APPENDIX B. PROGRAM LISTINGS Appendix C
Supplemental Items
Items that I was not able to integrate smoothly into the main body of the dissertation. Not required reading, but may help understand certain aspects of it.
C.1 Software requirements for the Knowledge Atoms design
The specification of this design in a software engineering fashion—a numbered list of requirements, straight out of Alves et al. 2004.
1 A page is an assemblage of views of knowledge atoms, and controls.
2 Each view represents an atom.
3 One of the atoms on a page is central, and is usually viewed at the top of the page.
4 The central view is expanded (see 7).
5 The others atoms on a page are the most related (see 17) to the central one.
6 Any expanded atom is more related (to the central one) than any contracted one.
7 A knowledge atom is a piece of information whose expanded view does not ex- ceed the page size, and is usually substantially smaller than that, say 1/10.
8 The contracted view of a knowledge atom does not exceed its expanded view, and is usually smaller than that, say 1/5.
9 The user operates the system by executing controls on the page.
10 A control exists to recentre on a chosen atom.
11 This operation has the effect of replacing the current page with a newly assembled one (where the chosen atom is central).
201 202 APPENDIX C. SUPPLEMENTAL ITEMS
12 This constitutes a travel from the previous to the chosen central atoms. ¡¡¡===
13 The system has the concepts of user and session.
14 Controls exist to terminate a session.
15 Sessions are owned by users.
16 A session is a set of travels.
17 Relatedness between atoms is based on travel information.
18 Controls exist to adjust the relatedness function, including restricting the set of available atoms, based on time and ownership of the atoms and of the travels.
19 Upon recentring or session termination, the expanded atoms are also considered travelled to, if evidence exists that the user has read them.
20 This evidence usually comes from the time spent on a page.
21 This evidence can also come from eye tracking technology (Saloj¨arvi et al. 2005).
22 A control exists to insert a new atom in the system.
23 Tools (perhaps controls) exist to automatically connect the new atom to others, i.e. to generate travels between them.
24 Controls exist to shatter an atom in two or more atoms, and to coalesce two or more items into one.
C.2 Kasim 1. First cycle of experiments
In the first cycle of exploratory experiments we have designed and implemented an adaptive hypertext simulator called Kasim (for knowledge atoms simulator), which uses equally homegrown software libraries for network database and spreading activation, called Mneson and Minibrain respectively. Naturally this software implements the theories exposed before. The intention is to interpret the intermediary and final results of the experiments as an evaluation of said theories.
The first Kasim experiment was based on a smallish simulated document called X3 specified with 3 sections, depth in [1, 2], and fanout (out-degree) in [1, 6]. Kasim produced a document with 32 atoms and the following structure. Figure C.1 shows all CHILD links. Figure C.2 shows all CHILD and NEXT links. C.2. KASIM 1. FIRST CYCLE OF EXPERIMENTS 203
Figure C.1: Hierarchical structure of document X3 (CHILD links)
#69 #75 #78
#63 #72 #54 #66 #87
#60 #51 #90 #81 #57 #43 #84 #48 #45
SECTIONS
#1 #8 #20 #40 #5 #14 #25 #2 #22 #37
#17 #34 #11 #31 #28
Naturally, the non-terminal nodes of the CHILD hierarchy represent section and subsection headings, and the terminal nodes represent normal paragraphs.
NEXT links connect sequent paragraphs in the same section or subsection, and go from the last paragraph of a section or subsection to the heading of the next section or subsection.
This structure represents a traditional document in terms of the di-nodul-tri-nexial, or shattered document model (section 3.2.1). The generated document is clearly a fairly realistic one.
Circa 100 sessions and exactly 5 oracles were generated. Each session pertains to one of the five oracles identified #97 to #101.1 The sessions are generated in sequence. A random oracle is chosen for each session in a uniformely distributed way.
Each page has 5 atom views. The start page has only contracted views. Other pages have 2 expanded views and 3 contracted views. For this page size of 5 and this number of oracles of 5, the optimal session size is 2 pages.
Figure C.3 shows the evolution of session size for each oracle. The horizontal axis represents the sequent sessions of the oracle. The vertical axis represents the session size in number of pages (= number of clicks - 1).
1These numbers are identifiers of objects in the database, which contains all objects involved. 204 APPENDIX C. SUPPLEMENTAL ITEMS
Figure C.2: Complete structure of document X3
SECTIONS
#1
CHILD NEXT
#2 CHILD
NEXT
#5
NEXT CHILD
#8 CHILD
NEXT CHILD
#11 CHILD
NEXT
#14
NEXT
#17
NEXT
#20
NEXT CHILD
#22 CHILD
NEXT
#25
CHILD NEXT
#28 CHILD
NEXT CHILD
#31 CHILD
NEXT CHILD
#34
NEXT
#37
NEXT
#40
NEXT
#43
CHILD NEXT
#45 CHILD
NEXT CHILD
#48
NEXT
#51
CHILD NEXT
#54 CHILD CHILD
NEXT CHILD
#57 CHILD
NEXT
#60
NEXT CHILD
#63
NEXT
#66
CHILD NEXT
#69 CHILD
NEXT CHILD
#72 CHILD
NEXT
#75
NEXT
#78
NEXT
#81
CHILD NEXT
#84 CHILD
NEXT CHILD
#87
NEXT
#90 C.2. KASIM 1. FIRST CYCLE OF EXPERIMENTS 205
Figure C.3: Evolution of session size for each oracle The horizontal axis is the sequence of sessions pertaining to each oracle. Oracle #97 Oracle #98 Oracle #99 10 10 10
9 9 9
8 8 8
7 7 7
6 6 6
5 5 5
4 4 4
3 3 3
2 2 2
1 1 1
0 0 0
Oracle #100 Oracle #101 10 10
9 9
8 8
7 7
6 6
5 5
4 4
3 3
2 2
1 1
0 0
For three oracles (#97, #98, #101) there was a decrease of session size, for one (#100) there was a slight increase, and for another (#99) the session size stayed the same.
Also, the evolution is much sharper in the first few sessions. Actually it seems that after the very first few sessions there is no noticeable evolution at all. That is, the optimal size is reached very soon. Surely this is due to the small size of the document and of the oracle set. In the next sections we report the results with a slightly larger document, and a quite larger number of oracles and sessions.
C.2.1 Second round: 60 atoms, 10 oracles
This second Kasim experiment was based on simulated document X4 specified with 4 sections, depth in [1, 3], fanout in [1, 6], and 10 oracles. Kasim produced a document with circa 60 atoms and the following structure. Figure C.4 shows all CHILD links. 206 APPENDIX C. SUPPLEMENTAL ITEMS
Figure C.4: Hierarchical structure of document X4 (CHILD links)
#325 #322
#328
CHILD CHILD #319 CHILD CHILD #331 CHILD
#307 CHILD
CHILD #316
#340 CHILD #334 CHILD CHILD
#313 #343 #337 #301 CHILD #298 #310 CHILD CHILD CHILD CHILD #304 CHILD #349 CHILD #295 #283 CHILD CHILD CHILD CHILD
CHILD #268 CHILD CHILD #286 #352 #346 #292
CHILD #289 #265 CHILD #355 #244 CHILD
#247 #241 CHILD #226 #223 CHILD #358 CHILD CHILD CHILD #364 CHILD #280 CHILD #238 #229 #271 #250 CHILD CHILD CHILD #379 #232 #214 CHILD CHILD CHILD CHILD #220 CHILD CHILD #367 CHILD CHILD CHILD #361 CHILD #277 CHILD CHILD #274 #253 #235 #376 CHILD CHILD CHILD #217 CHILD #415 CHILD #370 #256 #259 #373 CHILD #418 CHILD #412 CHILD CHILD #172 CHILD #406 #211 CHILD CHILD #262 CHILD CHILD #421 #409 CHILD #433 #400 #202 #397 CHILD
#436 #430 CHILD #208 CHILD CHILD CHILD #181 CHILD CHILD #175 #199 CHILD #403 CHILD CHILD CHILD #385 CHILD #394 #205 CHILD CHILD #439 #427 CHILD CHILD #184 CHILD CHILD CHILD CHILD #424 CHILD #196 CHILD #178 CHILD CHILD #388 #391 CHILD CHILD #382 #131 CHILD
#442 CHILD CHILD #451 CHILD #187 CHILD #193
#190 #445 #448 CHILD #484 #487 #460 CHILD
#481
CHILD #454 CHILD CHILD #463 CHILD CHILD CHILD #457 #478 CHILD CHILD #466 CHILD CHILD
#475 CHILD CHILD CHILD #169 #469 #472 CHILD
CHILD #166 #612 #609 #139
CHILD
#606 #615 CHILD CHILD CHILD #163 CHILD CHILD CHILD #499 #145
CHILD #597 CHILD #496 #585 #502 CHILD CHILD #618 #603 #588 CHILD #160 #594 CHILD CHILD #157 CHILD #142 CHILD CHILD CHILD CHILD CHILD #493 #490 CHILD #148 CHILD CHILD #136 CHILD CHILD #621 #600 #582 #505 CHILD CHILD #544 #541 #133 #113 #511 CHILD #154 #151 #624 CHILD #573 CHILD #508 #116 CHILD #110 CHILD #627 #576 CHILD #523 CHILD CHILD CHILD #538 CHILD CHILD #570 CHILD #104 CHILD CHILD #107 #591 CHILD #520 CHILD CHILD #514 CHILD CHILD #579 CHILD #119 #555 #526 CHILD #517 #535 #101 #98 CHILD CHILD CHILD #567 CHILD CHILD CHILD
#552 CHILD CHILD CHILD CHILD #529 #558 #532 CHILD #95 CHILD #86 #564 #80 #561 CHILD CHILD CHILD
#92 #125 CHILD #89 #122 CHILD
CHILD
#83 #657 #128 #654
CHILD CHILD #645 CHILD #660 CHILD CHILD #648 #651 CHILD CHILD #642 CHILD CHILD #549 #65 #636 #68 CHILD CHILD #672 #62 CHILD #53 #675 #639 CHILD #44 #669 CHILD CHILD #71 #56 CHILD #47 CHILD CHILD #41 #678 CHILD SECTIONS #1 CHILD #948 CHILD CHILD CHILD CHILD #59 CHILD CHILD CHILD #951 CHILD #666 CHILD #633 #945 #74 CHILD CHILD #50 CHILD #663 CHILD #77 CHILD CHILD CHILD #26 CHILD #38 CHILD CHILD #681 CHILD CHILD #630 #954 CHILD CHILD CHILD CHILD CHILD #693 CHILD #942 #5 CHILD CHILD #744 CHILD #930 #29 CHILD CHILD CHILD #35 #684 CHILD CHILD #547 #8 #32 #690 #957 CHILD CHILD CHILD CHILD #687 CHILD #741 CHILD #699 #732 #939 CHILD CHILD CHILD CHILD CHILD #702 #726 CHILD #933 CHILD #936 #11
CHILD #738 #906 CHILD #735 CHILD CHILD #705 #14 #17 #723 CHILD CHILD CHILD #729 #927 #696 CHILD #23 CHILD CHILD CHILD CHILD #2 #708 #720 #924 #20 CHILD CHILD CHILD CHILD
#711 #717 CHILD #909 CHILD #912 #921 #714 CHILD CHILD CHILD CHILD #915 #918
#747 #894
#762 #897 #759 CHILD #891 #903 CHILD CHILD CHILD CHILD #882 #765 #804 CHILD CHILD CHILD #753 CHILD #756 #900 CHILD #888 CHILD #816 CHILD CHILD #813 CHILD #768 #819 #885 CHILD CHILD #771 CHILD #798 CHILD CHILD #750 #810 #879 CHILD CHILD CHILD CHILD #822 CHILD #807 CHILD #774 CHILD CHILD CHILD #795 #801 CHILD CHILD #786 CHILD CHILD CHILD #870 CHILD #834 #876 CHILD CHILD CHILD CHILD #825 #777 CHILD #792 CHILD #789 #831 #828 #867 #873 #780 CHILD #783 #843
#864 CHILD #837 CHILD #846 CHILD CHILD
CHILD #840 #861 CHILD CHILD #849 CHILD CHILD #858
#852 #855 C.3. NOTES ON NETWORK DATA MODELS 207
Figure C.5: Evolution of session size
C.2.2 Conclusion of cycle one
The results of these experiments are indicative that our main hypothesis stand a good chance of confirmation with a more robust setup, and were also a proof of concept of the spreading activation technique. So, regarding the main concepts of this thesis, this first cycle was successful and provided encouragement for upgrading the experiments.
Regarding implementation, however, we have experienced difficulties with the Minibrain and Mneson modules. Minibrain revealed the defect of not always zeroing the total energy with the Waterline model; we suspect an issue with floating point approximations, leaving non null residuals where there should be absolute zeros, but have not pinpointed this issue in the code yet. Mneson, the graph database module, revealed a scalability problem. As all these issues were marginal to the present thesis, and at the time of these experiments, Ada 2005 technology become available (GNAT GPL 2007) with its linear algebra and container libraries, we have decided to reimplement Kasim using this much more validated foundation, as a means to avoid the scalability problem and focus on the main hypothesis. Kasim II would deal with the different configurations required for comparison, and scale better to large documents with a number of noogramicles in the thousands (see main text, specially chapter 4).
C.3 Notes on Network Data Models
In this section we describe the network metamodels of data used in this work, and discuss issues related with network data structures. 208 APPENDIX C. SUPPLEMENTAL ITEMS
Just to clarify: here network denotes the data structure, not communications.
C.3.1 Primary concepts of network data structures
A graph or network data structure is a structure of vertices, and connections between vertices. Newman 2003 provides a fairly comprehensive analysis of networks in general. The classic textbook Aho & Ullman 1992 dedicates an entire chapter to graph data models. In the literature at large a connection is also called an edge, or arc.
In the networks that interest us the most, connections are binary and directed.
A binary connection is one that connects exactly two vertices. An n-ary network, with n ≥ 2, is an hypergraph—but wherein the mention hyper is unrelated to hypertext. Hypergraphs are discussed briefly later.
A directed connection is one that has a direction from a source to a target vertex. We observe that (binary) directedness comes for free in computers: as normally a (binary) connection is represented as the corresponding pair of vertices, and memory is sequential, and a pair of values is normally represented as two consecutive memory blocks, this order may naturally be used to represent the directionality of the connection; and so, curiously enough, an undirected connection—seemingly less informative than a directed one—, is more costly to represent—at least for searching purposes, as the search for vertices x connected by an undirected connection to a given vertex a must consider all memory pairs (a, x) and well as (x, a). This observation adds to the motivation for implementing the network metamodel upon a directed base. We will return to this issue.
A typed connection is one that has a type, or label. We found it convenient that the type be represented by a vertex in the network, in order to be manipulated (e.g. queried) by the same devices put in place for vertex values. In this thesis this value is called a type instead of label or value, for historical reasons but that are still valid, as it will become aparent.
An untyped connection has no other property than the vertices it connects.
Terminologically, the properties of the connections carry over to the network: if the connections are typed, or labelled, we say we have a typed network; if the connections are directed, we say we have a directed network; etc.
We are interested in typed as well as untyped networks. In particular, we are interested in the relationship between the two kinds. In particular, we are interested in how to represent a typed network in an untyped one. Motivation for this will come.
In our conception of networks—either typed or untyped—, a vertex is either a datum or a pivot.
A datum is an atomic piece of data, or value, typically a number, or a symbol. A datum is identified by its value.
A pivot is a dataless vertex, that connects other vertices. Each distinct pivot has a unique serial number for its identification in formulae. We use the convention #n to name pivot with serial number n. C.3. NOTES ON NETWORK DATA MODELS 209
Our typed networks are like RDF networks, but without any of the unnecessary complications of XML syntax (e.g. indecideness element containment vs. attribute), special vertex types (resource vs. other), opposition data vs. metadata.
For a first quick grasp of these structures, see figure C.6, which shows a small database of book descriptions represented in XML and in a graphical depiction of the corresponding typed network.
Figure C.6: A subset of XQuery use case 1.1.2 (Chamberlin et al. 2005) represented in (a) the original XML format, (b) a typed network a la RDF. This example illustrates various issues discussed in the text.
(a)
title TCP/IP Illustrated price 65.95 book Advanced Programming title (b) bib book price in the Unix environment book 65.95 title price Data on the Web 39.95
C.3.2 The untyped network hypothesis
We have designed a specific network metamodel, the overall structure of which is an untyped base upon which typed connections are represented by means of certain connection patterns, the connection types being represented by vertices. 210 APPENDIX C. SUPPLEMENTAL ITEMS
We may call this design the untyped network hypothesis.
We have developed software to test this design, and to serve as the data module2 of our adaptive hypertext prototype for experiments. Therefore, the experimental results also provide indirect evaluation of this design of a network metamodel of data.
The untyped network hypothesis provides a unification of vertices and connections with regard to a number of aspects e.g. formalization, calculus, query optimization, representation, search, labelling.
Another rationale for the untyped network hypothesis was that it would facilitate the application of spreading activation. Namely, that, with the proper semantic structures (discussed below), spreading along the untyped, base connections, would give the expected results—thus simplifying the spreading activation algorithm. Furthermore, because types are nodes, the activation of types becomes an integral, seamless part of the activation pattern—sans the need for the extra data structures proposed in the literature e.g. which require a specific implementation for each layer of labelling.
C.3.2.1 Fundamental entities
The base structure is an untyped network, formally a graph (V,E) where V is the set of vertices and E is a set of edges, or connections, E ⊆ V × V . A connection (x, y) ∈ E may be notated x → y or y ← x; x is called the source, y is called the target, of the connection; we also say x is a source of y, y is a target of x.
A vertex is either a datum or a pivot. A data vertex or datum represents an atomic value of data e.g. an integer, a real number, a symbol. The value of a data vertex is the identity of the vertex. A pivot has no intrinsic value but it has nevertheless a unique identity. In practice a serial number is used to identify pivot vertices.
Note that connections are binary, directed, and untyped i.e. unlabelled. Note that being untyped entails not having an explicit identifier. A connection is identified solely by the (ordered) pair of vertices it represents. Hence, there are no duplicate connections. That is, for any pair of vertices (x, y) there can be at most one connection x → y.
Our software (Mneson) implements an order relation for vertices. From this a lexicographical order relation for connections is derived trivially. These order relations are used for the optimization of search.
Less importantly, but still usefully, these order relations provide a consistent default order for data reports.
2This software received the name Mneson (loosely, memory thing, from the greek lexeme mnesis and multi-use suffix -on). C.3. NOTES ON NETWORK DATA MODELS 211
Figure C.7: The direct attribute structure equates a connection.
Subject Attribute
C.3.2.2 Semantic structures
Semantic structures are special topologies (subgraphs, structures, patterns) that represent common semantic modelling structures like properties, sets, lists, attributes, typed connections. In this section we describe how these structures are represented in the untyped network base.3
Direct attribute
We call direct attribute—or untyped attribute, or simply attribute, or property, or predicate, depending on the context—to the very common pattern of saying something (the attribute) of something (the attributee, or subject), or that something (the subject) is something (predicate). In the untyped base, this structure is represented by a simple link connecting the two things involved, directed as shown on figure C.7.
Representing sets
Explicitly, extensively defined sets are represented as follows. A set X = {x1, . . . , xm} is represented as the set of connections {X → x1,...,X → xm}, i.e. the structure depicted in figure C.8.
Representing typed connections
A typed connection—or typed attribute, etc.—is represented in the untyped base with the road intersection pattern, cf. Figure C.9.
Figure C.10 recollects the example given before and adds the corresponding representation in the untyped network base, using the semantic structures just described.
3These structures were also called the Mneson conventions, or topologies, on previous Mneson documents, notably software items. Please adjust upon encountering such designations. 212 APPENDIX C. SUPPLEMENTAL ITEMS
Figure C.8: Untyped base structure to represent an explicit, extensively defined set
X = {x1, . . . , xm}.
X
. . .
x1 xm
Figure C.9: The road intersection pattern (a), for representing an (abstract) typed connection (b), in the untyped network base. Type
Source Target SourceType Target Pivot
(a) (b) C.3. NOTES ON NETWORK DATA MODELS 213
Figure C.10: A subset of XQuery use case 1.1.2 (Chamberlin et al. 2005) represented in: (a) the original XML format; (b) a typed network; (c) the untyped network base. This example illustrates various issues discussed in the text.
(a)
title TCP/IP Illustrated price 65.95 book Advanced Programming title (b) bib book price in the Unix environment book 65.95 title price Data on the Web 39.95
39.95
price 65.95 bib
(c) data+ Data on the Web book
title TCP/IP Illustrated
Advanced Programming in the Unix environment 214 APPENDIX C. SUPPLEMENTAL ITEMS
Figure C.10 also serves to demonstrate a couple of tools constructed. Item (c) was automatically pro- duced from the XML source. The network was produced with Mneson.XML.Mnesonise. The graph was produced with Mneson.Dump Dot (and, of course, the open source tool Dot, a.k.a. Graphviz).
The typed connection equates the very well known ternary relation of database theory, object– attribute name–attribute value, cf. e.g. Wiederhold 1985.
We observe also that the typed connection is the only kind of connection on RDF graphs.
C.3.2.3 The right semantic level
Albeit the formalization being done in terms of the untyped network hypothesis, often the right level for data modelling involves the higher order semantic structures discussed, specially typed connections.
Therefore, all data schemata in this thesis is written at the right semantic level for each case, irre- spectively of where that may be situated in the formal edifice. With the correspondences presented, the formalization in terms of the untyped base is inequivocably derivable.
C.3.3 Other designs
Other network database designs include typed networks like RDF (W3C) and a convoluted design from the 1960’s called CODASYL4.
Database theory seems immersed in the mythology that the relational model somehow has won over the hierarchical and the networked models, with these model wars having taken place in the 1960’s or so. With respect to the network model, the victory was over the CODASYL model, or so it is reported (Simsion & Witt 2005). The ensuing popularity of SQL and relational database engines (Oracle, Postgres, MySQL, SQLite...) is often taken as an indication of the superiority of the relational model. But, even if we let this argumentum ad populum pass, a number of facts—often overlooked in the literature—have the potential to invalidate the conclusion:
• CODASYL is a poor representative, if at all, of network database ideas
• one component of traditional data modelling is in fact networked at heart: the Entity- Relationship model (Chen 1976), often used as the conceptual (a.k.a. semantical) level in a three-level data systems design and implementation process (the other two levels being the logical and the physical, both relational-based traditionally)
• relational theory is indeed much more a physical, implementational model than a logical, abstract one
4CODASYL from “Conference on Data Systems Languages” refers to a set of standards for “network” DBMSs in which the principal constructs were Record Types, Data Items, and Sets. (Simsion & Witt 2005) C.3. NOTES ON NETWORK DATA MODELS 215
Williams 2002 provides extensive information of the history and nature of several data models, and proposes a certain network data model for the future of databases, with the vision that the aim of the relational model was to free programmers from having to know the physical structure of data; the aim of the associative model is to free them in addition from having to know its logical structure. Williams 2002 is a bit faulty in the theoretical details, but it is a bold statement against the unwarrant inertia of the relational model.
Anyway, lately, industry and science are finally catching up, or so it seems, with the good ideas of network data models: the dust has not settled yet, but some combination of the following models is here to stay—if only for another 50 years: RDF, triple stores, prevalent object systems, key-value stores, quad stores, and even hypergraphs (Chang et al. 2006, DeCandia et al. 2007, Iordanov 2010).
C.3.4 A network calculus
An untyped network is the most elegant mathematically, because it has only two types of elements: vertices and connections.
An untyped network may be mathematically represented as a set of (ordered) pairs of vertices. Each pair represents a connection.
Such representation is also of implementational value, because search in a set of ordered pairs5 has logarithmic or better complexity, with well known algorithms and structures like binary search, binary trees, b-tree, AVL trees, red-black tree, *-tree, etc. And such search can yield a number of useful results, notably the set of vertices that are targets of a given vertice, or sources thereof, or one such vertice, or the first, or the last, or variations thereof, including unions of such sets.
The direct mapping between the mathematical representation and the implementational structures brings great advantages in terms of software engineering, notably in the understandability, testabilty, reliability of the system.
Also, it facilitates immensely the theoretical analysis of the complexity of the database operations, by making possible the direct application of a network algebra, and of a number of useful theorems of this algebra.
C.3.5 Fundamental entities, definitions
Let a selection be a set of vertices. The network calculus is based on an algebra of selections. The operations include the set theoretical operations ∩, ∪, etc. and the unary adjacency operators T, S, with the following
Definition (def. T, def. S):
5Because vertices are distinct values, an order relation between vertices is easy to implement, and therefrom an order relation between ordered pairs comes naturally, namely the lexicographical order. 216 APPENDIX C. SUPPLEMENTAL ITEMS
TX = {y : ∃x ∈ X : x → y} SX = {y : ∃x ∈ X : y → x}
Unary operators have higher precedence than binary ones, e.g. TX ∪ Y is equivalent to (TX) ∪ Y , not to T(X ∪ Y ).
C.3.6 Useful theorems
Throughout this and the next section, X = {x1, . . . , xm}, Y = {y1, . . . , yn}.
Theorem C.3.1 (Null set) T∅ = ∅ S∅ = ∅
Theorem C.3.2 (Extension)
TX = T{x1} ∪ ... ∪ T{xm}
SX = S{x1} ∪ ... ∪ S{xm}
Theorem C.3.3 (Extraction of ∪)
T(X ∪ Y ) = TX ∪ TY S(X ∪ Y ) = SX ∪ SY
Unfortunately, we were not able to find a way of extracting ∩. These theorems are useful for theoretical complexity calculation and query optimization. The reason why it is unfortunate that a way to extract ∩ is missing, is that intersection seems to occur often in practical queries, notably in the form of the typed attribute structure. For example, given book objects (vertices) b and c with attribute price, the query for the price of b would be
T(T{b} ∩ T{price})
We hoped we could extract the ∩ and derive an equivalent formula like, say,
Π(b) ∩ Π(price) where Π is a (fictional) optimized query primitive. Compare with ∪ in6
T(T{b} ∪ T{c}) which is extractable to T(T{b}) ∪ T(T{c})) and therefore submitable to optimization via general optimized implementations of TT, ∪, or even the entire construct TT ∪ TT—all of which are very feasible.
6Meaning possibly all attribute values of books b and c. C.3. NOTES ON NETWORK DATA MODELS 217
C.3.7 Proofs, lemmas, axioms
In all proofs, the proof for the part of the theorem about T transposes, mutatis mutandis, to the part about S.
Proof of theorem C.3.1
T∅ = {y : ∃x ∈ ∅ : x → y} def. T = {y : ⊥} definition of ∅ = ∅ ax. comp.
QED.
Proof of theorem C.3.2
TX = {y : ∃x ∈ X : x → y} def. T
= {y : x1 → y ∨ ... ∨ xm → y} subst. ∃ = y : y ∈ {z : x1 → z} ∨ ... ∨ y ∈ {z : xm → z} ax. comp.
= {z : x1 → z} ∪ ... ∪ {z : xm → z} def. ∪
= T{x1} ∪ ... ∪ T{xm} def. T
QED for T. Similarly for S:
SX = {y : ∃x ∈ X : y → x} def. S
= {y : y → x1 ∨ ... ∨ y → xm} subst. ∃ = y : y ∈ {z : z → x1} ∨ ... ∨ y ∈ {z : z → xm} ax. comp.
= {z : z → x1} ∪ ... ∪ {z : z → xm} def. ∪
= S{x1} ∪ ... ∪ S{xm} def. S
QED.
Proof of theorem C.3.3
T(X ∪ Y ) = T{x1, . . . , xm, y1, . . . , yn}
= T{x1} ∪ ... ∪ T{xm} ∪ T{y1} ∪ ... ∪ T{yn} th. C.3.2 (extension)
= Tx1, . . . , xm ∪ Ty1, . . . , yn th. C.3.2 (extension) = TX ∪ TY
Substitution of ∃ by a finite disjunction (subst. ∃)
∃x ∈ {x1, . . . , xm} : P (x) = P (x1) ∨ ... ∨ P (xm) 218 APPENDIX C. SUPPLEMENTAL ITEMS
C.3.7.1 Axiom of Comprehension (ax. comp.)
Given a predicate P (x), the set exists S = {x : P (x)}, and ∀a(P (a) = a ∈ S). Appendix D
Detailed results of experiments
219 220 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Each configuration run is a simulation of sessions. Each session pertains to an oracle, i.e. it is a session of a user searching for the oracle. Normally we run a number of sessions ten times the number of oracles. For result reporting and statistical analysis, session results are grouped by oracle.
Each session is metered for its Duration (or Duratio1), Size and Cost. Duration is used for verification only. Duration is measured in seconds. For comparison we look at measures Size and Cost, and particularly at Cost.
For each configuration, the following report items are compiled. Except for the Outcomes table, an item is compiled for each quality measure Duration, Size, Cost. All these individual, per-configuration tables and charts are contained in Appendix D. Comparative tables and charts are presented in the next chapter. Naturally the values in the comparative items equate the respective values in the individual items.
Outcomes table. Statistics of success and failure of sessions.
Results table. Statistics of the quality measure.
Evolution table. Values of the quality measure along the experiment.
Evolution chart. The Evolution table in graphical form.
Moving average chart. The evolution of quality measure for all sessions distinctly, with a moving average line.
In the Evolution items, the oracle line values are compacted, as follows: the session order is considered for each oracle independently, and the last value is repeated for missing sessions (becase oracles may have slightly different numbers of sessions each). For example, a (toy) configuration with three oracles and the seven sessions with Size result
Session Oracle Size 1 2 5 2 3 7 3 2 4 4 1 2 5 2 3 6 1 1 7 3 6 has the results sequence by oracle as follows
Oracle 1 — 2, 1
Oracle 2 — 5, 4, 3
Oracle 3 — 7, 6 and thus gives the Evolution table
1The latin translate Duratio was used in the source code, because Duration is a reserved identifier in the programming language used; the form Duratio might surface on a few computer generated items e.g. charts. 221
1 2 3 Oracle 1 2 1 1 Oracle 2 5 4 3 Oracle 3 7 6 6 Total 14 11 10 Average 4.67 3.67 3.33 In the Moving Average charts the values are not separated by oracle, and therefore not compacted, nor repeated. They are the exact values in the experiment, and appear in the actual order of sessions in the experiment.
The statistics in the tables are as follows.
D.0.8 Common and bottom line statistics
The tables are divided into two parts of value lines:
• the oracle lines, carrying values for the sessions pertaining to each oracle.
• the bottom lines, with total and central statistics for all oracles (and therefore all sessions).
The following statistics are common to all tables except: Evolution tables do not have Micro avg, because they have no rates.
Oracle i. Introduces the line of values for an oracle group, i.e. for the sequence of sessions pertaining to the orcale. The i is just a sequential number in the table, for reference.
Total. The sum of all values in the column.
Average. The average, or mean, of all values in the column. Therefore, when the values are rates, this is the macro-average (Manning et al. 2009). The macro- average expresses a centrality, an expectation for an individual oracle.
Micro avg. Micro-average (Manning et al. 2009): the rate of averages. The micro- average expresses a centrality, an expectation, for the whole configuration. When the main statistic is not a rate, the micro-average equates the macro-average on the Average line, and the micro-average is not shown. See the sections on statistics for specific tables for the case-by-case definition of the respective micro- averages. 222 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
D.0.9 Statistics in the Outcomes tables
Node id. The node number of the oracle in the tree or sequence of atoms that repre- sents the legacy document. Not really a statistic, but useful for investigating the reason of possible deviant outcomes or results for an oracle, e.g. a Failure rate of 100% or a negative Gain. This id is the same for the oracles in all other configuration tables, but is only recorded here.
Total. The total number of sessions.
Timeout. The number of sessions that exceeded the maximum Duratio allowed. Such sessions are considered to have failed.
Sizeout. The number of sessions that exceeded the maximum Size allowed. Such sessions are considered to have failed.
Costout. The number of sessions that exceeded the maximum Cost allowed. Such sessions are considered to have failed.
Failed. The number of failed sessions:
Failed = Timeout + Sizeout + Costout
Successful. The number of successful sessions i.e. that have not failed:
Successful = Total − Failed
Fail rate. The percentage of sessions that have failed: Failed Fail rate = Total Success rate. The percentage of sessions that have succeeded (not failed): Successful Success rate = Total Micro avg. Micro-averages (see section on bottom line statistics for a general dis- cussion): Average of Failed Micro avg of Fail rate = Average of Total
Average of Succesful Micro avg of Gain = Average of Total 223
D.0.10 Statistics in the Results tables
Sum. The sum of all values for the measure.
Min. The minimum value attained.
Max. The maximum value attained.
Avg. The average, or mean value (naturally, over all values, not just the Min and the Max, or the First and the Last).
First. The value for the first session.
Last. The value for the last session.
Diff. The difference: Diff = First − Last
Rate. The compression rate: Last Rate = First Gain. The inverse of compression: the gain, or betterment attained: 1 First Gain = = Rate Last
Micro avg. Micro-averages (see section on bottom line statistics for a general dis- cussion): Average of Last Micro avg of Rate = Average of First
Average of First Micro avg of Gain = Average of Last
D.0.11 Statistics in the Evolution tables
The values in the Evolution tables are simply the values for each individual session. There are no complex statistics here. 224 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
D.0.12 Alternate terms
The concepts of first and last value, for some measure m e.g. cost, are also called initial and final, or m0 and m1, respectively. To clarify: first = initial = m0; last =
final = m1. (Historical reasons.) D.1. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT225 D.1 Complete results of configuration Shattered Document
The Shattered Documents approach, with Spreading Activation, and a Legacy Docu- ment (the ARM). Using functions Super Page, Choose, Recentre from Kasim2.Activation. Using the model of the ARM on Kasim2.Documents. Nominal parameters.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.94
Size0 = 5.62
Size1 = 3.48 Size gain = 1.62
Cost0 = 5.62
Cost1 = 3.48 Cost gain = 1.62 226 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 0 0 0 9 0.00 1.00 Oracle 2 30 6 0 0 0 0 6 0.00 1.00 Oracle 3 40 8 0 0 0 0 8 0.00 1.00 Oracle 4 46 7 0 0 0 0 7 0.00 1.00 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 0 0 0 0 7 0.00 1.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 0 0 0 8 0.00 1.00 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 0 0 0 9 0.00 1.00 Oracle 18 269 11 0 0 0 0 11 0.00 1.00 Oracle 19 290 6 0 0 0 0 6 0.00 1.00 Oracle 20 307 6 0 1 0 1 5 0.16 0.84 Oracle 21 366 10 0 1 0 1 9 0.10 0.90 Oracle 22 410 10 0 1 0 1 9 0.10 0.90 Oracle 23 455 15 0 1 0 1 14 0.07 0.93 Oracle 24 508 11 0 0 0 0 11 0.00 1.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 6 0 6 9 0.40 0.60 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 0 0 0 7 0.00 1.00 Oracle 29 542 5 0 1 0 1 4 0.20 0.80 Oracle 30 554 13 0 0 0 0 13 0.00 1.00 Oracle 31 601 11 0 0 0 0 11 0.00 1.00 Oracle 32 606 14 0 0 0 0 14 0.00 1.00 Oracle 33 616 12 0 0 0 0 12 0.00 1.00 Oracle 34 649 8 0 1 0 1 7 0.13 0.88 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 0 0 0 11 0.00 1.00 Oracle 39 786 7 0 1 0 1 6 0.14 0.86 Oracle 40 840 14 0 1 0 1 13 0.07 0.93 Oracle 41 846 8 0 0 0 0 8 0.00 1.00 Oracle 42 883 1 0 0 0 0 1 0.00 1.00 Oracle 43 890 10 0 1 0 1 9 0.10 0.90 Oracle 44 897 6 0 0 0 0 6 0.00 1.00 Oracle 45 899 9 0 0 0 0 9 0.00 1.00 Oracle 46 929 10 0 0 0 0 10 0.00 1.00 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 0 0 0 0 13 0.00 1.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 33 0 33 467 3.47 46.53 Average ··· 10.00 0.00 0.66 0.00 0.66 9.34 0.07 0.93 Micro avg ····················· 0.06 0.94 D.1. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT227
Duration Results of configuration Shattered Document
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 10.71 1.05 1.80 1.19 1.80 1.06 0.74 0.59 1.70 Oracle 2 7.76 1.04 2.20 1.29 2.20 1.04 1.16 0.47 2.13 Oracle 3 9.65 1.04 1.95 1.20 1.95 1.05 0.91 0.53 1.88 Oracle 4 8.48 1.04 2.24 1.21 2.24 1.04 1.20 0.46 2.16 Oracle 5 8.05 0.87 1.46 1.01 1.46 0.87 0.59 0.59 1.68 Oracle 6 12.24 1.04 1.77 1.11 1.77 1.09 0.69 0.61 1.64 Oracle 7 15.54 1.03 2.33 1.41 2.13 1.04 1.09 0.48 2.05 Oracle 8 14.41 0.95 2.62 1.31 1.45 1.09 0.36 0.75 1.33 Oracle 9 9.90 0.91 2.63 1.41 2.63 1.34 1.29 0.51 1.96 Oracle 10 20.42 0.62 1.33 1.02 1.32 1.03 0.29 0.78 1.28 Oracle 11 17.55 1.03 1.90 1.46 1.41 1.46 -0.05 1.04 0.96 Oracle 12 18.07 1.02 1.50 1.39 1.36 1.50 -0.14 1.10 0.91 Oracle 13 20.95 1.03 1.49 1.40 1.49 1.46 0.03 0.98 1.02 Oracle 14 14.33 0.92 1.37 1.30 1.37 1.33 0.04 0.97 1.03 Oracle 15 5.24 0.61 0.93 0.66 0.93 0.62 0.31 0.66 1.52 Oracle 16 7.65 1.04 1.84 1.27 1.84 1.04 0.80 0.57 1.77 Oracle 17 11.09 1.03 2.16 1.23 2.16 1.03 1.13 0.48 2.09 Oracle 18 12.93 1.03 1.74 1.17 1.74 1.31 0.43 0.76 1.33 Oracle 19 8.66 1.43 1.45 1.45 1.45 1.43 0.02 0.98 1.02 Oracle 20 10.40 1.45 2.77 1.73 2.77 1.47 1.30 0.53 1.88 Oracle 21 17.22 1.45 2.80 1.72 2.73 1.47 1.26 0.54 1.85 Oracle 22 16.23 1.45 2.78 1.63 2.78 1.45 1.32 0.52 1.91 Oracle 23 18.69 1.03 2.70 1.24 2.70 1.03 1.67 0.38 2.63 Oracle 24 11.75 0.94 2.27 1.07 2.27 0.95 1.31 0.42 2.39 Oracle 25 24.00 2.65 2.70 2.66 2.70 2.67 0.02 0.99 1.01 Oracle 26 26.52 1.02 2.67 1.77 2.66 1.02 1.64 0.38 2.60 Oracle 27 11.79 0.61 1.06 0.91 1.06 0.95 0.12 0.89 1.13 Oracle 28 11.52 0.94 2.27 1.65 2.26 0.95 1.31 0.42 2.39 Oracle 29 8.13 1.03 2.73 1.63 2.73 1.04 1.69 0.38 2.63 Oracle 30 10.87 0.61 1.69 0.84 1.69 0.87 0.81 0.52 1.94 Oracle 31 13.28 1.03 2.63 1.21 2.63 1.04 1.59 0.40 2.53 Oracle 32 16.23 1.03 2.52 1.16 2.52 1.03 1.49 0.41 2.45 Oracle 33 14.04 1.02 2.69 1.17 2.69 1.03 1.66 0.38 2.60 Oracle 34 10.26 1.03 2.70 1.28 2.70 1.04 1.66 0.38 2.59 Oracle 35 10.05 0.61 2.82 1.01 2.82 0.62 2.20 0.22 4.56 Oracle 36 13.34 1.02 1.04 1.02 1.02 1.02 0.00 1.00 1.00 Oracle 37 9.66 0.61 2.18 0.88 2.18 0.63 1.55 0.29 3.49 Oracle 38 10.26 0.61 2.17 0.93 2.17 0.61 1.56 0.28 3.54 Oracle 39 8.92 1.03 2.70 1.27 2.70 1.04 1.66 0.38 2.59 Oracle 40 22.53 1.45 2.73 1.61 2.73 1.46 1.26 0.54 1.86 Oracle 41 12.44 1.45 2.16 1.55 2.16 1.45 0.70 0.67 1.48 Oracle 42 2.56 2.56 2.56 2.56 2.56 2.56 0.00 1.00 1.00 Oracle 43 18.10 1.46 2.99 1.81 2.99 1.83 1.16 0.61 1.63 Oracle 44 11.18 1.46 2.70 1.86 2.70 1.84 0.87 0.68 1.48 Oracle 45 14.26 1.45 1.84 1.59 1.45 1.84 -0.38 1.26 0.79 Oracle 46 9.39 0.61 2.78 0.94 2.78 0.62 2.16 0.22 4.52 Oracle 47 9.68 0.61 2.27 0.97 2.27 0.62 1.65 0.27 3.65 Oracle 48 12.75 0.61 2.27 0.98 2.27 0.98 1.30 0.43 2.33 Oracle 49 7.88 0.61 0.95 0.79 0.95 0.95 0.01 0.99 1.01 Oracle 50 24.54 2.71 2.78 2.73 2.72 2.72 0.00 1.00 1.00 Total 652.10 54.91 108.61 67.67 106.07 60.61 45.45 30.73 97.87 Average 13.04 1.10 2.17 1.35 2.13 1.21 0.91 0.62 1.96 Micro avg ····················· 0.57 1.75 228 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 1.80 1.08 1.05 1.48 1.06 1.06 1.05 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 Oracle 2 2.20 1.39 1.04 1.05 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 3 1.95 1.43 1.05 1.05 1.05 1.04 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 4 2.24 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 5 1.46 1.31 0.88 0.88 0.88 0.88 0.88 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 Oracle 6 1.77 1.05 1.05 1.04 1.04 1.04 1.04 1.05 1.04 1.04 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 7 2.13 2.33 1.38 1.38 1.38 1.38 1.04 1.03 1.05 1.38 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 8 1.45 2.62 0.95 1.36 1.03 1.04 1.44 1.03 1.05 1.37 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 9 2.63 0.91 1.04 1.34 1.33 1.33 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 Oracle 10 1.32 1.04 1.05 1.33 0.62 0.62 0.62 0.62 1.30 1.02 1.03 1.03 1.03 1.31 1.04 1.32 1.03 1.04 1.04 1.03 Oracle 11 1.41 1.03 1.90 1.45 1.46 1.46 1.46 1.52 1.46 1.47 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 12 1.36 1.03 1.02 1.45 1.46 1.45 1.45 1.48 1.47 1.46 1.45 1.46 1.50 1.50 1.50 1.50 1.50 1.50 1.50 1.50 Oracle 13 1.49 1.49 1.49 1.38 1.35 1.03 1.03 1.45 1.45 1.48 1.45 1.46 1.45 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 14 1.37 1.37 0.92 1.33 1.33 1.34 1.37 1.34 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 Oracle 15 0.93 0.62 0.62 0.61 0.62 0.61 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 16 1.84 1.34 1.05 1.04 1.34 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 17 2.16 1.36 1.04 1.04 1.05 1.03 1.03 1.35 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 18 1.74 1.33 1.04 1.04 1.03 1.31 1.03 1.03 1.03 1.04 1.31 1.31 1.31 1.31 1.31 1.31 1.31 1.31 1.31 1.31 Oracle 19 1.45 1.45 1.45 1.45 1.45 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 Oracle 20 2.77 1.80 1.45 1.45 1.46 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 Oracle 21 2.73 2.80 1.45 1.46 1.46 1.46 1.46 1.46 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 Oracle 22 2.78 1.76 1.46 1.46 1.45 1.46 1.48 1.47 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 Oracle 23 2.70 2.54 1.04 1.03 1.03 1.05 1.04 1.04 1.03 1.03 1.03 1.03 1.03 1.04 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 24 2.27 0.95 0.95 0.95 0.94 0.95 0.95 0.95 0.94 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 25 2.70 2.68 2.65 2.66 2.66 2.66 2.66 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 26 2.66 2.66 2.65 2.66 2.67 2.65 2.32 1.02 1.02 1.03 1.02 1.05 1.02 1.03 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 27 1.06 0.62 0.94 0.61 0.95 0.95 0.96 0.96 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 28 2.26 2.27 1.44 1.45 2.23 0.94 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 29 2.73 2.30 1.04 1.03 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 30 1.69 0.87 0.87 0.63 0.87 0.61 0.62 0.87 0.61 0.62 0.88 0.88 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 Oracle 31 2.63 1.04 1.04 1.03 1.03 1.04 1.34 1.03 1.03 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 32 2.52 1.30 1.03 1.04 1.04 1.03 1.04 1.04 1.03 1.03 1.03 1.04 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 33 2.69 1.04 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.02 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 34 2.70 1.35 1.04 1.03 1.03 1.03 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 35 2.82 0.95 0.96 0.61 0.95 0.96 0.62 0.62 0.95 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 36 1.02 1.02 1.02 1.02 1.04 1.02 1.03 1.02 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 37 2.18 0.62 0.94 0.95 0.62 0.61 0.95 0.95 0.61 0.62 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 Oracle 38 2.17 1.05 0.61 0.62 0.91 0.91 0.62 0.91 0.92 0.92 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 39 2.70 1.04 1.03 1.03 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 40 2.73 2.21 1.47 1.45 1.45 1.45 1.52 1.46 1.50 1.45 1.45 1.46 1.45 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 41 2.16 1.47 1.46 1.47 1.45 1.45 1.53 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 Oracle 42 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 2.56 Oracle 43 2.99 1.87 1.83 1.55 1.47 1.83 1.82 1.46 1.46 1.83 1.83 1.83 1.83 1.83 1.83 1.83 1.83 1.83 1.83 1.83 Oracle 44 2.70 1.86 1.46 1.48 1.86 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 Oracle 45 1.45 1.46 1.46 1.46 1.83 1.47 1.46 1.83 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 Oracle 46 2.78 0.61 0.61 0.98 0.98 0.62 0.62 0.62 0.97 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 47 2.27 1.05 0.95 0.63 0.61 0.66 0.97 0.96 0.96 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 48 2.27 2.27 0.97 0.97 0.62 0.63 0.98 0.62 0.61 0.61 0.62 0.62 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 Oracle 49 0.95 0.63 0.95 0.95 0.95 0.61 0.66 0.62 0.61 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 50 2.72 2.78 2.73 2.72 2.71 2.72 2.71 2.73 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 Total 106 75 62 63 63 61 61 60 60 61 60 60 61 61 61 61 61 61 61 61 Average 2.13 1.49 1.24 1.25 1.27 1.22 1.23 1.20 1.21 1.21 1.20 1.20 1.21 1.22 1.21 1.22 1.21 1.21 1.21 1.21 D.1. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT229
Size Results of configuration Shattered Document
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 30 3 5 3.34 5 3 2 0.60 1.66 Oracle 2 22 3 6 3.66 6 3 3 0.50 2.00 Oracle 3 27 3 5 3.38 5 3 2 0.60 1.66 Oracle 4 24 3 6 3.43 6 3 3 0.50 2.00 Oracle 5 26 3 4 3.25 4 3 1 0.75 1.34 Oracle 6 35 3 5 3.18 5 3 2 0.60 1.66 Oracle 7 43 3 6 3.91 5 3 2 0.60 1.66 Oracle 8 41 3 7 3.73 4 3 1 0.75 1.34 Oracle 9 29 3 7 4.14 7 4 3 0.57 1.75 Oracle 10 61 2 4 3.05 4 3 1 0.75 1.34 Oracle 11 48 3 5 4.00 4 4 0 1.00 1.00 Oracle 12 50 3 4 3.84 4 4 0 1.00 1.00 Oracle 13 58 3 4 3.87 4 4 0 1.00 1.00 Oracle 14 43 3 4 3.91 4 4 0 1.00 1.00 Oracle 15 17 2 3 2.13 3 2 1 0.66 1.50 Oracle 16 21 3 4 3.50 4 3 1 0.75 1.34 Oracle 17 31 3 5 3.45 5 3 2 0.60 1.66 Oracle 18 38 3 5 3.45 5 4 1 0.80 1.25 Oracle 19 24 4 4 4.00 4 4 0 1.00 1.00 Oracle 20 28 4 7 4.66 7 4 3 0.57 1.75 Oracle 21 46 4 7 4.60 7 4 3 0.57 1.75 Oracle 22 44 4 7 4.40 7 4 3 0.57 1.75 Oracle 23 53 3 7 3.53 7 3 4 0.43 2.34 Oracle 24 36 3 6 3.27 6 3 3 0.50 2.00 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 72 3 7 4.80 7 3 4 0.43 2.34 Oracle 27 37 2 3 2.84 3 3 0 1.00 1.00 Oracle 28 32 3 6 4.57 6 3 3 0.50 2.00 Oracle 29 22 3 7 4.40 7 3 4 0.43 2.34 Oracle 30 36 2 5 2.77 5 3 2 0.60 1.66 Oracle 31 38 3 7 3.45 7 3 4 0.43 2.34 Oracle 32 47 3 7 3.36 7 3 4 0.43 2.34 Oracle 33 40 3 7 3.34 7 3 4 0.43 2.34 Oracle 34 29 3 7 3.63 7 3 4 0.43 2.34 Oracle 35 30 2 7 3.00 7 2 5 0.29 3.50 Oracle 36 39 3 3 3.00 3 3 0 1.00 1.00 Oracle 37 30 2 6 2.73 6 2 4 0.34 3.00 Oracle 38 32 2 6 2.91 6 2 4 0.34 3.00 Oracle 39 25 3 7 3.57 7 3 4 0.43 2.34 Oracle 40 61 4 7 4.36 7 4 3 0.57 1.75 Oracle 41 34 4 6 4.25 6 4 2 0.66 1.50 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 48 4 7 4.80 7 5 2 0.71 1.40 Oracle 44 30 4 7 5.00 7 5 2 0.71 1.40 Oracle 45 39 4 5 4.34 4 5 -1 1.25 0.80 Oracle 46 28 2 7 2.80 7 2 5 0.29 3.50 Oracle 47 29 2 6 2.90 6 2 4 0.34 3.00 Oracle 48 38 2 6 2.92 6 3 3 0.50 2.00 Oracle 49 25 2 3 2.50 3 3 0 1.00 1.00 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 1849 160 287 190.91 281 174 107 32.77 87.60 Average 36.98 3.20 5.74 3.82 5.62 3.48 2.14 0.66 1.75 Micro avg ····················· 0.62 1.62 230 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 5 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 2 6 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 3 5 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 4 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 5 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 6 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 7 5 6 4 4 4 4 3 3 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 8 4 7 3 4 3 3 4 3 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 9 7 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 10 4 3 3 4 2 2 2 2 4 3 3 3 3 4 3 4 3 3 3 3 Oracle 11 4 3 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 12 4 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 13 4 4 4 4 4 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 14 4 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 15 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 16 4 4 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 17 5 4 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 18 5 4 3 3 3 4 3 3 3 3 4 4 4 4 4 4 4 4 4 4 Oracle 19 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 20 7 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 21 7 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 22 7 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 23 7 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 24 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 7 7 7 7 7 7 6 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 27 3 2 3 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 28 6 6 4 4 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 29 7 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 30 5 3 3 2 3 2 2 3 2 2 3 3 3 3 3 3 3 3 3 3 Oracle 31 7 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 32 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 33 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 34 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 35 7 3 3 2 3 3 2 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 36 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 37 6 2 3 3 2 2 3 3 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 38 6 3 2 2 3 3 2 3 3 3 2 2 2 2 2 2 2 2 2 2 Oracle 39 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 40 7 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 41 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 7 5 5 4 4 5 5 4 4 5 5 5 5 5 5 5 5 5 5 5 Oracle 44 7 5 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 45 4 4 4 4 5 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 46 7 2 2 3 3 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 47 6 3 3 2 2 2 3 3 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 48 6 6 3 3 2 2 3 2 2 2 2 2 3 3 3 3 3 3 3 3 Oracle 49 3 2 3 3 3 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 281 209 178 179 182 175 175 173 174 174 173 173 174 175 174 175 174 174 174 174 Average 5.62 4.18 3.56 3.58 3.64 3.50 3.50 3.46 3.48 3.48 3.46 3.46 3.48 3.50 3.48 3.50 3.48 3.48 3.48 3.48 D.1. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT231
Cost Results of configuration Shattered Document
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 30.00 3.00 5.00 3.34 5.00 3.00 2.00 0.60 1.66 Oracle 2 22.00 3.00 6.00 3.66 6.00 3.00 3.00 0.50 2.00 Oracle 3 27.00 3.00 5.00 3.38 5.00 3.00 2.00 0.60 1.66 Oracle 4 24.00 3.00 6.00 3.43 6.00 3.00 3.00 0.50 2.00 Oracle 5 26.00 3.00 4.00 3.25 4.00 3.00 1.00 0.75 1.34 Oracle 6 35.00 3.00 5.00 3.18 5.00 3.00 2.00 0.60 1.66 Oracle 7 43.00 3.00 6.00 3.91 5.00 3.00 2.00 0.60 1.66 Oracle 8 41.00 3.00 7.00 3.73 4.00 3.00 1.00 0.75 1.34 Oracle 9 29.00 3.00 7.00 4.14 7.00 4.00 3.00 0.57 1.75 Oracle 10 61.00 2.00 4.00 3.05 4.00 3.00 1.00 0.75 1.34 Oracle 11 48.00 3.00 5.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 12 50.00 3.00 4.00 3.84 4.00 4.00 0.00 1.00 1.00 Oracle 13 58.00 3.00 4.00 3.87 4.00 4.00 0.00 1.00 1.00 Oracle 14 43.00 3.00 4.00 3.91 4.00 4.00 0.00 1.00 1.00 Oracle 15 17.00 2.00 3.00 2.13 3.00 2.00 1.00 0.66 1.50 Oracle 16 21.00 3.00 4.00 3.50 4.00 3.00 1.00 0.75 1.34 Oracle 17 31.00 3.00 5.00 3.45 5.00 3.00 2.00 0.60 1.66 Oracle 18 38.00 3.00 5.00 3.45 5.00 4.00 1.00 0.80 1.25 Oracle 19 24.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 20 28.00 4.00 7.00 4.66 7.00 4.00 3.00 0.57 1.75 Oracle 21 46.00 4.00 7.00 4.60 7.00 4.00 3.00 0.57 1.75 Oracle 22 44.00 4.00 7.00 4.40 7.00 4.00 3.00 0.57 1.75 Oracle 23 53.00 3.00 7.00 3.53 7.00 3.00 4.00 0.43 2.34 Oracle 24 36.00 3.00 6.00 3.27 6.00 3.00 3.00 0.50 2.00 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 72.00 3.00 7.00 4.80 7.00 3.00 4.00 0.43 2.34 Oracle 27 37.00 2.00 3.00 2.84 3.00 3.00 0.00 1.00 1.00 Oracle 28 32.00 3.00 6.00 4.57 6.00 3.00 3.00 0.50 2.00 Oracle 29 22.00 3.00 7.00 4.40 7.00 3.00 4.00 0.43 2.34 Oracle 30 36.00 2.00 5.00 2.77 5.00 3.00 2.00 0.60 1.66 Oracle 31 38.00 3.00 7.00 3.45 7.00 3.00 4.00 0.43 2.34 Oracle 32 47.00 3.00 7.00 3.36 7.00 3.00 4.00 0.43 2.34 Oracle 33 40.00 3.00 7.00 3.34 7.00 3.00 4.00 0.43 2.34 Oracle 34 29.00 3.00 7.00 3.63 7.00 3.00 4.00 0.43 2.34 Oracle 35 30.00 2.00 7.00 3.00 7.00 2.00 5.00 0.29 3.50 Oracle 36 39.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 37 30.00 2.00 6.00 2.73 6.00 2.00 4.00 0.34 3.00 Oracle 38 32.00 2.00 6.00 2.91 6.00 2.00 4.00 0.34 3.00 Oracle 39 25.00 3.00 7.00 3.57 7.00 3.00 4.00 0.43 2.34 Oracle 40 61.00 4.00 7.00 4.36 7.00 4.00 3.00 0.57 1.75 Oracle 41 34.00 4.00 6.00 4.25 6.00 4.00 2.00 0.66 1.50 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 48.00 4.00 7.00 4.80 7.00 5.00 2.00 0.71 1.40 Oracle 44 30.00 4.00 7.00 5.00 7.00 5.00 2.00 0.71 1.40 Oracle 45 39.00 4.00 5.00 4.34 4.00 5.00 -1.00 1.25 0.80 Oracle 46 28.00 2.00 7.00 2.80 7.00 2.00 5.00 0.29 3.50 Oracle 47 29.00 2.00 6.00 2.90 6.00 2.00 4.00 0.34 3.00 Oracle 48 38.00 2.00 6.00 2.92 6.00 3.00 3.00 0.50 2.00 Oracle 49 25.00 2.00 3.00 2.50 3.00 3.00 0.00 1.00 1.00 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 1849.00 160.00 287.00 190.91 281.00 174.00 107.00 32.77 87.60 Average 36.98 3.20 5.74 3.82 5.62 3.48 2.14 0.66 1.75 Micro avg ····················· 0.62 1.62 232 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 5.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 2 6.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 3 5.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 4 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 5 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 6 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 7 5.00 6.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 8 4.00 7.00 3.00 4.00 3.00 3.00 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 9 7.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 10 4.00 3.00 3.00 4.00 2.00 2.00 2.00 2.00 4.00 3.00 3.00 3.00 3.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 Oracle 11 4.00 3.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 12 4.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 13 4.00 4.00 4.00 4.00 4.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 14 4.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 15 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 16 4.00 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 17 5.00 4.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 18 5.00 4.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 19 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 20 7.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 21 7.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 22 7.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 23 7.00 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 24 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 7.00 7.00 7.00 7.00 7.00 7.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 27 3.00 2.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 28 6.00 6.00 4.00 4.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 29 7.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 30 5.00 3.00 3.00 2.00 3.00 2.00 2.00 3.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 31 7.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 32 7.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 33 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 34 7.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 35 7.00 3.00 3.00 2.00 3.00 3.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 36 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 37 6.00 2.00 3.00 3.00 2.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 38 6.00 3.00 2.00 2.00 3.00 3.00 2.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 39 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 40 7.00 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 41 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 7.00 5.00 5.00 4.00 4.00 5.00 5.00 4.00 4.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 44 7.00 5.00 4.00 4.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 45 4.00 4.00 4.00 4.00 5.00 4.00 4.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 46 7.00 2.00 2.00 3.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 47 6.00 3.00 3.00 2.00 2.00 2.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 48 6.00 6.00 3.00 3.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 49 3.00 2.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 281 209 178 179 182 175 175 173 174 174 173 173 174 175 174 175 174 174 174 174 Average 5.62 4.18 3.56 3.58 3.64 3.50 3.50 3.46 3.48 3.48 3.46 3.46 3.48 3.50 3.48 3.50 3.48 3.48 3.48 3.48 D.2. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM CLICKS233 D.2 Complete results of configuration Shattered Document with Random Clicks
Shattered Document format with random clicks i.e. random choices by the user. Using functions Super Page, Recentre from Kasim2.Activation, Random Click from Kasim2.Comparate.Experiment (body).
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.02
Size0 = 6.90
Size1 = 6.98 Size gain = 0.99
Cost0 = 6.90
Cost1 = 6.98 Cost gain = 0.99 234 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Random Clicks
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 8 0 8 1 0.89 0.11 Oracle 2 30 6 0 6 0 6 0 1.00 0.00 Oracle 3 40 8 0 8 0 8 0 1.00 0.00 Oracle 4 46 7 0 7 0 7 0 1.00 0.00 Oracle 5 56 8 0 8 0 8 0 1.00 0.00 Oracle 6 85 11 0 11 0 11 0 1.00 0.00 Oracle 7 108 11 0 11 0 11 0 1.00 0.00 Oracle 8 125 11 0 11 0 11 0 1.00 0.00 Oracle 9 128 7 0 7 0 7 0 1.00 0.00 Oracle 10 130 20 0 20 0 20 0 1.00 0.00 Oracle 11 138 12 0 12 0 12 0 1.00 0.00 Oracle 12 157 13 0 13 0 13 0 1.00 0.00 Oracle 13 171 15 0 15 0 15 0 1.00 0.00 Oracle 14 173 11 0 11 0 11 0 1.00 0.00 Oracle 15 241 8 0 7 0 7 1 0.88 0.13 Oracle 16 265 6 0 6 0 6 0 1.00 0.00 Oracle 17 266 9 0 9 0 9 0 1.00 0.00 Oracle 18 269 11 0 11 0 11 0 1.00 0.00 Oracle 19 290 6 0 6 0 6 0 1.00 0.00 Oracle 20 307 6 0 6 0 6 0 1.00 0.00 Oracle 21 366 10 0 9 0 9 1 0.90 0.10 Oracle 22 410 10 0 10 0 10 0 1.00 0.00 Oracle 23 455 15 0 15 0 15 0 1.00 0.00 Oracle 24 508 11 0 11 0 11 0 1.00 0.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 15 0 15 0 1.00 0.00 Oracle 27 516 13 0 13 0 13 0 1.00 0.00 Oracle 28 521 7 0 7 0 7 0 1.00 0.00 Oracle 29 542 5 0 5 0 5 0 1.00 0.00 Oracle 30 554 13 0 13 0 13 0 1.00 0.00 Oracle 31 601 11 0 11 0 11 0 1.00 0.00 Oracle 32 606 14 0 13 0 13 1 0.93 0.07 Oracle 33 616 12 0 12 0 12 0 1.00 0.00 Oracle 34 649 8 0 8 0 8 0 1.00 0.00 Oracle 35 723 10 0 10 0 10 0 1.00 0.00 Oracle 36 725 13 0 13 0 13 0 1.00 0.00 Oracle 37 738 11 0 11 0 11 0 1.00 0.00 Oracle 38 741 11 0 11 0 11 0 1.00 0.00 Oracle 39 786 7 0 7 0 7 0 1.00 0.00 Oracle 40 840 14 0 13 0 13 1 0.93 0.07 Oracle 41 846 8 0 8 0 8 0 1.00 0.00 Oracle 42 883 1 0 1 0 1 0 1.00 0.00 Oracle 43 890 10 0 10 0 10 0 1.00 0.00 Oracle 44 897 6 0 6 0 6 0 1.00 0.00 Oracle 45 899 9 0 9 0 9 0 1.00 0.00 Oracle 46 929 10 0 10 0 10 0 1.00 0.00 Oracle 47 936 10 0 9 0 9 1 0.90 0.10 Oracle 48 939 13 0 13 0 13 0 1.00 0.00 Oracle 49 944 10 0 10 0 10 0 1.00 0.00 Oracle 50 999 9 0 8 0 8 1 0.89 0.11 Total ··· 500 0 493 0 493 7 49.31 0.69 Average ··· 10.00 0.00 9.86 0.00 9.86 0.14 0.98 0.02 Micro avg ····················· 0.98 0.02 D.2. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM CLICKS235
Duration Results of configuration Shattered Document with Random Clicks
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 8.31 0.53 1.05 0.92 1.05 0.91 0.13 0.88 1.14 Oracle 2 26.07 0.95 21.23 4.34 21.23 0.96 20.26 0.05 22.01 Oracle 3 8.13 0.93 1.09 1.02 1.09 0.98 0.10 0.91 1.10 Oracle 4 26.81 0.89 21.11 3.83 21.11 0.89 20.22 0.04 23.63 Oracle 5 7.52 0.91 1.02 0.94 0.91 0.93 -0.02 1.03 0.97 Oracle 6 11.01 0.90 1.17 1.00 1.08 0.91 0.17 0.84 1.19 Oracle 7 10.62 0.90 1.15 0.97 1.07 0.91 0.16 0.85 1.18 Oracle 8 10.95 0.91 1.10 0.99 1.10 0.94 0.16 0.85 1.17 Oracle 9 7.17 0.91 1.25 1.02 1.25 0.97 0.28 0.77 1.29 Oracle 10 30.95 0.94 11.15 1.55 1.10 0.94 0.17 0.84 1.19 Oracle 11 11.70 0.91 1.20 0.98 1.20 0.95 0.25 0.79 1.27 Oracle 12 12.95 0.91 1.23 0.99 1.23 0.92 0.31 0.75 1.34 Oracle 13 15.45 0.91 1.35 1.03 1.17 0.93 0.24 0.80 1.26 Oracle 14 21.13 0.90 11.05 1.92 1.37 0.90 0.47 0.66 1.52 Oracle 15 6.92 0.25 1.05 0.87 0.25 0.88 -0.63 3.54 0.28 Oracle 16 6.02 0.91 1.21 1.01 1.21 0.93 0.29 0.77 1.31 Oracle 17 9.68 0.96 1.69 1.08 1.69 1.00 0.69 0.59 1.70 Oracle 18 10.80 0.91 1.20 0.98 1.20 0.91 0.28 0.77 1.30 Oracle 19 16.00 0.91 11.11 2.66 11.11 0.91 10.20 0.09 12.13 Oracle 20 6.04 0.90 1.21 1.01 1.21 0.90 0.31 0.74 1.35 Oracle 21 9.62 0.91 1.08 0.96 0.93 1.04 -0.11 1.12 0.90 Oracle 22 20.20 0.93 11.11 2.02 1.16 0.96 0.20 0.84 1.20 Oracle 23 14.67 0.90 1.10 0.98 1.09 0.90 0.19 0.83 1.21 Oracle 24 10.50 0.90 1.05 0.95 0.99 0.90 0.09 0.91 1.10 Oracle 25 9.07 0.92 1.10 1.01 1.06 0.92 0.13 0.88 1.15 Oracle 26 14.39 0.91 1.12 0.96 1.01 0.92 0.09 0.91 1.09 Oracle 27 13.20 0.88 1.43 1.02 1.43 1.01 0.42 0.70 1.42 Oracle 28 7.67 0.91 1.35 1.09 1.08 0.91 0.16 0.85 1.18 Oracle 29 4.94 0.93 1.11 0.98 1.11 0.97 0.14 0.88 1.14 Oracle 30 22.91 0.91 11.02 1.77 1.07 1.05 0.02 0.98 1.02 Oracle 31 11.08 0.91 1.37 1.01 1.37 0.94 0.42 0.69 1.45 Oracle 32 13.30 0.77 1.17 0.95 0.98 0.77 0.20 0.80 1.26 Oracle 33 11.66 0.89 1.14 0.97 1.02 0.94 0.09 0.91 1.10 Oracle 34 7.45 0.91 0.95 0.93 0.95 0.93 0.02 0.98 1.02 Oracle 35 9.89 0.91 1.09 0.99 1.09 0.91 0.19 0.83 1.21 Oracle 36 12.70 0.91 1.13 0.98 1.02 0.98 0.04 0.96 1.04 Oracle 37 11.15 0.91 1.30 1.02 1.30 1.01 0.30 0.77 1.30 Oracle 38 11.64 0.91 1.31 1.05 1.31 0.95 0.36 0.73 1.38 Oracle 39 6.62 0.89 1.01 0.95 1.00 0.89 0.10 0.90 1.12 Oracle 40 13.88 0.89 1.20 0.99 1.10 0.89 0.20 0.81 1.23 Oracle 41 17.95 0.91 11.01 2.24 0.99 0.97 0.02 0.98 1.02 Oracle 42 1.02 1.02 1.02 1.02 1.02 1.02 0.00 1.00 1.00 Oracle 43 10.05 0.90 1.46 1.01 1.46 0.90 0.56 0.62 1.63 Oracle 44 5.88 0.92 1.12 0.98 1.12 0.95 0.16 0.86 1.17 Oracle 45 8.86 0.91 1.15 0.98 1.15 0.91 0.23 0.80 1.26 Oracle 46 20.27 0.95 11.23 2.02 11.23 0.98 10.25 0.09 11.48 Oracle 47 20.23 0.90 11.12 2.02 1.15 1.05 0.10 0.91 1.09 Oracle 48 12.77 0.91 1.12 0.98 1.12 1.09 0.03 0.98 1.02 Oracle 49 9.52 0.90 1.02 0.95 1.02 0.98 0.03 0.97 1.03 Oracle 50 18.45 0.91 10.74 2.05 0.97 0.92 0.05 0.95 1.05 Total 625.80 44.52 187.73 64.95 115.87 47.15 68.73 41.93 123.57 Average 12.52 0.89 3.76 1.30 2.32 0.95 1.38 0.84 2.47 Micro avg ····················· 0.41 2.46 236 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Random Clicks
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 1.05 1.04 0.98 0.93 0.93 0.53 0.91 1.02 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 2 21.23 1.01 0.95 0.98 0.95 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 Oracle 3 1.09 1.05 0.98 0.98 1.09 0.93 1.03 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 Oracle 4 21.11 1.02 0.94 0.99 0.93 0.94 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 Oracle 5 0.91 0.92 0.97 0.91 1.02 0.91 0.96 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 Oracle 6 1.08 1.17 0.95 1.03 1.07 0.95 1.00 0.99 0.96 0.90 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 7 1.07 1.15 0.91 0.98 0.93 0.93 0.92 0.95 0.97 0.90 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 8 1.10 1.07 1.02 1.02 0.91 0.92 1.01 1.04 0.97 0.97 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 Oracle 9 1.25 1.03 1.13 0.96 0.92 0.91 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 Oracle 10 1.10 1.09 1.09 1.11 1.03 1.09 11.15 1.10 0.98 0.94 1.13 1.29 0.94 0.94 0.95 0.98 1.07 0.98 1.06 0.94 Oracle 11 1.20 0.95 0.98 0.94 0.91 0.91 1.03 0.95 1.02 0.92 0.91 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 12 1.23 1.03 1.08 0.96 0.97 0.91 1.01 0.94 0.96 0.99 1.00 0.93 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 Oracle 13 1.17 1.35 1.02 1.19 1.03 0.97 0.99 1.03 1.02 0.95 0.95 0.91 0.98 0.95 0.93 0.93 0.93 0.93 0.93 0.93 Oracle 14 1.37 1.16 11.05 1.00 0.93 0.93 0.91 1.00 0.95 0.95 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 15 0.25 0.94 1.05 1.03 0.97 0.89 0.91 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 Oracle 16 1.21 0.91 0.97 0.95 1.04 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 Oracle 17 1.69 1.14 0.97 0.97 1.00 0.97 0.96 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 Oracle 18 1.20 1.02 1.00 1.15 0.94 0.93 0.92 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 19 11.11 1.09 0.95 0.98 0.97 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 20 1.21 0.98 1.07 0.91 0.96 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 21 0.93 0.93 0.94 0.98 0.94 0.91 0.95 0.93 1.08 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 22 1.16 1.14 11.11 0.99 0.98 0.97 0.96 0.93 1.01 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 Oracle 23 1.09 0.98 1.10 0.96 0.99 1.00 0.95 0.98 0.98 0.99 0.98 0.92 0.92 0.91 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 24 0.99 1.05 1.05 0.95 0.91 0.94 0.92 0.92 0.94 0.94 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 25 1.06 1.03 1.00 1.05 1.05 1.10 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 Oracle 26 1.01 1.01 1.12 0.98 0.98 0.97 0.91 0.93 0.91 0.91 0.94 0.91 0.98 0.92 0.92 0.92 0.92 0.92 0.92 0.92 Oracle 27 1.43 1.20 1.12 1.00 1.03 0.94 0.93 0.95 0.90 0.91 0.88 0.91 1.01 1.01 1.01 1.01 1.01 1.01 1.01 1.01 Oracle 28 1.08 1.35 1.14 1.17 1.09 0.92 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 29 1.11 0.95 0.98 0.93 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 Oracle 30 1.07 0.95 11.02 1.20 0.93 0.92 0.97 0.99 0.98 0.91 0.95 0.97 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 31 1.37 1.07 1.16 0.91 0.99 0.91 0.95 0.93 0.91 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 Oracle 32 0.98 0.94 0.98 1.17 1.00 0.94 0.91 0.93 0.92 0.91 1.05 0.91 0.91 0.77 0.77 0.77 0.77 0.77 0.77 0.77 Oracle 33 1.02 0.98 1.02 1.14 0.89 0.99 0.94 0.91 0.97 0.95 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 Oracle 34 0.95 0.91 0.94 0.94 0.92 0.94 0.92 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 Oracle 35 1.09 1.03 0.95 1.09 0.98 0.91 0.93 1.00 0.98 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 36 1.02 0.97 1.01 0.94 0.96 0.95 1.13 0.91 1.01 1.00 0.91 0.92 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 Oracle 37 1.30 0.97 0.99 1.02 1.03 0.95 0.98 1.05 0.91 0.93 1.01 1.01 1.01 1.01 1.01 1.01 1.01 1.01 1.01 1.01 Oracle 38 1.31 1.05 1.01 1.16 1.04 1.21 0.97 1.11 0.91 0.92 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 39 1.00 0.91 1.01 0.91 0.97 0.92 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 Oracle 40 1.10 1.20 0.98 1.01 0.91 1.02 0.98 0.98 0.95 0.92 1.05 0.91 0.99 0.89 0.89 0.89 0.89 0.89 0.89 0.89 Oracle 41 0.99 11.01 1.21 0.92 0.93 1.01 0.91 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 Oracle 42 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 43 1.46 1.02 0.97 1.01 0.94 0.92 0.93 0.95 0.96 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 44 1.12 1.02 0.92 0.93 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 45 1.15 1.05 0.96 0.97 0.92 0.91 1.00 0.98 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 46 11.23 0.98 1.09 1.02 1.09 0.95 0.98 0.98 0.96 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 Oracle 47 1.15 1.13 11.12 0.95 1.00 0.99 0.95 1.00 0.90 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 48 1.12 1.00 0.95 0.97 0.93 0.95 0.93 1.05 0.91 0.92 1.00 0.95 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 49 1.02 0.94 0.95 1.02 0.97 0.91 0.92 0.91 0.90 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 Oracle 50 0.97 10.74 0.98 1.03 1.00 0.96 0.91 0.94 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 Total 116 72 91 50 49 47 58 48 47 47 48 47 48 47 47 47 47 47 47 47 Average 2.32 1.43 1.82 1.01 0.98 0.95 1.16 0.96 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 D.2. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM CLICKS237
Size Results of configuration Shattered Document with Ran- dom Clicks
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 60 4 7 6.66 7 7 0 1.00 1.00 Oracle 2 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 3 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 4 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 5 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 6 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 7 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 8 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 9 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 10 140 7 7 7.00 7 7 0 1.00 1.00 Oracle 11 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 12 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 13 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 14 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 15 51 2 7 6.38 2 7 -5 3.50 0.29 Oracle 16 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 17 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 18 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 19 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 20 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 21 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 22 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 23 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 24 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 27 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 28 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 29 35 7 7 7.00 7 7 0 1.00 1.00 Oracle 30 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 31 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 32 97 6 7 6.93 7 6 1 0.86 1.16 Oracle 33 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 34 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 35 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 36 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 37 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 38 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 39 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 40 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 41 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 44 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 45 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 46 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 47 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 48 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 49 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 50 61 5 7 6.78 7 7 0 1.00 1.00 Total 3489 339 350 348.75 345 349 -4 52.36 49.45 Average 69.78 6.78 7.00 6.98 6.90 6.98 -0.08 1.05 0.99 Micro avg ····················· 1.01 0.99 238 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Ran- dom Clicks
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7 7 7 7 7 4 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 2 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 3 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 4 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 5 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 6 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 8 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 9 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 10 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 11 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 12 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 13 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 14 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 15 2 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 16 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 17 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 18 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 19 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 20 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 21 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 22 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 23 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 24 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 27 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 28 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 29 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 30 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 31 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 32 7 7 7 7 7 7 7 7 7 7 7 7 7 6 6 6 6 6 6 6 Oracle 33 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 34 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 35 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 36 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 37 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 38 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 39 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 40 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 41 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 44 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 45 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 46 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 47 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 48 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 49 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 50 7 5 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 345 348 350 350 350 347 350 350 350 350 350 350 350 349 349 349 349 349 349 349 Average 6.90 6.96 7.00 7.00 7.00 6.94 7.00 7.00 7.00 7.00 7.00 7.00 7.00 6.98 6.98 6.98 6.98 6.98 6.98 6.98 D.2. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM CLICKS239
Cost Results of configuration Shattered Document with Ran- dom Clicks
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 60.00 4.00 7.00 6.66 7.00 7.00 0.00 1.00 1.00 Oracle 2 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 3 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 4 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 5 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 6 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 7 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 8 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 9 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 10 140.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 11 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 12 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 13 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 14 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 15 51.00 2.00 7.00 6.38 2.00 7.00 -5.00 3.50 0.29 Oracle 16 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 17 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 18 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 19 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 20 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 21 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 22 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 23 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 24 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 27 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 28 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 29 35.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 30 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 31 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 32 97.00 6.00 7.00 6.93 7.00 6.00 1.00 0.86 1.16 Oracle 33 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 34 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 35 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 36 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 37 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 38 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 39 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 40 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 41 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 44 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 45 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 46 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 47 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 48 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 49 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 50 61.00 5.00 7.00 6.78 7.00 7.00 0.00 1.00 1.00 Total 3489.00 339.00 350.00 348.75 345.00 349.00 -4.00 52.36 49.45 Average 69.78 6.78 7.00 6.98 6.90 6.98 -0.08 1.05 0.99 Micro avg ····················· 1.01 0.99 240 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Ran- dom Clicks
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7.00 7.00 7.00 7.00 7.00 4.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 2 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 3 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 4 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 5 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 6 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 7 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 8 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 9 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 10 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 11 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 12 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 13 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 14 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 15 2.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 16 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 17 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 18 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 19 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 20 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 21 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 22 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 23 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 24 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 27 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 28 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 29 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 30 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 31 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 32 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 33 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 34 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 35 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 36 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 37 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 38 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 39 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 40 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 41 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 44 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 45 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 46 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 47 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 48 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 49 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 50 7.00 5.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 345 348 350 350 350 347 350 350 350 350 350 350 350 349 349 349 349 349 349 349 Average 6.90 6.96 7.00 7.00 7.00 6.94 7.00 7.00 7.00 7.00 7.00 7.00 7.00 6.98 6.98 6.98 6.98 6.98 6.98 6.98 D.3. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM PAGES241 D.3 Complete results of configuration Shattered Document with Random Pages
Shattered Document format with random pages. Using functions Super Page, Choose, Random Page from Kasim2.Activation.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.01
Size0 = 7.00
Size1 = 7.00 Size gain = 1.00
Cost0 = 7.00
Cost1 = 7.00 Cost gain = 1.00 242 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Random Pages
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 9 0 9 0 1.00 0.00 Oracle 2 30 6 0 6 0 6 0 1.00 0.00 Oracle 3 40 8 0 8 0 8 0 1.00 0.00 Oracle 4 46 7 0 7 0 7 0 1.00 0.00 Oracle 5 56 8 0 8 0 8 0 1.00 0.00 Oracle 6 85 11 0 11 0 11 0 1.00 0.00 Oracle 7 108 11 0 11 0 11 0 1.00 0.00 Oracle 8 125 11 0 11 0 11 0 1.00 0.00 Oracle 9 128 7 0 7 0 7 0 1.00 0.00 Oracle 10 130 20 0 20 0 20 0 1.00 0.00 Oracle 11 138 12 0 12 0 12 0 1.00 0.00 Oracle 12 157 13 0 13 0 13 0 1.00 0.00 Oracle 13 171 15 0 15 0 15 0 1.00 0.00 Oracle 14 173 11 0 11 0 11 0 1.00 0.00 Oracle 15 241 8 0 8 0 8 0 1.00 0.00 Oracle 16 265 6 0 6 0 6 0 1.00 0.00 Oracle 17 266 9 0 9 0 9 0 1.00 0.00 Oracle 18 269 11 0 11 0 11 0 1.00 0.00 Oracle 19 290 6 0 6 0 6 0 1.00 0.00 Oracle 20 307 6 0 6 0 6 0 1.00 0.00 Oracle 21 366 10 0 10 0 10 0 1.00 0.00 Oracle 22 410 10 0 10 0 10 0 1.00 0.00 Oracle 23 455 15 0 15 0 15 0 1.00 0.00 Oracle 24 508 11 0 11 0 11 0 1.00 0.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 15 0 15 0 1.00 0.00 Oracle 27 516 13 0 13 0 13 0 1.00 0.00 Oracle 28 521 7 0 7 0 7 0 1.00 0.00 Oracle 29 542 5 0 5 0 5 0 1.00 0.00 Oracle 30 554 13 0 13 0 13 0 1.00 0.00 Oracle 31 601 11 0 11 0 11 0 1.00 0.00 Oracle 32 606 14 0 14 0 14 0 1.00 0.00 Oracle 33 616 12 0 12 0 12 0 1.00 0.00 Oracle 34 649 8 0 8 0 8 0 1.00 0.00 Oracle 35 723 10 0 9 0 9 1 0.90 0.10 Oracle 36 725 13 0 12 0 12 1 0.92 0.08 Oracle 37 738 11 0 11 0 11 0 1.00 0.00 Oracle 38 741 11 0 11 0 11 0 1.00 0.00 Oracle 39 786 7 0 7 0 7 0 1.00 0.00 Oracle 40 840 14 0 13 0 13 1 0.93 0.07 Oracle 41 846 8 0 7 0 7 1 0.88 0.13 Oracle 42 883 1 0 1 0 1 0 1.00 0.00 Oracle 43 890 10 0 10 0 10 0 1.00 0.00 Oracle 44 897 6 0 6 0 6 0 1.00 0.00 Oracle 45 899 9 0 9 0 9 0 1.00 0.00 Oracle 46 929 10 0 10 0 10 0 1.00 0.00 Oracle 47 936 10 0 10 0 10 0 1.00 0.00 Oracle 48 939 13 0 13 0 13 0 1.00 0.00 Oracle 49 944 10 0 10 0 10 0 1.00 0.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 496 0 496 4 49.63 0.38 Average ··· 10.00 0.00 9.92 0.00 9.92 0.08 0.99 0.01 Micro avg ····················· 0.99 0.01 D.3. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM PAGES243
Duration Results of configuration Shattered Document with Random Pages
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 8.08 0.86 0.98 0.90 0.91 0.87 0.05 0.95 1.05 Oracle 2 5.09 0.79 0.97 0.84 0.81 0.79 0.02 0.97 1.03 Oracle 3 7.91 0.80 1.45 0.99 1.03 0.82 0.21 0.80 1.25 Oracle 4 6.77 0.80 1.33 0.97 1.33 0.96 0.37 0.73 1.38 Oracle 5 6.64 0.71 0.95 0.83 0.71 0.84 -0.13 1.18 0.85 Oracle 6 10.18 0.76 1.13 0.92 1.06 0.86 0.20 0.81 1.23 Oracle 7 10.03 0.80 1.06 0.91 1.06 0.86 0.20 0.80 1.24 Oracle 8 10.84 0.78 1.46 0.98 1.46 0.84 0.62 0.58 1.73 Oracle 9 6.65 0.81 1.16 0.95 1.16 0.87 0.29 0.75 1.33 Oracle 10 19.91 0.86 1.24 0.99 1.24 0.87 0.38 0.70 1.44 Oracle 11 11.25 0.73 1.34 0.94 1.05 0.92 0.13 0.88 1.14 Oracle 12 12.45 0.87 1.30 0.96 1.05 0.97 0.09 0.92 1.09 Oracle 13 14.66 0.77 1.39 0.98 0.98 0.85 0.14 0.86 1.16 Oracle 14 10.77 0.75 1.39 0.98 1.05 0.90 0.15 0.86 1.16 Oracle 15 7.70 0.79 1.26 0.96 1.26 0.90 0.36 0.72 1.40 Oracle 16 5.77 0.88 1.16 0.96 1.16 0.97 0.19 0.84 1.20 Oracle 17 8.70 0.85 1.36 0.97 1.36 0.89 0.47 0.66 1.53 Oracle 18 10.51 0.77 1.29 0.95 1.02 0.77 0.24 0.77 1.30 Oracle 19 5.95 0.55 1.42 0.99 1.42 0.55 0.87 0.39 2.59 Oracle 20 5.71 0.85 1.05 0.95 1.05 0.90 0.16 0.85 1.18 Oracle 21 9.59 0.80 1.09 0.96 0.98 0.91 0.05 0.94 1.06 Oracle 22 8.64 0.30 1.11 0.87 1.11 0.88 0.23 0.79 1.27 Oracle 23 13.58 0.73 1.36 0.91 0.95 0.81 0.14 0.85 1.17 Oracle 24 9.71 0.80 1.02 0.88 0.97 0.85 0.12 0.88 1.13 Oracle 25 8.65 0.75 1.48 0.96 0.84 0.91 -0.06 1.07 0.93 Oracle 26 13.28 0.73 1.00 0.88 1.00 0.90 0.10 0.90 1.11 Oracle 27 12.41 0.75 1.31 0.95 1.02 0.80 0.22 0.79 1.27 Oracle 28 7.58 0.90 1.41 1.09 1.41 1.02 0.41 0.72 1.40 Oracle 29 5.13 0.81 1.36 1.02 1.36 0.81 0.55 0.60 1.67 Oracle 30 11.15 0.75 0.95 0.86 0.91 0.80 0.12 0.88 1.14 Oracle 31 10.82 0.81 1.30 0.98 0.81 0.88 -0.06 1.08 0.93 Oracle 32 12.04 0.70 1.01 0.86 0.88 0.86 0.02 0.98 1.02 Oracle 33 11.50 0.86 1.07 0.96 1.05 0.91 0.15 0.86 1.16 Oracle 34 7.37 0.84 0.99 0.92 0.86 0.84 0.03 0.97 1.03 Oracle 35 8.81 0.47 1.29 0.88 0.91 0.91 0.00 1.00 1.00 Oracle 36 11.62 0.40 1.05 0.89 0.83 0.92 -0.09 1.12 0.90 Oracle 37 10.22 0.70 1.19 0.93 1.17 0.90 0.27 0.77 1.30 Oracle 38 11.20 0.80 1.31 1.02 1.17 0.89 0.28 0.76 1.32 Oracle 39 6.77 0.70 1.30 0.97 1.01 0.82 0.19 0.81 1.23 Oracle 40 14.03 0.70 1.59 1.00 1.15 1.20 -0.05 1.05 0.95 Oracle 41 7.14 0.31 1.13 0.89 1.13 0.98 0.14 0.88 1.14 Oracle 42 0.97 0.97 0.97 0.97 0.97 0.97 0.00 1.00 1.00 Oracle 43 10.20 0.85 1.51 1.02 1.51 0.85 0.65 0.57 1.77 Oracle 44 5.70 0.89 1.06 0.95 0.89 0.91 -0.02 1.02 0.98 Oracle 45 8.70 0.85 1.12 0.97 0.97 0.85 0.12 0.88 1.14 Oracle 46 9.62 0.85 1.27 0.96 1.27 0.91 0.35 0.72 1.39 Oracle 47 8.61 0.30 1.27 0.86 0.94 0.90 0.04 0.96 1.05 Oracle 48 11.78 0.76 1.03 0.91 0.99 0.90 0.09 0.91 1.11 Oracle 49 8.89 0.70 1.07 0.89 1.06 0.80 0.26 0.76 1.33 Oracle 50 7.75 0.71 0.99 0.86 0.93 0.86 0.06 0.93 1.08 Total 469.02 37.32 60.30 47.01 53.23 43.92 9.31 42.38 61.28 Average 9.38 0.75 1.20 0.94 1.06 0.88 0.19 0.84 1.23 Micro avg ····················· 0.83 1.21 244 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Random Pages
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 0.91 0.91 0.90 0.91 0.86 0.88 0.87 0.98 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 Oracle 2 0.81 0.97 0.84 0.87 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 Oracle 3 1.03 1.45 1.03 0.97 0.82 0.98 0.80 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 Oracle 4 1.33 0.83 0.80 0.98 1.00 0.88 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 Oracle 5 0.71 0.93 0.88 0.75 0.80 0.80 0.95 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 Oracle 6 1.06 0.92 0.76 1.13 1.02 0.91 0.80 0.98 0.88 0.87 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 Oracle 7 1.06 0.99 0.95 0.84 0.88 0.91 0.88 0.91 0.95 0.80 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 Oracle 8 1.46 1.16 0.78 1.02 0.92 0.95 0.96 0.80 0.91 1.05 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 Oracle 9 1.16 1.05 0.92 0.98 0.86 0.81 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 Oracle 10 1.24 1.16 1.14 1.09 1.20 0.91 0.95 0.88 1.11 0.91 1.02 0.91 1.12 0.88 0.86 0.91 0.93 0.93 0.91 0.87 Oracle 11 1.05 0.99 0.89 0.91 0.73 0.97 1.34 0.81 0.86 0.91 0.87 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 Oracle 12 1.05 1.30 0.87 0.91 0.91 0.87 0.96 0.97 0.95 0.88 0.91 0.91 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 Oracle 13 0.98 1.23 1.39 1.38 0.93 0.92 0.85 0.98 0.77 0.88 0.88 0.84 0.86 0.92 0.85 0.85 0.85 0.85 0.85 0.85 Oracle 14 1.05 1.00 1.39 0.90 0.96 1.00 0.99 0.92 0.91 0.75 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 15 1.26 1.24 0.80 1.02 0.79 0.84 0.84 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 16 1.16 0.92 0.94 0.88 0.91 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 Oracle 17 1.36 0.91 0.90 1.09 0.89 0.85 0.90 0.91 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 Oracle 18 1.02 1.29 1.01 0.91 0.90 0.96 0.88 0.91 0.91 0.95 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 Oracle 19 1.42 0.96 1.07 1.10 0.85 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 20 1.05 0.97 0.85 0.97 0.96 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 21 0.98 1.09 1.08 1.03 0.99 0.94 0.91 0.80 0.86 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 22 1.11 1.07 1.03 0.95 0.30 0.91 0.88 0.79 0.74 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 Oracle 23 0.95 1.36 0.73 1.02 0.82 0.89 0.91 1.02 0.85 0.92 0.77 0.90 0.81 0.84 0.81 0.81 0.81 0.81 0.81 0.81 Oracle 24 0.97 0.98 0.80 1.02 0.80 0.85 0.88 0.90 0.85 0.80 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 Oracle 25 0.84 1.02 1.48 0.95 0.85 0.90 0.75 0.95 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 26 1.00 0.88 0.98 0.97 0.97 0.87 0.95 0.84 0.86 0.73 0.78 0.90 0.90 0.76 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 27 1.02 1.13 1.31 0.95 1.00 0.96 0.86 0.98 0.75 0.89 0.89 0.85 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 Oracle 28 1.41 1.22 1.04 0.96 1.02 0.90 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 29 1.36 1.08 0.98 0.91 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 Oracle 30 0.91 0.84 0.95 0.91 0.90 0.87 0.75 0.95 0.81 0.84 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 Oracle 31 0.81 1.15 0.97 0.98 1.08 1.30 0.92 0.92 0.95 0.84 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 Oracle 32 0.88 0.85 1.01 0.86 0.80 0.91 0.91 0.75 0.87 0.91 0.83 0.70 0.90 0.86 0.86 0.86 0.86 0.86 0.86 0.86 Oracle 33 1.05 0.92 1.07 1.04 0.95 0.89 0.95 0.92 0.86 0.96 0.98 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 34 0.86 0.99 0.95 0.87 0.95 0.96 0.94 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 Oracle 35 0.91 1.29 0.47 0.80 0.95 0.88 0.91 0.89 0.80 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 36 0.83 0.98 0.99 0.89 1.01 0.92 0.40 0.88 1.05 0.95 0.95 0.84 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.92 Oracle 37 1.17 0.96 0.86 0.85 0.93 1.19 0.70 0.81 0.96 0.88 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 38 1.17 1.30 1.31 1.04 0.80 0.86 0.95 1.19 0.80 0.90 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 Oracle 39 1.01 0.98 1.30 0.96 0.70 1.00 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 0.82 Oracle 40 1.15 1.59 0.98 0.95 1.05 0.95 0.95 0.91 0.95 0.81 0.87 0.70 0.95 1.20 1.20 1.20 1.20 1.20 1.20 1.20 Oracle 41 1.13 1.09 1.05 0.92 0.31 0.83 0.84 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 Oracle 42 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 Oracle 43 1.51 1.10 1.02 0.92 1.05 0.86 0.96 0.98 0.92 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 Oracle 44 0.89 1.06 0.91 1.00 0.94 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 45 0.97 1.12 1.01 1.06 0.96 0.94 0.91 0.87 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 Oracle 46 1.27 0.91 0.90 0.85 0.88 1.05 0.95 0.95 0.93 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 47 0.94 1.27 0.90 0.81 0.94 0.86 0.30 0.85 0.85 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 48 0.99 1.03 0.88 0.91 0.96 1.03 0.90 0.82 0.88 0.76 0.89 0.86 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 49 1.06 0.83 1.07 0.96 0.86 0.85 0.84 0.92 0.70 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 Oracle 50 0.93 0.73 0.71 0.99 0.91 0.91 0.80 0.89 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 Total 53 53 49 48 44 46 44 45 44 44 44 43 44 44 44 44 44 44 44 44 Average 1.06 1.06 0.98 0.96 0.89 0.91 0.88 0.90 0.88 0.88 0.88 0.87 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 D.3. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM PAGES245
Size Results of configuration Shattered Document with Ran- dom Pages
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 2 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 3 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 4 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 5 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 6 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 7 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 8 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 9 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 10 140 7 7 7.00 7 7 0 1.00 1.00 Oracle 11 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 12 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 13 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 14 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 15 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 16 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 17 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 18 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 19 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 20 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 21 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 22 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 23 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 24 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 27 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 28 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 29 35 7 7 7.00 7 7 0 1.00 1.00 Oracle 30 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 31 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 32 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 33 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 34 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 35 66 3 7 6.60 7 7 0 1.00 1.00 Oracle 36 87 3 7 6.70 7 7 0 1.00 1.00 Oracle 37 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 38 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 39 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 40 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 41 51 2 7 6.38 7 7 0 1.00 1.00 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 44 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 45 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 46 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 47 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 48 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 49 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 3487 337 350 348.66 350 350 0 50.00 50.00 Average 69.74 6.74 7.00 6.98 7.00 7.00 0.00 1.00 1.00 Micro avg ····················· 1.00 1.00 246 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Ran- dom Pages
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 2 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 3 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 4 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 5 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 6 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 8 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 9 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 10 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 11 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 12 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 13 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 14 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 15 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 16 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 17 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 18 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 19 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 20 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 21 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 22 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 23 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 24 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 27 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 28 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 29 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 30 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 31 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 32 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 33 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 34 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 35 7 7 3 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 36 7 7 7 7 7 7 3 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 37 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 38 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 39 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 40 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 41 7 7 7 7 2 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 44 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 45 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 46 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 47 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 48 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 49 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 350 350 346 350 345 350 346 350 350 350 350 350 350 350 350 350 350 350 350 350 Average 7.00 7.00 6.92 7.00 6.90 7.00 6.92 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 D.3. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM PAGES247
Cost Results of configuration Shattered Document with Ran- dom Pages
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 2 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 3 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 4 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 5 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 6 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 7 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 8 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 9 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 10 140.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 11 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 12 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 13 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 14 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 15 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 16 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 17 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 18 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 19 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 20 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 21 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 22 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 23 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 24 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 27 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 28 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 29 35.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 30 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 31 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 32 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 33 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 34 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 35 66.00 3.00 7.00 6.60 7.00 7.00 0.00 1.00 1.00 Oracle 36 87.00 3.00 7.00 6.70 7.00 7.00 0.00 1.00 1.00 Oracle 37 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 38 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 39 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 40 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 41 51.00 2.00 7.00 6.38 7.00 7.00 0.00 1.00 1.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 44 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 45 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 46 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 47 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 48 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 49 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 3487.00 337.00 350.00 348.66 350.00 350.00 0.00 50.00 50.00 Average 69.74 6.74 7.00 6.98 7.00 7.00 0.00 1.00 1.00 Micro avg ····················· 1.00 1.00 248 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Ran- dom Pages
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 2 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 3 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 4 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 5 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 6 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 7 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 8 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 9 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 10 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 11 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 12 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 13 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 14 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 15 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 16 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 17 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 18 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 19 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 20 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 21 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 22 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 23 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 24 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 27 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 28 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 29 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 30 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 31 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 32 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 33 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 34 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 35 7.00 7.00 3.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 36 7.00 7.00 7.00 7.00 7.00 7.00 3.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 37 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 38 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 39 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 40 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 41 7.00 7.00 7.00 7.00 2.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 44 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 45 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 46 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 47 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 48 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 49 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 350 350 346 350 345 350 346 350 350 350 350 350 350 350 350 350 350 350 350 350 Average 7.00 7.00 6.92 7.00 6.90 7.00 6.92 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 D.4. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM LEARNING249 D.4 Complete results of configuration Shattered Document with Random Learning
Shattered Document format with random learning i.e. the travel matrix is updated randomly. Using functions Super Page, Choose, Recentre from Kasim2.Activation.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Randomly Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.65
Size0 = 5.78
Size1 = 5.34 Size gain = 1.09
Cost0 = 5.78
Cost1 = 5.34 Cost gain = 1.09 250 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Random Learning
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 8 0 8 1 0.89 0.11 Oracle 2 30 6 0 5 0 5 1 0.84 0.16 Oracle 3 40 8 0 4 0 4 4 0.50 0.50 Oracle 4 46 7 0 5 0 5 2 0.71 0.29 Oracle 5 56 8 0 8 0 8 0 1.00 0.00 Oracle 6 85 11 0 7 0 7 4 0.63 0.37 Oracle 7 108 11 0 2 0 2 9 0.18 0.82 Oracle 8 125 11 0 7 0 7 4 0.63 0.37 Oracle 9 128 7 0 3 0 3 4 0.43 0.57 Oracle 10 130 20 0 6 0 6 14 0.30 0.70 Oracle 11 138 12 0 3 0 3 9 0.25 0.75 Oracle 12 157 13 0 1 0 1 12 0.08 0.92 Oracle 13 171 15 0 2 0 2 13 0.13 0.87 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 3 0 3 5 0.38 0.63 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 0 0 0 9 0.00 1.00 Oracle 18 269 11 0 0 0 0 11 0.00 1.00 Oracle 19 290 6 0 0 0 0 6 0.00 1.00 Oracle 20 307 6 0 1 0 1 5 0.16 0.84 Oracle 21 366 10 0 4 0 4 6 0.40 0.60 Oracle 22 410 10 0 0 0 0 10 0.00 1.00 Oracle 23 455 15 0 7 0 7 8 0.47 0.53 Oracle 24 508 11 0 2 0 2 9 0.18 0.82 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 0 0 0 15 0.00 1.00 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 4 0 4 3 0.57 0.43 Oracle 29 542 5 0 5 0 5 0 1.00 0.00 Oracle 30 554 13 0 8 0 8 5 0.62 0.38 Oracle 31 601 11 0 10 0 10 1 0.91 0.09 Oracle 32 606 14 0 10 0 10 4 0.71 0.29 Oracle 33 616 12 0 5 0 5 7 0.41 0.59 Oracle 34 649 8 0 2 0 2 6 0.25 0.75 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 1 0 1 10 0.09 0.91 Oracle 38 741 11 0 0 0 0 11 0.00 1.00 Oracle 39 786 7 0 6 0 6 1 0.86 0.14 Oracle 40 840 14 0 12 0 12 2 0.86 0.14 Oracle 41 846 8 0 7 0 7 1 0.88 0.13 Oracle 42 883 1 0 1 0 1 0 1.00 0.00 Oracle 43 890 10 0 1 0 1 9 0.10 0.90 Oracle 44 897 6 0 1 0 1 5 0.16 0.84 Oracle 45 899 9 0 1 0 1 8 0.11 0.89 Oracle 46 929 10 0 2 0 2 8 0.20 0.80 Oracle 47 936 10 0 1 0 1 9 0.10 0.90 Oracle 48 939 13 0 1 0 1 12 0.08 0.92 Oracle 49 944 10 0 2 0 2 8 0.20 0.80 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 176 0 176 324 19.28 30.72 Average ··· 10.00 0.00 3.52 0.00 3.52 6.48 0.38 0.62 Micro avg ····················· 0.35 0.65 D.4. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM LEARNING251
Duration Results of configuration Shattered Document with Random Learning
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 48.59 1.38 6.54 5.40 1.38 5.37 -3.98 3.89 0.26 Oracle 2 31.20 1.78 6.40 5.20 1.78 5.38 -3.59 3.01 0.34 Oracle 3 29.86 1.86 5.97 3.73 1.95 5.11 -3.16 2.63 0.38 Oracle 4 32.55 1.80 6.16 4.65 1.80 5.63 -3.83 3.12 0.32 Oracle 5 43.48 5.01 5.57 5.44 5.55 5.43 0.12 0.98 1.02 Oracle 6 43.32 1.74 5.49 3.94 1.74 2.07 -0.33 1.19 0.84 Oracle 7 27.13 1.02 5.80 2.47 1.88 1.85 0.02 0.98 1.02 Oracle 8 43.70 1.34 5.50 3.97 1.45 1.43 0.02 0.98 1.02 Oracle 9 23.05 2.26 5.42 3.30 2.60 2.31 0.30 0.89 1.13 Oracle 10 51.89 1.30 6.02 2.59 1.31 1.30 0.02 0.98 1.02 Oracle 11 27.11 0.98 5.41 2.26 0.98 1.02 -0.04 1.04 0.96 Oracle 12 20.31 1.08 4.94 1.56 1.39 1.35 0.04 0.97 1.03 Oracle 13 24.55 0.95 2.97 1.63 1.48 0.98 0.49 0.67 1.49 Oracle 14 8.70 0.66 1.80 0.79 1.80 0.95 0.84 0.53 1.88 Oracle 15 18.26 1.36 3.27 2.28 1.36 1.77 -0.41 1.30 0.77 Oracle 16 7.79 1.04 1.37 1.30 1.34 1.36 -0.02 1.01 0.99 Oracle 17 15.55 1.45 1.84 1.73 1.84 1.75 0.09 0.95 1.05 Oracle 18 14.96 1.30 1.71 1.36 1.30 1.35 -0.05 1.05 0.96 Oracle 19 11.27 1.41 2.59 1.88 1.41 1.46 -0.05 1.03 0.97 Oracle 20 15.59 2.23 2.73 2.60 2.23 2.70 -0.47 1.21 0.83 Oracle 21 27.70 2.69 2.85 2.77 2.72 2.80 -0.07 1.02 0.98 Oracle 22 24.66 2.30 2.69 2.47 2.59 2.31 0.28 0.89 1.13 Oracle 23 37.35 2.09 2.70 2.49 2.51 2.60 -0.09 1.04 0.96 Oracle 24 21.05 1.41 2.72 1.91 2.65 1.42 1.23 0.54 1.87 Oracle 25 23.84 2.62 2.73 2.65 2.70 2.73 -0.04 1.02 0.98 Oracle 26 33.36 1.80 2.76 2.23 1.80 2.23 -0.42 1.23 0.81 Oracle 27 23.63 0.95 2.81 1.82 0.95 2.65 -1.70 2.77 0.36 Oracle 28 13.09 0.52 2.66 1.87 2.66 1.02 1.64 0.38 2.63 Oracle 29 13.58 2.70 2.77 2.72 2.77 2.70 0.06 0.98 1.02 Oracle 30 26.40 1.02 2.71 2.03 2.71 1.05 1.66 0.38 2.59 Oracle 31 29.28 2.56 2.71 2.66 2.68 2.69 -0.01 1.00 1.00 Oracle 32 33.67 1.45 2.72 2.41 2.68 2.69 -0.01 1.01 0.99 Oracle 33 31.89 2.59 2.73 2.66 2.72 2.73 -0.01 1.00 1.00 Oracle 34 13.54 0.94 2.70 1.70 1.76 0.94 0.82 0.53 1.88 Oracle 35 14.95 1.38 2.30 1.49 2.30 1.44 0.86 0.63 1.60 Oracle 36 13.26 0.95 1.81 1.02 1.81 0.98 0.84 0.54 1.85 Oracle 37 22.94 0.52 2.66 2.09 2.66 2.17 0.49 0.81 1.23 Oracle 38 20.47 1.31 2.24 1.86 2.20 1.71 0.49 0.78 1.29 Oracle 39 19.05 2.69 2.79 2.72 2.69 2.79 -0.10 1.04 0.96 Oracle 40 37.91 2.65 2.80 2.71 2.65 2.78 -0.13 1.05 0.95 Oracle 41 21.53 2.66 2.72 2.69 2.72 2.69 0.03 0.99 1.01 Oracle 42 2.70 2.70 2.70 2.70 2.70 2.70 0.00 1.00 1.00 Oracle 43 12.05 1.01 2.73 1.20 2.73 1.03 1.71 0.38 2.66 Oracle 44 9.76 1.40 2.69 1.63 2.69 1.42 1.27 0.53 1.89 Oracle 45 10.53 0.97 2.73 1.17 2.73 1.00 1.73 0.37 2.73 Oracle 46 23.63 1.80 2.75 2.36 2.74 2.75 -0.01 1.00 1.00 Oracle 47 21.77 1.78 2.66 2.18 2.27 1.82 0.45 0.80 1.25 Oracle 48 28.41 1.35 2.80 2.19 2.26 2.25 0.01 1.00 1.00 Oracle 49 18.53 1.38 2.76 1.85 2.26 1.41 0.85 0.63 1.61 Oracle 50 24.60 2.69 2.80 2.73 2.77 2.72 0.05 0.98 1.02 Total 1194.01 84.80 167.70 123.04 110.63 112.74 -2.12 54.73 59.49 Average 23.88 1.70 3.35 2.46 2.21 2.26 -0.04 1.09 1.19 Micro avg ····················· 1.02 0.98 252 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Random Learning
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 1.38 6.20 6.09 6.54 6.38 5.73 5.33 5.59 5.37 5.37 5.37 5.37 5.37 5.37 5.37 5.37 5.37 5.37 5.37 5.37 Oracle 2 1.78 6.40 6.13 5.77 5.73 5.38 5.38 5.38 5.38 5.38 5.38 5.38 5.38 5.38 5.38 5.38 5.38 5.38 5.38 5.38 Oracle 3 1.95 1.93 2.00 1.86 5.97 5.55 5.50 5.11 5.11 5.11 5.11 5.11 5.11 5.11 5.11 5.11 5.11 5.11 5.11 5.11 Oracle 4 1.80 1.81 5.97 6.16 5.61 5.56 5.63 5.63 5.63 5.63 5.63 5.63 5.63 5.63 5.63 5.63 5.63 5.63 5.63 5.63 Oracle 5 5.55 5.57 5.55 5.31 5.56 5.51 5.01 5.43 5.43 5.43 5.43 5.43 5.43 5.43 5.43 5.43 5.43 5.43 5.43 5.43 Oracle 6 1.74 1.76 5.17 5.29 5.05 5.49 4.90 4.95 4.90 2.01 2.07 2.07 2.07 2.07 2.07 2.07 2.07 2.07 2.07 2.07 Oracle 7 1.88 1.02 5.41 5.80 1.84 1.88 1.93 1.83 1.87 1.83 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 Oracle 8 1.45 1.34 1.52 5.50 5.50 5.48 5.45 5.43 5.45 5.14 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 1.43 Oracle 9 2.60 2.60 2.72 5.42 5.13 2.26 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 Oracle 10 1.31 1.34 1.38 1.30 1.32 1.33 1.34 1.31 1.72 6.02 1.88 2.02 1.73 5.42 2.51 5.19 5.15 5.16 3.17 1.30 Oracle 11 0.98 1.42 1.41 5.41 2.91 5.16 1.39 1.40 1.45 1.39 3.18 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 12 1.39 1.34 1.36 1.78 1.10 1.09 1.09 1.08 1.09 4.94 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 Oracle 13 1.48 1.50 1.50 1.46 1.46 1.50 1.50 1.91 1.95 2.95 2.97 1.46 0.95 0.97 0.98 0.98 0.98 0.98 0.98 0.98 Oracle 14 1.80 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 15 1.36 1.84 3.27 3.14 1.77 3.19 1.92 1.77 1.77 1.77 1.77 1.77 1.77 1.77 1.77 1.77 1.77 1.77 1.77 1.77 Oracle 16 1.34 1.04 1.34 1.34 1.37 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 Oracle 17 1.84 1.76 1.74 1.77 1.74 1.73 1.45 1.79 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 Oracle 18 1.30 1.34 1.32 1.31 1.71 1.31 1.31 1.34 1.35 1.31 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 Oracle 19 1.41 2.16 2.16 2.59 1.49 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 20 2.23 2.63 2.73 2.63 2.67 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 21 2.72 2.79 2.77 2.69 2.77 2.70 2.77 2.85 2.84 2.80 2.80 2.80 2.80 2.80 2.80 2.80 2.80 2.80 2.80 2.80 Oracle 22 2.59 2.60 2.59 2.69 2.34 2.62 2.30 2.30 2.32 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 Oracle 23 2.51 2.49 2.69 2.66 2.70 2.66 2.67 2.66 2.68 2.09 2.59 2.11 2.09 2.13 2.60 2.60 2.60 2.60 2.60 2.60 Oracle 24 2.65 1.43 1.42 1.41 1.41 1.85 2.66 2.66 2.72 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 Oracle 25 2.70 2.63 2.62 2.63 2.63 2.63 2.65 2.64 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 Oracle 26 1.80 1.84 1.84 1.81 2.23 2.19 2.18 2.19 2.21 2.76 2.62 2.61 2.66 2.19 2.23 2.23 2.23 2.23 2.23 2.23 Oracle 27 0.95 0.95 1.38 0.95 0.97 2.81 2.64 2.63 0.95 1.38 2.67 2.69 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Oracle 28 2.66 2.66 2.63 2.62 0.52 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 29 2.77 2.70 2.73 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 30 2.71 2.65 2.65 2.63 2.63 2.64 2.64 1.02 1.02 1.02 2.68 1.08 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 31 2.68 2.67 2.65 2.56 2.67 2.67 2.67 2.65 2.64 2.71 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 32 2.68 2.67 2.67 2.72 2.66 2.66 2.66 1.45 1.45 1.51 2.54 2.66 2.66 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 33 2.72 2.63 2.63 2.63 2.62 2.61 2.59 2.72 2.71 2.60 2.72 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 Oracle 34 1.76 1.76 1.75 2.70 2.70 0.95 0.98 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 Oracle 35 2.30 1.39 1.45 1.41 1.38 1.38 1.38 1.42 1.41 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 Oracle 36 1.81 0.96 0.95 0.97 0.96 0.95 0.95 0.95 0.95 0.95 0.96 0.95 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 Oracle 37 2.66 2.55 0.52 2.14 2.17 2.16 2.14 2.14 2.15 2.14 2.17 2.17 2.17 2.17 2.17 2.17 2.17 2.17 2.17 2.17 Oracle 38 2.20 2.24 2.20 2.12 1.31 1.78 1.72 1.71 1.72 1.76 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 Oracle 39 2.69 2.77 2.70 2.69 2.72 2.70 2.79 2.79 2.79 2.79 2.79 2.79 2.79 2.79 2.79 2.79 2.79 2.79 2.79 2.79 Oracle 40 2.65 2.67 2.80 2.69 2.69 2.68 2.74 2.67 2.67 2.72 2.70 2.70 2.73 2.78 2.78 2.78 2.78 2.78 2.78 2.78 Oracle 41 2.72 2.66 2.70 2.70 2.69 2.70 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 42 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 43 2.73 1.02 1.01 1.02 1.19 1.02 1.02 1.04 1.01 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 44 2.69 1.40 1.40 1.40 1.45 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 Oracle 45 2.73 0.97 0.98 0.97 0.97 0.97 0.97 0.98 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 Oracle 46 2.74 2.75 2.68 2.26 1.86 1.83 2.30 1.80 2.67 2.75 2.75 2.75 2.75 2.75 2.75 2.75 2.75 2.75 2.75 2.75 Oracle 47 2.27 2.27 1.78 2.27 1.79 2.66 2.25 2.24 2.40 1.82 1.82 1.82 1.82 1.82 1.82 1.82 1.82 1.82 1.82 1.82 Oracle 48 2.26 2.22 2.66 2.80 1.35 1.36 2.24 2.25 2.24 2.25 2.26 2.26 2.25 2.25 2.25 2.25 2.25 2.25 2.25 2.25 Oracle 49 2.26 2.28 2.76 2.66 1.38 1.45 1.43 1.44 1.48 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 Oracle 50 2.77 2.69 2.73 2.72 2.73 2.70 2.73 2.80 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 Total 111 112 124 138 126 127 122 120 120 124 119 114 113 116 114 117 117 117 115 113 Average 2.21 2.23 2.49 2.76 2.52 2.55 2.45 2.40 2.39 2.48 2.38 2.27 2.26 2.33 2.28 2.34 2.33 2.34 2.29 2.26 D.4. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM LEARNING253
Size Results of configuration Shattered Document with Ran- dom Learning
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 60 4 7 6.66 4 7 -3 1.75 0.57 Oracle 2 40 5 7 6.66 5 7 -2 1.40 0.71 Oracle 3 48 5 7 6.00 5 7 -2 1.40 0.71 Oracle 4 45 5 7 6.43 5 7 -2 1.40 0.71 Oracle 5 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 6 69 5 7 6.27 5 5 0 1.00 1.00 Oracle 7 57 3 7 5.18 5 5 0 1.00 1.00 Oracle 8 65 4 7 5.91 4 4 0 1.00 1.00 Oracle 9 47 6 7 6.71 7 6 1 0.86 1.16 Oracle 10 104 4 7 5.20 4 4 0 1.00 1.00 Oracle 11 58 3 7 4.84 3 3 0 1.00 1.00 Oracle 12 51 3 7 3.92 4 4 0 1.00 1.00 Oracle 13 65 3 7 4.34 4 3 1 0.75 1.34 Oracle 14 26 2 5 2.37 5 3 2 0.60 1.66 Oracle 15 45 4 7 5.63 4 5 -1 1.25 0.80 Oracle 16 23 3 4 3.84 4 4 0 1.00 1.00 Oracle 17 44 4 5 4.89 5 5 0 1.00 1.00 Oracle 18 45 4 5 4.09 4 4 0 1.00 1.00 Oracle 19 31 4 7 5.16 4 4 0 1.00 1.00 Oracle 20 41 6 7 6.84 6 7 -1 1.16 0.86 Oracle 21 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 22 65 6 7 6.50 7 6 1 0.86 1.16 Oracle 23 101 6 7 6.73 7 7 0 1.00 1.00 Oracle 24 57 4 7 5.18 7 4 3 0.57 1.75 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 90 5 7 6.00 5 6 -1 1.20 0.84 Oracle 27 65 3 7 5.00 3 7 -4 2.34 0.43 Oracle 28 36 2 7 5.14 7 3 4 0.43 2.34 Oracle 29 35 7 7 7.00 7 7 0 1.00 1.00 Oracle 30 71 3 7 5.46 7 3 4 0.43 2.34 Oracle 31 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 32 89 4 7 6.36 7 7 0 1.00 1.00 Oracle 33 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 34 38 3 7 4.75 5 3 2 0.60 1.66 Oracle 35 42 4 6 4.20 6 4 2 0.66 1.50 Oracle 36 41 3 5 3.16 5 3 2 0.60 1.66 Oracle 37 64 2 7 5.82 7 6 1 0.86 1.16 Oracle 38 58 4 6 5.27 6 5 1 0.84 1.20 Oracle 39 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 40 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 41 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 34 3 7 3.40 7 3 4 0.43 2.34 Oracle 44 27 4 7 4.50 7 4 3 0.57 1.75 Oracle 45 31 3 7 3.45 7 3 4 0.43 2.34 Oracle 46 62 5 7 6.20 7 7 0 1.00 1.00 Oracle 47 58 5 7 5.80 6 5 1 0.84 1.20 Oracle 48 76 4 7 5.84 6 6 0 1.00 1.00 Oracle 49 50 4 7 5.00 6 4 2 0.66 1.50 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 2777 231 337 281.70 289 267 22 47.88 58.70 Average 55.54 4.62 6.74 5.63 5.78 5.34 0.44 0.96 1.17 Micro avg ····················· 0.92 1.09 254 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Ran- dom Learning
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 4 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 2 5 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 3 5 5 5 5 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 4 5 5 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 5 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 6 5 5 7 7 7 7 7 7 7 5 5 5 5 5 5 5 5 5 5 5 Oracle 7 5 3 7 7 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 8 4 4 4 7 7 7 7 7 7 7 4 4 4 4 4 4 4 4 4 4 Oracle 9 7 7 7 7 7 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 Oracle 10 4 4 4 4 4 4 4 4 5 7 5 5 5 7 6 7 7 7 7 4 Oracle 11 3 4 4 7 7 7 4 4 4 4 7 3 3 3 3 3 3 3 3 3 Oracle 12 4 4 4 5 3 3 3 3 3 7 4 4 4 4 4 4 4 4 4 4 Oracle 13 4 4 4 4 4 4 4 5 5 7 7 4 3 3 3 3 3 3 3 3 Oracle 14 5 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 Oracle 15 4 5 7 7 5 7 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 16 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 17 5 5 5 5 5 5 4 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 18 4 4 4 4 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 19 4 6 6 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 20 6 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 21 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 22 7 7 7 7 6 7 6 6 6 6 6 6 6 6 6 6 6 6 6 6 Oracle 23 7 7 7 7 7 7 7 7 7 6 7 6 6 6 7 7 7 7 7 7 Oracle 24 7 4 4 4 4 5 7 7 7 4 4 4 4 4 4 4 4 4 4 4 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 5 5 5 5 6 6 6 6 6 7 7 7 7 6 6 6 6 6 6 6 Oracle 27 3 3 4 3 3 7 7 7 3 4 7 7 7 7 7 7 7 7 7 7 Oracle 28 7 7 7 7 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 29 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 30 7 7 7 7 7 7 7 3 3 3 7 3 3 3 3 3 3 3 3 3 Oracle 31 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 32 7 7 7 7 7 7 7 4 4 4 7 7 7 7 7 7 7 7 7 7 Oracle 33 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 34 5 5 5 7 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 35 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 36 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 37 7 7 2 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 Oracle 38 6 6 6 6 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 39 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 40 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 41 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 44 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 45 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 46 7 7 7 6 5 5 6 5 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 47 6 6 5 6 5 7 6 6 6 5 5 5 5 5 5 5 5 5 5 5 Oracle 48 6 6 7 7 4 4 6 6 6 6 6 6 6 6 6 6 6 6 6 6 Oracle 49 6 6 7 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 289 275 283 296 275 281 278 272 271 274 281 269 268 269 269 270 270 270 270 267 Average 5.78 5.50 5.66 5.92 5.50 5.62 5.56 5.44 5.42 5.48 5.62 5.38 5.36 5.38 5.38 5.40 5.40 5.40 5.40 5.34 D.4. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM LEARNING255
Cost Results of configuration Shattered Document with Ran- dom Learning
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 60.00 4.00 7.00 6.66 4.00 7.00 -3.00 1.75 0.57 Oracle 2 40.00 5.00 7.00 6.66 5.00 7.00 -2.00 1.40 0.71 Oracle 3 48.00 5.00 7.00 6.00 5.00 7.00 -2.00 1.40 0.71 Oracle 4 45.00 5.00 7.00 6.43 5.00 7.00 -2.00 1.40 0.71 Oracle 5 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 6 69.00 5.00 7.00 6.27 5.00 5.00 0.00 1.00 1.00 Oracle 7 57.00 3.00 7.00 5.18 5.00 5.00 0.00 1.00 1.00 Oracle 8 65.00 4.00 7.00 5.91 4.00 4.00 0.00 1.00 1.00 Oracle 9 47.00 6.00 7.00 6.71 7.00 6.00 1.00 0.86 1.16 Oracle 10 104.00 4.00 7.00 5.20 4.00 4.00 0.00 1.00 1.00 Oracle 11 58.00 3.00 7.00 4.84 3.00 3.00 0.00 1.00 1.00 Oracle 12 51.00 3.00 7.00 3.92 4.00 4.00 0.00 1.00 1.00 Oracle 13 65.00 3.00 7.00 4.34 4.00 3.00 1.00 0.75 1.34 Oracle 14 26.00 2.00 5.00 2.37 5.00 3.00 2.00 0.60 1.66 Oracle 15 45.00 4.00 7.00 5.63 4.00 5.00 -1.00 1.25 0.80 Oracle 16 23.00 3.00 4.00 3.84 4.00 4.00 0.00 1.00 1.00 Oracle 17 44.00 4.00 5.00 4.89 5.00 5.00 0.00 1.00 1.00 Oracle 18 45.00 4.00 5.00 4.09 4.00 4.00 0.00 1.00 1.00 Oracle 19 31.00 4.00 7.00 5.16 4.00 4.00 0.00 1.00 1.00 Oracle 20 41.00 6.00 7.00 6.84 6.00 7.00 -1.00 1.16 0.86 Oracle 21 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 22 65.00 6.00 7.00 6.50 7.00 6.00 1.00 0.86 1.16 Oracle 23 101.00 6.00 7.00 6.73 7.00 7.00 0.00 1.00 1.00 Oracle 24 57.00 4.00 7.00 5.18 7.00 4.00 3.00 0.57 1.75 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 90.00 5.00 7.00 6.00 5.00 6.00 -1.00 1.20 0.84 Oracle 27 65.00 3.00 7.00 5.00 3.00 7.00 -4.00 2.34 0.43 Oracle 28 36.00 2.00 7.00 5.14 7.00 3.00 4.00 0.43 2.34 Oracle 29 35.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 30 71.00 3.00 7.00 5.46 7.00 3.00 4.00 0.43 2.34 Oracle 31 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 32 89.00 4.00 7.00 6.36 7.00 7.00 0.00 1.00 1.00 Oracle 33 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 34 38.00 3.00 7.00 4.75 5.00 3.00 2.00 0.60 1.66 Oracle 35 42.00 4.00 6.00 4.20 6.00 4.00 2.00 0.66 1.50 Oracle 36 41.00 3.00 5.00 3.16 5.00 3.00 2.00 0.60 1.66 Oracle 37 64.00 2.00 7.00 5.82 7.00 6.00 1.00 0.86 1.16 Oracle 38 58.00 4.00 6.00 5.27 6.00 5.00 1.00 0.84 1.20 Oracle 39 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 40 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 41 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 34.00 3.00 7.00 3.40 7.00 3.00 4.00 0.43 2.34 Oracle 44 27.00 4.00 7.00 4.50 7.00 4.00 3.00 0.57 1.75 Oracle 45 31.00 3.00 7.00 3.45 7.00 3.00 4.00 0.43 2.34 Oracle 46 62.00 5.00 7.00 6.20 7.00 7.00 0.00 1.00 1.00 Oracle 47 58.00 5.00 7.00 5.80 6.00 5.00 1.00 0.84 1.20 Oracle 48 76.00 4.00 7.00 5.84 6.00 6.00 0.00 1.00 1.00 Oracle 49 50.00 4.00 7.00 5.00 6.00 4.00 2.00 0.66 1.50 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 2777.00 231.00 337.00 281.70 289.00 267.00 22.00 47.88 58.70 Average 55.54 4.62 6.74 5.63 5.78 5.34 0.44 0.96 1.17 Micro avg ····················· 0.92 1.09 256 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Ran- dom Learning
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 4.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 2 5.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 3 5.00 5.00 5.00 5.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 4 5.00 5.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 5 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 6 5.00 5.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 7 5.00 3.00 7.00 7.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 8 4.00 4.00 4.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 9 7.00 7.00 7.00 7.00 7.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 10 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 5.00 7.00 5.00 5.00 5.00 7.00 6.00 7.00 7.00 7.00 7.00 4.00 Oracle 11 3.00 4.00 4.00 7.00 7.00 7.00 4.00 4.00 4.00 4.00 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 12 4.00 4.00 4.00 5.00 3.00 3.00 3.00 3.00 3.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 13 4.00 4.00 4.00 4.00 4.00 4.00 4.00 5.00 5.00 7.00 7.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 14 5.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 15 4.00 5.00 7.00 7.00 5.00 7.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 16 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 17 5.00 5.00 5.00 5.00 5.00 5.00 4.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 18 4.00 4.00 4.00 4.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 19 4.00 6.00 6.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 20 6.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 21 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 22 7.00 7.00 7.00 7.00 6.00 7.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 23 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 6.00 7.00 6.00 6.00 6.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 24 7.00 4.00 4.00 4.00 4.00 5.00 7.00 7.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 5.00 5.00 5.00 5.00 6.00 6.00 6.00 6.00 6.00 7.00 7.00 7.00 7.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 27 3.00 3.00 4.00 3.00 3.00 7.00 7.00 7.00 3.00 4.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 28 7.00 7.00 7.00 7.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 29 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 30 7.00 7.00 7.00 7.00 7.00 7.00 7.00 3.00 3.00 3.00 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 31 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 32 7.00 7.00 7.00 7.00 7.00 7.00 7.00 4.00 4.00 4.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 33 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 34 5.00 5.00 5.00 7.00 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 35 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 36 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 37 7.00 7.00 2.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 38 6.00 6.00 6.00 6.00 4.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 39 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 40 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 41 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 44 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 45 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 46 7.00 7.00 7.00 6.00 5.00 5.00 6.00 5.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 47 6.00 6.00 5.00 6.00 5.00 7.00 6.00 6.00 6.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 48 6.00 6.00 7.00 7.00 4.00 4.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 49 6.00 6.00 7.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 289 275 283 296 275 281 278 272 271 274 281 269 268 269 269 270 270 270 270 267 Average 5.78 5.50 5.66 5.92 5.50 5.62 5.56 5.44 5.42 5.48 5.62 5.38 5.36 5.38 5.38 5.40 5.40 5.40 5.40 5.34 D.5. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM PAGES AND RANDOM LEARNING257 D.5 Complete results of configuration Shattered Document with Random Pages and Random Learning
Shattered Document format with both random pages and random learning. Using functions Super Page, Choose, Random Page from Kasim2.Activation.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Randomly Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.01
Size0 = 7.00
Size1 = 7.00 Size gain = 1.00
Cost0 = 7.00
Cost1 = 7.00 Cost gain = 1.00 258 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Random Pages and Random Learning
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 9 0 9 0 1.00 0.00 Oracle 2 30 6 0 6 0 6 0 1.00 0.00 Oracle 3 40 8 0 8 0 8 0 1.00 0.00 Oracle 4 46 7 0 7 0 7 0 1.00 0.00 Oracle 5 56 8 0 8 0 8 0 1.00 0.00 Oracle 6 85 11 0 11 0 11 0 1.00 0.00 Oracle 7 108 11 0 11 0 11 0 1.00 0.00 Oracle 8 125 11 0 11 0 11 0 1.00 0.00 Oracle 9 128 7 0 5 0 5 2 0.71 0.29 Oracle 10 130 20 0 20 0 20 0 1.00 0.00 Oracle 11 138 12 0 12 0 12 0 1.00 0.00 Oracle 12 157 13 0 13 0 13 0 1.00 0.00 Oracle 13 171 15 0 15 0 15 0 1.00 0.00 Oracle 14 173 11 0 11 0 11 0 1.00 0.00 Oracle 15 241 8 0 8 0 8 0 1.00 0.00 Oracle 16 265 6 0 6 0 6 0 1.00 0.00 Oracle 17 266 9 0 9 0 9 0 1.00 0.00 Oracle 18 269 11 0 11 0 11 0 1.00 0.00 Oracle 19 290 6 0 6 0 6 0 1.00 0.00 Oracle 20 307 6 0 6 0 6 0 1.00 0.00 Oracle 21 366 10 0 10 0 10 0 1.00 0.00 Oracle 22 410 10 0 10 0 10 0 1.00 0.00 Oracle 23 455 15 0 15 0 15 0 1.00 0.00 Oracle 24 508 11 0 11 0 11 0 1.00 0.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 15 0 15 0 1.00 0.00 Oracle 27 516 13 0 13 0 13 0 1.00 0.00 Oracle 28 521 7 0 7 0 7 0 1.00 0.00 Oracle 29 542 5 0 5 0 5 0 1.00 0.00 Oracle 30 554 13 0 13 0 13 0 1.00 0.00 Oracle 31 601 11 0 11 0 11 0 1.00 0.00 Oracle 32 606 14 0 14 0 14 0 1.00 0.00 Oracle 33 616 12 0 12 0 12 0 1.00 0.00 Oracle 34 649 8 0 8 0 8 0 1.00 0.00 Oracle 35 723 10 0 10 0 10 0 1.00 0.00 Oracle 36 725 13 0 13 0 13 0 1.00 0.00 Oracle 37 738 11 0 11 0 11 0 1.00 0.00 Oracle 38 741 11 0 11 0 11 0 1.00 0.00 Oracle 39 786 7 0 7 0 7 0 1.00 0.00 Oracle 40 840 14 0 14 0 14 0 1.00 0.00 Oracle 41 846 8 0 8 0 8 0 1.00 0.00 Oracle 42 883 1 0 1 0 1 0 1.00 0.00 Oracle 43 890 10 0 10 0 10 0 1.00 0.00 Oracle 44 897 6 0 6 0 6 0 1.00 0.00 Oracle 45 899 9 0 9 0 9 0 1.00 0.00 Oracle 46 929 10 0 10 0 10 0 1.00 0.00 Oracle 47 936 10 0 10 0 10 0 1.00 0.00 Oracle 48 939 13 0 13 0 13 0 1.00 0.00 Oracle 49 944 10 0 10 0 10 0 1.00 0.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 498 0 498 2 49.71 0.29 Average ··· 10.00 0.00 9.96 0.00 9.96 0.04 0.99 0.01 Micro avg ····················· 0.99 0.01 D.5. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM PAGES AND RANDOM LEARNING259
Duration Results of configuration Shattered Document with Random Pages and Random Learning
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 2.52 0.27 0.31 0.28 0.29 0.28 0.01 0.97 1.04 Oracle 2 1.64 0.27 0.27 0.27 0.27 0.27 0.00 1.01 0.99 Oracle 3 2.24 0.27 0.29 0.28 0.29 0.27 0.02 0.95 1.06 Oracle 4 1.93 0.27 0.28 0.27 0.27 0.28 0.00 1.01 0.99 Oracle 5 2.23 0.27 0.30 0.28 0.27 0.28 -0.01 1.02 0.98 Oracle 6 3.07 0.27 0.29 0.28 0.28 0.29 -0.01 1.03 0.97 Oracle 7 3.05 0.27 0.28 0.27 0.28 0.27 0.01 0.96 1.04 Oracle 8 3.08 0.27 0.30 0.28 0.27 0.27 0.01 0.98 1.02 Oracle 9 1.79 0.20 0.28 0.26 0.28 0.27 0.01 0.98 1.02 Oracle 10 5.52 0.27 0.29 0.27 0.27 0.27 0.00 0.99 1.01 Oracle 11 3.28 0.27 0.28 0.27 0.27 0.27 0.00 0.98 1.02 Oracle 12 3.60 0.27 0.29 0.27 0.27 0.27 0.00 1.00 1.00 Oracle 13 4.13 0.27 0.28 0.27 0.28 0.27 0.01 0.96 1.04 Oracle 14 3.09 0.27 0.30 0.28 0.27 0.28 -0.01 1.02 0.98 Oracle 15 2.23 0.27 0.29 0.28 0.28 0.27 0.01 0.96 1.04 Oracle 16 1.64 0.27 0.28 0.27 0.27 0.28 -0.01 1.04 0.96 Oracle 17 2.50 0.27 0.30 0.28 0.30 0.27 0.02 0.92 1.09 Oracle 18 3.02 0.27 0.28 0.27 0.27 0.27 0.00 0.99 1.01 Oracle 19 1.69 0.27 0.30 0.28 0.28 0.27 0.01 0.97 1.03 Oracle 20 1.66 0.27 0.28 0.27 0.27 0.27 0.00 0.99 1.01 Oracle 21 2.78 0.27 0.29 0.28 0.28 0.27 0.01 0.98 1.03 Oracle 22 2.75 0.27 0.28 0.27 0.28 0.27 0.01 0.96 1.04 Oracle 23 4.23 0.27 0.34 0.28 0.34 0.27 0.06 0.81 1.23 Oracle 24 3.04 0.27 0.28 0.27 0.28 0.28 0.01 0.98 1.02 Oracle 25 2.51 0.27 0.29 0.28 0.27 0.27 0.00 0.99 1.01 Oracle 26 4.13 0.27 0.29 0.27 0.28 0.27 0.02 0.95 1.05 Oracle 27 3.62 0.27 0.29 0.28 0.29 0.28 0.01 0.98 1.02 Oracle 28 1.97 0.27 0.29 0.28 0.27 0.27 0.01 0.98 1.02 Oracle 29 1.37 0.27 0.28 0.27 0.27 0.27 -0.01 1.02 0.98 Oracle 30 3.65 0.27 0.29 0.28 0.29 0.27 0.02 0.95 1.05 Oracle 31 3.04 0.27 0.28 0.27 0.27 0.28 -0.01 1.02 0.98 Oracle 32 3.88 0.27 0.29 0.28 0.28 0.27 0.01 0.98 1.02 Oracle 33 3.29 0.27 0.28 0.27 0.27 0.27 0.00 0.99 1.01 Oracle 34 2.24 0.27 0.32 0.28 0.27 0.27 0.00 0.99 1.01 Oracle 35 2.75 0.27 0.29 0.27 0.27 0.27 0.00 0.99 1.01 Oracle 36 3.61 0.27 0.29 0.28 0.27 0.28 -0.01 1.02 0.98 Oracle 37 3.03 0.27 0.29 0.27 0.27 0.27 0.00 1.01 0.99 Oracle 38 3.05 0.27 0.30 0.28 0.27 0.28 -0.01 1.02 0.98 Oracle 39 1.92 0.27 0.28 0.27 0.28 0.27 0.00 0.99 1.01 Oracle 40 3.87 0.27 0.29 0.27 0.28 0.27 0.00 0.99 1.01 Oracle 41 2.19 0.27 0.28 0.27 0.27 0.27 0.01 0.98 1.02 Oracle 42 0.28 0.28 0.28 0.28 0.28 0.28 0.00 1.00 1.00 Oracle 43 2.77 0.27 0.30 0.27 0.27 0.27 0.00 0.99 1.01 Oracle 44 1.67 0.27 0.29 0.28 0.27 0.28 -0.01 1.02 0.98 Oracle 45 2.50 0.27 0.29 0.28 0.28 0.27 0.01 0.97 1.03 Oracle 46 2.76 0.27 0.29 0.27 0.27 0.27 0.00 0.99 1.01 Oracle 47 2.77 0.27 0.29 0.27 0.28 0.27 0.00 0.99 1.01 Oracle 48 3.55 0.27 0.27 0.27 0.27 0.27 0.01 0.98 1.02 Oracle 49 2.75 0.27 0.28 0.27 0.27 0.27 0.00 1.00 1.00 Oracle 50 2.47 0.27 0.28 0.27 0.27 0.27 0.00 1.00 1.00 Total 138.33 13.50 14.47 13.83 13.99 13.77 0.22 49.28 50.80 Average 2.77 0.27 0.29 0.27 0.28 0.27 0.01 0.98 1.02 Micro avg ····················· 0.98 1.02 260 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Random Pages and Random Learning
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 0.29 0.31 0.27 0.27 0.27 0.27 0.27 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 2 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 3 0.29 0.28 0.27 0.28 0.29 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 4 0.27 0.27 0.27 0.27 0.27 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 5 0.27 0.28 0.27 0.27 0.27 0.30 0.27 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 6 0.28 0.28 0.29 0.27 0.28 0.27 0.27 0.28 0.27 0.27 0.29 0.29 0.29 0.29 0.29 0.29 0.29 0.29 0.29 0.29 Oracle 7 0.28 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.28 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 8 0.27 0.29 0.28 0.27 0.27 0.29 0.27 0.27 0.27 0.30 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 9 0.28 0.27 0.20 0.20 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 10 0.27 0.28 0.27 0.27 0.28 0.28 0.29 0.27 0.28 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 11 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 12 0.27 0.27 0.28 0.27 0.27 0.29 0.28 0.27 0.29 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 13 0.28 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 14 0.27 0.28 0.28 0.30 0.27 0.30 0.27 0.27 0.27 0.27 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 15 0.28 0.28 0.27 0.27 0.29 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 16 0.27 0.27 0.27 0.27 0.27 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 17 0.30 0.27 0.28 0.28 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 18 0.27 0.28 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 19 0.28 0.28 0.27 0.30 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 20 0.27 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 21 0.28 0.28 0.29 0.27 0.28 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 22 0.28 0.27 0.28 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 23 0.34 0.28 0.28 0.28 0.27 0.28 0.27 0.27 0.27 0.27 0.31 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 24 0.28 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 25 0.27 0.28 0.29 0.28 0.27 0.29 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 26 0.28 0.27 0.27 0.29 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 27 0.29 0.28 0.28 0.27 0.28 0.27 0.27 0.27 0.27 0.28 0.27 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 28 0.27 0.27 0.29 0.29 0.27 0.29 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 29 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 30 0.29 0.27 0.28 0.29 0.29 0.29 0.29 0.28 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 31 0.27 0.27 0.28 0.28 0.27 0.27 0.28 0.27 0.27 0.27 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 32 0.28 0.29 0.27 0.27 0.28 0.27 0.27 0.28 0.28 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 33 0.27 0.28 0.28 0.27 0.27 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 34 0.27 0.32 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 35 0.27 0.29 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 36 0.27 0.27 0.27 0.27 0.27 0.27 0.28 0.27 0.29 0.29 0.27 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 37 0.27 0.29 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 38 0.27 0.27 0.27 0.30 0.29 0.27 0.27 0.27 0.27 0.27 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 39 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 40 0.28 0.27 0.29 0.27 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 41 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 42 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 43 0.27 0.30 0.28 0.27 0.29 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 44 0.27 0.28 0.27 0.27 0.29 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 0.28 Oracle 45 0.28 0.29 0.28 0.27 0.28 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 46 0.27 0.27 0.28 0.29 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 47 0.28 0.29 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 48 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 49 0.27 0.27 0.27 0.27 0.27 0.27 0.28 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Oracle 50 0.27 0.27 0.27 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 Total 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 Average 0.28 0.28 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 0.27 D.5. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM PAGES AND RANDOM LEARNING261
Size Results of configuration Shattered Document with Ran- dom Pages and Random Learning
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 2 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 3 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 4 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 5 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 6 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 7 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 8 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 9 37 1 7 5.29 7 7 0 1.00 1.00 Oracle 10 140 7 7 7.00 7 7 0 1.00 1.00 Oracle 11 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 12 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 13 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 14 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 15 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 16 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 17 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 18 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 19 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 20 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 21 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 22 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 23 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 24 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 27 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 28 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 29 35 7 7 7.00 7 7 0 1.00 1.00 Oracle 30 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 31 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 32 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 33 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 34 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 35 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 36 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 37 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 38 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 39 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 40 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 41 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 44 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 45 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 46 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 47 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 48 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 49 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 3488 344 350 348.29 350 350 0 50.00 50.00 Average 69.76 6.88 7.00 6.97 7.00 7.00 0.00 1.00 1.00 Micro avg ····················· 1.00 1.00 262 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Ran- dom Pages and Random Learning
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 2 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 3 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 4 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 5 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 6 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 8 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 9 7 7 1 1 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 10 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 11 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 12 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 13 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 14 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 15 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 16 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 17 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 18 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 19 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 20 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 21 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 22 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 23 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 24 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 27 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 28 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 29 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 30 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 31 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 32 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 33 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 34 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 35 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 36 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 37 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 38 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 39 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 40 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 41 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 44 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 45 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 46 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 47 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 48 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 49 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 350 350 344 344 350 350 350 350 350 350 350 350 350 350 350 350 350 350 350 350 Average 7.00 7.00 6.88 6.88 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 D.5. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM PAGES AND RANDOM LEARNING263
Cost Results of configuration Shattered Document with Ran- dom Pages and Random Learning
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 2 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 3 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 4 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 5 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 6 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 7 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 8 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 9 37.00 1.00 7.00 5.29 7.00 7.00 0.00 1.00 1.00 Oracle 10 140.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 11 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 12 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 13 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 14 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 15 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 16 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 17 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 18 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 19 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 20 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 21 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 22 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 23 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 24 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 27 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 28 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 29 35.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 30 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 31 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 32 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 33 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 34 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 35 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 36 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 37 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 38 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 39 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 40 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 41 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 44 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 45 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 46 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 47 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 48 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 49 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 3488.00 344.00 350.00 348.29 350.00 350.00 0.00 50.00 50.00 Average 69.76 6.88 7.00 6.97 7.00 7.00 0.00 1.00 1.00 Micro avg ····················· 1.00 1.00 264 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Ran- dom Pages and Random Learning
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 2 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 3 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 4 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 5 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 6 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 7 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 8 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 9 7.00 7.00 1.00 1.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 10 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 11 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 12 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 13 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 14 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 15 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 16 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 17 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 18 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 19 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 20 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 21 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 22 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 23 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 24 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 27 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 28 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 29 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 30 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 31 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 32 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 33 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 34 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 35 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 36 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 37 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 38 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 39 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 40 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 41 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 44 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 45 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 46 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 47 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 48 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 49 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 350 350 344 344 350 350 350 350 350 350 350 350 350 350 350 350 350 350 350 350 Average 7.00 7.00 6.88 6.88 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 D.6. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM DOCUMENT (FIXED)265 D.6 Complete results of configuration Shattered Document with Random Document (fixed)
Shattered Document format with Random Document. Starting with a random doc- ument and then evolving normally (with adaptation). The random document is as follows: each node X has exacly N connections to random nodes not X. Normally, N = 1. Using functions Super Page, Choose, Recentre from Kasim2.Activation. (Bug fixed).
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 1.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 0.00 Weight of heading-to-paragraph Child links = 0.00 Weight of heading-to-heading Child links = 0.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.98
Size0 = 4.10
Size1 = 3.40 Size gain = 1.20
Cost0 = 4.00
Cost1 = 3.38 Cost gain = 1.18 266 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Random Document (fixed)
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 0 0 0 9 0.00 1.00 Oracle 2 30 6 0 0 0 0 6 0.00 1.00 Oracle 3 40 8 0 0 0 0 8 0.00 1.00 Oracle 4 46 7 0 0 0 0 7 0.00 1.00 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 0 0 0 0 7 0.00 1.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 1 0 0 1 7 0.13 0.88 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 0 0 0 9 0.00 1.00 Oracle 18 269 11 0 0 0 0 11 0.00 1.00 Oracle 19 290 6 0 0 0 0 6 0.00 1.00 Oracle 20 307 6 0 0 0 0 6 0.00 1.00 Oracle 21 366 10 0 0 0 0 10 0.00 1.00 Oracle 22 410 10 0 0 0 0 10 0.00 1.00 Oracle 23 455 15 0 0 0 0 15 0.00 1.00 Oracle 24 508 11 0 0 0 0 11 0.00 1.00 Oracle 25 512 9 8 0 0 8 1 0.89 0.11 Oracle 26 513 15 0 0 0 0 15 0.00 1.00 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 0 0 0 7 0.00 1.00 Oracle 29 542 5 0 0 0 0 5 0.00 1.00 Oracle 30 554 13 2 0 0 2 11 0.16 0.84 Oracle 31 601 11 0 0 0 0 11 0.00 1.00 Oracle 32 606 14 0 0 0 0 14 0.00 1.00 Oracle 33 616 12 0 0 0 0 12 0.00 1.00 Oracle 34 649 8 0 0 0 0 8 0.00 1.00 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 0 0 0 11 0.00 1.00 Oracle 39 786 7 1 0 0 1 6 0.14 0.86 Oracle 40 840 14 0 0 0 0 14 0.00 1.00 Oracle 41 846 8 0 0 0 0 8 0.00 1.00 Oracle 42 883 1 0 0 0 0 1 0.00 1.00 Oracle 43 890 10 0 0 0 0 10 0.00 1.00 Oracle 44 897 6 0 0 0 0 6 0.00 1.00 Oracle 45 899 9 0 0 0 0 9 0.00 1.00 Oracle 46 929 10 0 0 0 0 10 0.00 1.00 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 1 0 0 1 12 0.08 0.92 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 0 0 0 9 0.00 1.00 Total ··· 500 13 0 0 13 487 1.39 48.61 Average ··· 10.00 0.26 0.00 0.00 0.26 9.74 0.03 0.97 Micro avg ····················· 0.02 0.98 D.6. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM DOCUMENT (FIXED)267
Duration Results of configuration Shattered Document with Random Document (fixed)
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 7.38 0.79 0.92 0.82 0.92 0.86 0.06 0.93 1.08 Oracle 2 3.91 0.56 0.72 0.66 0.70 0.70 -0.01 1.01 0.99 Oracle 3 5.00 0.55 0.94 0.63 0.94 0.56 0.38 0.60 1.66 Oracle 4 5.71 0.78 0.93 0.81 0.93 0.78 0.14 0.84 1.19 Oracle 5 4.66 0.55 0.69 0.59 0.69 0.55 0.13 0.81 1.23 Oracle 6 3.95 0.34 0.47 0.36 0.34 0.34 -0.01 1.02 0.98 Oracle 7 6.27 0.55 0.69 0.57 0.69 0.55 0.14 0.80 1.25 Oracle 8 8.80 0.77 0.93 0.80 0.93 0.77 0.15 0.84 1.20 Oracle 9 5.10 0.55 0.91 0.73 0.79 0.55 0.23 0.70 1.42 Oracle 10 16.20 0.78 0.92 0.81 0.92 0.78 0.14 0.84 1.18 Oracle 11 7.39 0.55 0.70 0.62 0.69 0.69 0.00 0.99 1.01 Oracle 12 7.48 0.55 0.69 0.58 0.69 0.56 0.13 0.81 1.23 Oracle 13 9.04 0.55 0.82 0.60 0.71 0.56 0.15 0.79 1.27 Oracle 14 3.92 0.34 0.48 0.36 0.34 0.34 0.01 0.98 1.02 Oracle 15 6.58 0.77 1.03 0.82 1.03 0.77 0.25 0.76 1.32 Oracle 16 3.61 0.56 0.69 0.60 0.69 0.56 0.13 0.81 1.23 Oracle 17 5.80 0.55 0.73 0.64 0.72 0.55 0.16 0.77 1.29 Oracle 18 8.86 0.79 0.92 0.80 0.92 0.79 0.13 0.86 1.16 Oracle 19 4.98 0.79 0.92 0.83 0.92 0.79 0.13 0.85 1.17 Oracle 20 5.08 0.79 0.96 0.84 0.93 0.79 0.13 0.85 1.17 Oracle 21 5.64 0.55 0.58 0.56 0.56 0.58 -0.02 1.02 0.98 Oracle 22 6.43 0.55 0.82 0.64 0.80 0.59 0.21 0.73 1.36 Oracle 23 8.63 0.55 0.69 0.58 0.69 0.55 0.13 0.80 1.24 Oracle 24 8.79 0.78 0.82 0.80 0.79 0.81 -0.02 1.03 0.97 Oracle 25 9.05 1.00 1.02 1.01 1.00 1.01 -0.01 1.01 0.99 Oracle 26 8.45 0.55 0.59 0.56 0.56 0.59 -0.02 1.03 0.97 Oracle 27 10.37 0.70 0.85 0.80 0.70 0.79 -0.09 1.13 0.89 Oracle 28 4.05 0.55 0.70 0.58 0.70 0.56 0.13 0.81 1.23 Oracle 29 4.23 0.79 0.92 0.84 0.92 0.79 0.13 0.85 1.17 Oracle 30 10.98 0.78 1.02 0.84 1.02 0.78 0.24 0.77 1.30 Oracle 31 6.33 0.55 0.70 0.58 0.70 0.56 0.13 0.80 1.24 Oracle 32 6.69 0.46 0.51 0.48 0.48 0.47 0.02 0.96 1.04 Oracle 33 6.88 0.55 0.69 0.57 0.69 0.55 0.13 0.81 1.23 Oracle 34 4.63 0.55 0.69 0.58 0.69 0.56 0.13 0.82 1.22 Oracle 35 7.96 0.77 0.83 0.80 0.78 0.77 0.02 0.98 1.02 Oracle 36 10.52 0.78 0.92 0.81 0.92 0.78 0.14 0.84 1.18 Oracle 37 6.59 0.55 0.92 0.60 0.92 0.56 0.37 0.60 1.65 Oracle 38 4.17 0.34 0.48 0.38 0.48 0.34 0.14 0.70 1.43 Oracle 39 5.95 0.79 1.02 0.85 1.02 0.80 0.23 0.78 1.28 Oracle 40 8.03 0.55 0.69 0.57 0.69 0.58 0.11 0.84 1.19 Oracle 41 4.66 0.55 0.69 0.59 0.69 0.57 0.12 0.83 1.21 Oracle 42 0.70 0.70 0.70 0.70 0.70 0.70 0.00 1.00 1.00 Oracle 43 5.81 0.55 0.72 0.58 0.56 0.58 -0.01 1.02 0.98 Oracle 44 3.91 0.56 0.71 0.65 0.71 0.69 0.02 0.97 1.03 Oracle 45 4.25 0.46 0.51 0.47 0.48 0.46 0.01 0.98 1.02 Oracle 46 5.75 0.55 0.69 0.58 0.69 0.55 0.13 0.80 1.24 Oracle 47 7.91 0.79 0.80 0.79 0.79 0.79 0.00 1.00 1.00 Oracle 48 8.02 0.55 1.02 0.62 1.02 0.56 0.46 0.55 1.82 Oracle 49 4.70 0.46 0.48 0.47 0.47 0.46 0.01 0.98 1.02 Oracle 50 7.09 0.77 0.84 0.79 0.77 0.84 -0.05 1.07 0.93 Total 326.92 31.20 38.65 33.12 37.47 32.09 5.38 43.45 58.88 Average 6.54 0.63 0.77 0.66 0.75 0.64 0.11 0.87 1.18 Micro avg ····················· 0.86 1.16 268 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Random Document (fixed)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 0.92 0.79 0.79 0.80 0.83 0.79 0.80 0.80 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 Oracle 2 0.70 0.56 0.72 0.56 0.68 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 Oracle 3 0.94 0.70 0.55 0.56 0.57 0.56 0.55 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 4 0.93 0.82 0.79 0.81 0.80 0.79 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 Oracle 5 0.69 0.56 0.56 0.56 0.56 0.55 0.62 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 6 0.34 0.47 0.34 0.38 0.34 0.34 0.34 0.35 0.34 0.36 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 Oracle 7 0.69 0.55 0.56 0.56 0.56 0.56 0.56 0.56 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 8 0.93 0.79 0.79 0.78 0.79 0.80 0.79 0.79 0.78 0.78 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 Oracle 9 0.79 0.91 0.70 0.71 0.70 0.74 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 10 0.92 0.81 0.86 0.79 0.80 0.79 0.79 0.80 0.79 0.87 0.80 0.80 0.80 0.80 0.81 0.80 0.79 0.80 0.82 0.78 Oracle 11 0.69 0.56 0.56 0.69 0.69 0.56 0.55 0.57 0.56 0.56 0.70 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 Oracle 12 0.69 0.56 0.56 0.59 0.56 0.56 0.56 0.56 0.57 0.56 0.59 0.55 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 13 0.71 0.55 0.58 0.56 0.58 0.56 0.56 0.82 0.58 0.70 0.57 0.56 0.58 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 14 0.34 0.48 0.34 0.34 0.37 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 Oracle 15 1.03 0.83 0.78 0.78 0.80 0.79 0.78 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 Oracle 16 0.69 0.56 0.56 0.66 0.57 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 17 0.72 0.59 0.69 0.73 0.70 0.70 0.56 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 18 0.92 0.79 0.81 0.79 0.80 0.80 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 Oracle 19 0.92 0.79 0.91 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 Oracle 20 0.93 0.79 0.82 0.79 0.96 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 Oracle 21 0.56 0.56 0.57 0.56 0.56 0.55 0.56 0.56 0.56 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 22 0.80 0.82 0.79 0.59 0.58 0.55 0.55 0.56 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 Oracle 23 0.69 0.56 0.58 0.56 0.56 0.56 0.56 0.56 0.56 0.59 0.56 0.57 0.56 0.57 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 24 0.79 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.78 0.82 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 0.81 Oracle 25 1.00 1.00 1.01 1.01 1.00 1.01 1.02 1.00 1.01 1.01 1.01 1.01 1.01 1.01 1.01 1.01 1.01 1.01 1.01 1.01 Oracle 26 0.56 0.55 0.55 0.56 0.57 0.56 0.56 0.56 0.57 0.56 0.56 0.56 0.55 0.58 0.59 0.59 0.59 0.59 0.59 0.59 Oracle 27 0.70 0.80 0.80 0.80 0.80 0.81 0.81 0.80 0.85 0.81 0.80 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 Oracle 28 0.70 0.56 0.56 0.56 0.56 0.55 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 29 0.92 0.79 0.83 0.89 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 Oracle 30 1.02 1.01 0.92 0.79 0.84 0.79 0.80 0.80 0.82 0.80 0.80 0.80 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 Oracle 31 0.70 0.56 0.56 0.56 0.57 0.57 0.56 0.55 0.56 0.57 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 32 0.48 0.48 0.48 0.51 0.48 0.47 0.48 0.48 0.48 0.48 0.47 0.46 0.50 0.47 0.47 0.47 0.47 0.47 0.47 0.47 Oracle 33 0.69 0.56 0.56 0.57 0.56 0.56 0.56 0.56 0.55 0.56 0.56 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 34 0.69 0.58 0.57 0.55 0.56 0.56 0.55 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 35 0.78 0.78 0.80 0.80 0.83 0.80 0.80 0.79 0.82 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 0.77 Oracle 36 0.92 0.80 0.80 0.80 0.80 0.88 0.79 0.79 0.78 0.80 0.79 0.79 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 Oracle 37 0.92 0.59 0.59 0.56 0.56 0.58 0.56 0.55 0.56 0.55 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 38 0.48 0.48 0.35 0.47 0.34 0.35 0.34 0.35 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 Oracle 39 1.02 0.95 0.79 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 Oracle 40 0.69 0.56 0.56 0.57 0.56 0.56 0.56 0.56 0.59 0.56 0.56 0.58 0.55 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 41 0.69 0.55 0.56 0.57 0.55 0.56 0.60 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 42 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 Oracle 43 0.56 0.56 0.56 0.56 0.56 0.57 0.72 0.57 0.55 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 44 0.71 0.69 0.69 0.56 0.58 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 0.69 Oracle 45 0.48 0.47 0.47 0.47 0.47 0.51 0.46 0.47 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 Oracle 46 0.69 0.56 0.56 0.56 0.56 0.56 0.57 0.56 0.56 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 47 0.79 0.80 0.79 0.79 0.80 0.80 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 0.79 Oracle 48 1.02 0.79 0.59 0.56 0.56 0.56 0.58 0.56 0.56 0.57 0.55 0.55 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 49 0.47 0.47 0.47 0.48 0.47 0.47 0.47 0.48 0.48 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 Oracle 50 0.77 0.78 0.78 0.80 0.78 0.80 0.78 0.77 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 Total 37 34 33 33 33 32 32 32 32 32 32 32 32 32 32 32 32 32 32 32 Average 0.75 0.67 0.66 0.65 0.65 0.65 0.64 0.64 0.64 0.65 0.64 0.64 0.64 0.64 0.64 0.64 0.64 0.64 0.64 0.64 D.6. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM DOCUMENT (FIXED)269
Size Results of configuration Shattered Document with Ran- dom Document (fixed)
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 37 4 5 4.11 5 4 1 0.80 1.25 Oracle 2 22 3 4 3.66 4 4 0 1.00 1.00 Oracle 3 27 3 5 3.38 5 3 2 0.60 1.66 Oracle 4 29 4 5 4.14 5 4 1 0.80 1.25 Oracle 5 25 3 4 3.13 4 3 1 0.75 1.34 Oracle 6 23 2 3 2.09 2 2 0 1.00 1.00 Oracle 7 34 3 4 3.09 4 3 1 0.75 1.34 Oracle 8 45 4 5 4.09 5 4 1 0.80 1.25 Oracle 9 28 3 5 4.00 4 3 1 0.75 1.34 Oracle 10 81 4 5 4.05 5 4 1 0.80 1.25 Oracle 11 41 3 4 3.41 4 4 0 1.00 1.00 Oracle 12 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 13 47 3 4 3.13 4 3 1 0.75 1.34 Oracle 14 23 2 3 2.09 2 2 0 1.00 1.00 Oracle 15 33 4 5 4.13 5 4 1 0.80 1.25 Oracle 16 19 3 4 3.16 4 3 1 0.75 1.34 Oracle 17 32 3 4 3.55 4 3 1 0.75 1.34 Oracle 18 45 4 5 4.09 5 4 1 0.80 1.25 Oracle 19 25 4 5 4.16 5 4 1 0.80 1.25 Oracle 20 26 4 5 4.34 5 4 1 0.80 1.25 Oracle 21 30 3 3 3.00 3 3 0 1.00 1.00 Oracle 22 33 3 4 3.30 4 3 1 0.75 1.34 Oracle 23 46 3 4 3.07 4 3 1 0.75 1.34 Oracle 24 44 4 4 4.00 4 4 0 1.00 1.00 Oracle 25 45 5 5 5.00 5 5 0 1.00 1.00 Oracle 26 45 3 3 3.00 3 3 0 1.00 1.00 Oracle 27 52 4 4 4.00 4 4 0 1.00 1.00 Oracle 28 22 3 4 3.14 4 3 1 0.75 1.34 Oracle 29 21 4 5 4.20 5 4 1 0.80 1.25 Oracle 30 55 4 5 4.23 5 4 1 0.80 1.25 Oracle 31 34 3 4 3.09 4 3 1 0.75 1.34 Oracle 32 42 3 3 3.00 3 3 0 1.00 1.00 Oracle 33 37 3 4 3.09 4 3 1 0.75 1.34 Oracle 34 25 3 4 3.13 4 3 1 0.75 1.34 Oracle 35 40 4 4 4.00 4 4 0 1.00 1.00 Oracle 36 53 4 5 4.08 5 4 1 0.80 1.25 Oracle 37 35 3 5 3.18 5 3 2 0.60 1.66 Oracle 38 25 2 3 2.27 3 2 1 0.66 1.50 Oracle 39 30 4 5 4.29 5 4 1 0.80 1.25 Oracle 40 43 3 4 3.07 4 3 1 0.75 1.34 Oracle 41 25 3 4 3.13 4 3 1 0.75 1.34 Oracle 42 4 4 4 4.00 4 4 0 1.00 1.00 Oracle 43 30 3 3 3.00 3 3 0 1.00 1.00 Oracle 44 22 3 4 3.66 4 4 0 1.00 1.00 Oracle 45 27 3 3 3.00 3 3 0 1.00 1.00 Oracle 46 31 3 4 3.10 4 3 1 0.75 1.34 Oracle 47 40 4 4 4.00 4 4 0 1.00 1.00 Oracle 48 42 3 5 3.23 5 3 2 0.60 1.66 Oracle 49 30 3 3 3.00 3 3 0 1.00 1.00 Oracle 50 36 4 4 4.00 4 4 0 1.00 1.00 Total 1726 167 208 175.15 205 170 35 42.07 60.84 Average 34.52 3.34 4.16 3.50 4.10 3.40 0.70 0.84 1.22 Micro avg ····················· 0.83 1.20 270 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Ran- dom Document (fixed)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 2 4 3 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 3 5 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 4 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 5 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 6 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 8 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 9 4 5 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 10 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 11 4 3 3 4 4 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 Oracle 12 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 13 4 3 3 3 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 14 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 15 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 16 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 17 4 3 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 18 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 19 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 20 5 4 4 4 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 21 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 22 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 23 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 24 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 25 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 26 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 27 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 28 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 29 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 30 5 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 31 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 32 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 33 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 34 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 35 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 36 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 37 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 38 3 3 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 39 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 40 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 41 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 42 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 43 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 44 4 4 4 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 45 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 46 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 47 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 48 5 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 49 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 50 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Total 205 178 173 171 172 171 169 169 169 170 170 170 170 170 170 170 170 170 170 170 Average 4.10 3.56 3.46 3.42 3.44 3.42 3.38 3.38 3.38 3.40 3.40 3.40 3.40 3.40 3.40 3.40 3.40 3.40 3.40 3.40 D.6. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH RANDOM DOCUMENT (FIXED)271
Cost Results of configuration Shattered Document with Ran- dom Document (fixed)
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 37.00 4.00 5.00 4.11 5.00 4.00 1.00 0.80 1.25 Oracle 2 22.00 3.00 4.00 3.66 4.00 4.00 0.00 1.00 1.00 Oracle 3 27.00 3.00 5.00 3.38 5.00 3.00 2.00 0.60 1.66 Oracle 4 29.00 4.00 5.00 4.14 5.00 4.00 1.00 0.80 1.25 Oracle 5 25.00 3.00 4.00 3.13 4.00 3.00 1.00 0.75 1.34 Oracle 6 23.00 2.00 3.00 2.09 2.00 2.00 0.00 1.00 1.00 Oracle 7 34.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 8 45.00 4.00 5.00 4.09 5.00 4.00 1.00 0.80 1.25 Oracle 9 28.00 3.00 5.00 4.00 4.00 3.00 1.00 0.75 1.34 Oracle 10 81.00 4.00 5.00 4.05 5.00 4.00 1.00 0.80 1.25 Oracle 11 41.00 3.00 4.00 3.41 4.00 4.00 0.00 1.00 1.00 Oracle 12 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 13 47.00 3.00 4.00 3.13 4.00 3.00 1.00 0.75 1.34 Oracle 14 23.00 2.00 3.00 2.09 2.00 2.00 0.00 1.00 1.00 Oracle 15 32.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 16 19.00 3.00 4.00 3.16 4.00 3.00 1.00 0.75 1.34 Oracle 17 32.00 3.00 4.00 3.55 4.00 3.00 1.00 0.75 1.34 Oracle 18 45.00 4.00 5.00 4.09 5.00 4.00 1.00 0.80 1.25 Oracle 19 25.00 4.00 5.00 4.16 5.00 4.00 1.00 0.80 1.25 Oracle 20 26.00 4.00 5.00 4.34 5.00 4.00 1.00 0.80 1.25 Oracle 21 30.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 22 33.00 3.00 4.00 3.30 4.00 3.00 1.00 0.75 1.34 Oracle 23 46.00 3.00 4.00 3.07 4.00 3.00 1.00 0.75 1.34 Oracle 24 44.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 25 37.00 4.00 5.00 4.11 4.00 4.00 0.00 1.00 1.00 Oracle 26 45.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 27 52.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 28 22.00 3.00 4.00 3.14 4.00 3.00 1.00 0.75 1.34 Oracle 29 21.00 4.00 5.00 4.20 5.00 4.00 1.00 0.80 1.25 Oracle 30 53.00 4.00 5.00 4.08 4.00 4.00 0.00 1.00 1.00 Oracle 31 34.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 32 42.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 33 37.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 34 25.00 3.00 4.00 3.13 4.00 3.00 1.00 0.75 1.34 Oracle 35 40.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 36 53.00 4.00 5.00 4.08 5.00 4.00 1.00 0.80 1.25 Oracle 37 35.00 3.00 5.00 3.18 5.00 3.00 2.00 0.60 1.66 Oracle 38 25.00 2.00 3.00 2.27 3.00 2.00 1.00 0.66 1.50 Oracle 39 29.00 4.00 5.00 4.14 4.00 4.00 0.00 1.00 1.00 Oracle 40 43.00 3.00 4.00 3.07 4.00 3.00 1.00 0.75 1.34 Oracle 41 25.00 3.00 4.00 3.13 4.00 3.00 1.00 0.75 1.34 Oracle 42 4.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 43 30.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 44 22.00 3.00 4.00 3.66 4.00 4.00 0.00 1.00 1.00 Oracle 45 27.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 46 31.00 3.00 4.00 3.10 4.00 3.00 1.00 0.75 1.34 Oracle 47 40.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 48 41.00 3.00 4.00 3.16 4.00 3.00 1.00 0.75 1.34 Oracle 49 30.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 50 36.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Total 1713.00 166.00 206.00 173.77 200.00 169.00 31.00 42.82 59.75 Average 34.26 3.32 4.12 3.48 4.00 3.38 0.62 0.86 1.20 Micro avg ····················· 0.84 1.18 272 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Ran- dom Document (fixed)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 2 4.00 3.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 3 5.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 4 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 5 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 6 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 7 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 8 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 9 4.00 5.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 10 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 11 4.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 12 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 13 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 14 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 15 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 16 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 17 4.00 3.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 18 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 19 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 20 5.00 4.00 4.00 4.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 21 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 22 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 23 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 24 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 25 4.00 4.00 4.00 4.00 4.00 4.00 4.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 26 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 27 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 28 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 29 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 30 4.00 4.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 31 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 32 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 33 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 34 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 35 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 36 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 37 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 38 3.00 3.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 39 4.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 40 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 41 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 42 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 43 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 44 4.00 4.00 4.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 45 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 46 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 47 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 48 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 49 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 50 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Total 200 176 172 170 171 170 168 169 168 169 169 169 169 169 169 169 169 169 169 169 Average 4.00 3.52 3.44 3.40 3.42 3.40 3.36 3.38 3.36 3.38 3.38 3.38 3.38 3.38 3.38 3.38 3.38 3.38 3.38 3.38 D.7. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL 273 D.7 Complete results of configuration Structural
The legacy document in its original structure, without adaptation. Using special functions Start, Choose, Recentre from Kasim2.Structural. Using the model of the ARM on Kasim2.Documents.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 100 Page size (number of items) = 10 Quantity of expanded items = 0 Update travel matrix strategy = False Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 6.00 Weight of Next links = 0.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.68
Size0 = 3.16
Size1 = 3.16 Size gain = 1.00
Cost0 = 9.15
Cost1 = 9.15 Cost gain = 1.00 274 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Structural
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 2 0 0 0 0 2 0.00 1.00 Oracle 2 30 2 0 0 0 0 2 0.00 1.00 Oracle 3 40 2 0 0 0 0 2 0.00 1.00 Oracle 4 46 2 0 0 0 0 2 0.00 1.00 Oracle 5 56 2 0 0 0 0 2 0.00 1.00 Oracle 6 85 2 0 0 0 0 2 0.00 1.00 Oracle 7 108 2 0 0 0 0 2 0.00 1.00 Oracle 8 125 2 0 0 0 0 2 0.00 1.00 Oracle 9 128 2 0 0 0 0 2 0.00 1.00 Oracle 10 130 2 0 0 0 0 2 0.00 1.00 Oracle 11 138 2 0 0 0 0 2 0.00 1.00 Oracle 12 157 2 0 0 0 0 2 0.00 1.00 Oracle 13 171 2 0 0 0 0 2 0.00 1.00 Oracle 14 173 2 0 0 0 0 2 0.00 1.00 Oracle 15 241 2 0 0 0 0 2 0.00 1.00 Oracle 16 265 2 0 0 0 0 2 0.00 1.00 Oracle 17 266 2 0 0 0 0 2 0.00 1.00 Oracle 18 269 2 0 0 0 0 2 0.00 1.00 Oracle 19 290 2 0 0 0 0 2 0.00 1.00 Oracle 20 307 2 0 0 2 2 0 1.00 0.00 Oracle 21 366 2 0 0 2 2 0 1.00 0.00 Oracle 22 410 2 0 0 2 2 0 1.00 0.00 Oracle 23 455 2 0 0 0 0 2 0.00 1.00 Oracle 24 508 2 0 0 0 0 2 0.00 1.00 Oracle 25 512 2 0 0 0 0 2 0.00 1.00 Oracle 26 513 2 0 0 0 0 2 0.00 1.00 Oracle 27 516 2 0 0 0 0 2 0.00 1.00 Oracle 28 521 2 0 0 0 0 2 0.00 1.00 Oracle 29 542 2 0 0 2 2 0 1.00 0.00 Oracle 30 554 2 0 0 2 2 0 1.00 0.00 Oracle 31 601 2 0 0 2 2 0 1.00 0.00 Oracle 32 606 2 0 0 2 2 0 1.00 0.00 Oracle 33 616 2 0 0 2 2 0 1.00 0.00 Oracle 34 649 2 0 0 2 2 0 1.00 0.00 Oracle 35 723 2 0 0 0 0 2 0.00 1.00 Oracle 36 725 2 0 0 0 0 2 0.00 1.00 Oracle 37 738 2 0 0 0 0 2 0.00 1.00 Oracle 38 741 2 0 0 0 0 2 0.00 1.00 Oracle 39 786 2 0 0 2 2 0 1.00 0.00 Oracle 40 840 2 0 0 2 2 0 1.00 0.00 Oracle 41 846 2 0 0 2 2 0 1.00 0.00 Oracle 42 883 2 0 0 0 0 2 0.00 1.00 Oracle 43 890 2 0 0 2 2 0 1.00 0.00 Oracle 44 897 2 0 0 2 2 0 1.00 0.00 Oracle 45 899 2 0 0 2 2 0 1.00 0.00 Oracle 46 929 2 0 0 0 0 2 0.00 1.00 Oracle 47 936 2 0 0 0 0 2 0.00 1.00 Oracle 48 939 2 0 0 0 0 2 0.00 1.00 Oracle 49 944 2 0 0 0 0 2 0.00 1.00 Oracle 50 999 2 0 0 2 2 0 1.00 0.00 Total ··· 100 0 0 32 32 68 16.00 34.00 Average ··· 2.00 0.00 0.00 0.64 0.64 1.36 0.32 0.68 Micro avg ····················· 0.32 0.68 D.7. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL 275
Duration Results of configuration Structural
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 0.62 0.30 0.31 0.30 0.31 0.30 0.01 0.97 1.03 Oracle 2 0.61 0.30 0.30 0.30 0.30 0.30 0.01 0.98 1.02 Oracle 3 0.62 0.30 0.31 0.31 0.30 0.31 0.00 1.01 0.99 Oracle 4 0.63 0.31 0.32 0.31 0.32 0.31 0.01 0.98 1.02 Oracle 5 0.41 0.20 0.21 0.20 0.20 0.21 -0.01 1.02 0.98 Oracle 6 0.62 0.30 0.31 0.31 0.31 0.30 0.01 0.98 1.02 Oracle 7 0.41 0.20 0.21 0.20 0.20 0.21 -0.02 1.07 0.93 Oracle 8 0.42 0.21 0.21 0.21 0.21 0.21 0.00 0.98 1.02 Oracle 9 0.42 0.20 0.23 0.21 0.20 0.23 -0.02 1.13 0.88 Oracle 10 0.41 0.20 0.21 0.20 0.20 0.21 -0.01 1.05 0.95 Oracle 11 0.38 0.19 0.20 0.19 0.20 0.19 0.00 1.00 1.00 Oracle 12 0.48 0.20 0.27 0.23 0.27 0.20 0.08 0.72 1.40 Oracle 13 0.63 0.30 0.32 0.31 0.32 0.30 0.02 0.94 1.06 Oracle 14 0.63 0.31 0.32 0.31 0.32 0.31 0.01 0.97 1.03 Oracle 15 0.40 0.20 0.20 0.20 0.20 0.20 0.00 1.02 0.98 Oracle 16 0.41 0.19 0.22 0.20 0.22 0.19 0.02 0.88 1.14 Oracle 17 0.40 0.20 0.20 0.20 0.20 0.20 0.01 0.97 1.03 Oracle 18 0.40 0.20 0.20 0.20 0.20 0.20 -0.01 1.04 0.96 Oracle 19 0.40 0.20 0.20 0.20 0.20 0.20 0.01 0.96 1.04 Oracle 20 0.43 0.21 0.22 0.22 0.22 0.21 0.01 0.96 1.04 Oracle 21 0.44 0.21 0.22 0.22 0.21 0.22 -0.01 1.04 0.96 Oracle 22 0.43 0.21 0.22 0.21 0.22 0.21 0.00 0.99 1.01 Oracle 23 0.42 0.21 0.21 0.21 0.21 0.21 0.00 1.00 1.00 Oracle 24 0.41 0.20 0.20 0.20 0.20 0.20 0.01 0.96 1.04 Oracle 25 0.39 0.20 0.20 0.20 0.20 0.20 0.00 0.98 1.02 Oracle 26 0.40 0.20 0.20 0.20 0.20 0.20 0.01 0.96 1.04 Oracle 27 0.44 0.20 0.24 0.22 0.24 0.20 0.05 0.81 1.23 Oracle 28 0.39 0.20 0.20 0.20 0.20 0.20 0.01 0.98 1.02 Oracle 29 0.44 0.22 0.22 0.22 0.22 0.22 0.00 1.01 0.99 Oracle 30 0.39 0.20 0.20 0.20 0.20 0.20 0.01 0.97 1.03 Oracle 31 0.44 0.21 0.23 0.22 0.21 0.23 -0.02 1.07 0.94 Oracle 32 0.48 0.22 0.26 0.23 0.26 0.22 0.03 0.87 1.15 Oracle 33 0.45 0.22 0.23 0.23 0.23 0.22 0.02 0.94 1.07 Oracle 34 0.44 0.22 0.22 0.22 0.22 0.22 0.00 0.99 1.01 Oracle 35 0.42 0.20 0.22 0.21 0.22 0.20 0.02 0.94 1.07 Oracle 36 0.41 0.20 0.23 0.21 0.20 0.23 -0.03 1.16 0.86 Oracle 37 0.40 0.20 0.20 0.20 0.20 0.20 0.01 0.97 1.03 Oracle 38 0.41 0.20 0.21 0.20 0.20 0.21 -0.02 1.08 0.92 Oracle 39 0.43 0.21 0.22 0.21 0.21 0.22 0.00 1.02 0.98 Oracle 40 0.43 0.21 0.22 0.21 0.22 0.21 0.00 0.98 1.02 Oracle 41 0.48 0.22 0.26 0.24 0.26 0.22 0.03 0.87 1.16 Oracle 42 0.40 0.20 0.21 0.20 0.21 0.20 0.02 0.94 1.07 Oracle 43 0.45 0.23 0.23 0.23 0.23 0.23 0.00 1.00 1.00 Oracle 44 0.43 0.21 0.21 0.21 0.21 0.21 0.00 1.00 1.00 Oracle 45 0.43 0.21 0.22 0.22 0.22 0.21 0.01 0.95 1.05 Oracle 46 0.42 0.20 0.22 0.21 0.22 0.20 0.02 0.88 1.13 Oracle 47 0.41 0.20 0.21 0.20 0.21 0.20 0.02 0.92 1.09 Oracle 48 0.41 0.20 0.21 0.20 0.21 0.20 0.02 0.93 1.08 Oracle 49 0.40 0.20 0.21 0.20 0.20 0.21 -0.02 1.09 0.91 Oracle 50 0.45 0.22 0.23 0.23 0.22 0.23 -0.02 1.06 0.94 Total 22.41 10.89 11.52 11.20 11.34 11.07 0.27 49.02 51.33 Average 0.45 0.22 0.23 0.23 0.23 0.22 0.01 0.98 1.02 Micro avg ····················· 0.98 1.02 276 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Structural
1 2 Oracle 1 0.31 0.30 Oracle 2 0.30 0.30 Oracle 3 0.30 0.31 Oracle 4 0.32 0.31 Oracle 5 0.20 0.21 Oracle 6 0.31 0.30 Oracle 7 0.20 0.21 Oracle 8 0.21 0.21 Oracle 9 0.20 0.23 Oracle 10 0.20 0.21 Oracle 11 0.20 0.19 Oracle 12 0.27 0.20 Oracle 13 0.32 0.30 Oracle 14 0.32 0.31 Oracle 15 0.20 0.20 Oracle 16 0.22 0.19 Oracle 17 0.20 0.20 Oracle 18 0.20 0.20 Oracle 19 0.20 0.20 Oracle 20 0.22 0.21 Oracle 21 0.21 0.22 Oracle 22 0.22 0.21 Oracle 23 0.21 0.21 Oracle 24 0.20 0.20 Oracle 25 0.20 0.20 Oracle 26 0.20 0.20 Oracle 27 0.24 0.20 Oracle 28 0.20 0.20 Oracle 29 0.22 0.22 Oracle 30 0.20 0.20 Oracle 31 0.21 0.23 Oracle 32 0.26 0.22 Oracle 33 0.23 0.22 Oracle 34 0.22 0.22 Oracle 35 0.22 0.20 Oracle 36 0.20 0.23 Oracle 37 0.20 0.20 Oracle 38 0.20 0.21 Oracle 39 0.21 0.22 Oracle 40 0.22 0.21 Oracle 41 0.26 0.22 Oracle 42 0.21 0.20 Oracle 43 0.23 0.23 Oracle 44 0.21 0.21 Oracle 45 0.22 0.21 Oracle 46 0.22 0.20 Oracle 47 0.21 0.20 Oracle 48 0.21 0.20 Oracle 49 0.20 0.21 Oracle 50 0.22 0.23 Total 11 11 Average 0.23 0.22 D.7. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL 277
Size Results of configuration Structural
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 2 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 3 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 4 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 5 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 6 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 7 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 8 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 9 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 10 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 11 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 12 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 13 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 14 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 15 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 16 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 17 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 18 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 19 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 20 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 21 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 22 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 23 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 24 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 25 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 26 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 27 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 28 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 29 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 30 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 31 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 32 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 33 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 34 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 35 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 36 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 37 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 38 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 39 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 40 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 41 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 42 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 43 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 44 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 45 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 46 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 47 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 48 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 49 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 50 6 3 3 3.00 3 3 0 1.00 1.00 Total 316 158 158 158.00 158 158 0 50.00 50.00 Average 6.32 3.16 3.16 3.16 3.16 3.16 0.00 1.00 1.00 Micro avg ····················· 1.00 1.00 278 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Structural
1 2 Oracle 1 4 4 Oracle 2 4 4 Oracle 3 4 4 Oracle 4 4 4 Oracle 5 4 4 Oracle 6 4 4 Oracle 7 3 3 Oracle 8 3 3 Oracle 9 3 3 Oracle 10 3 3 Oracle 11 3 3 Oracle 12 3 3 Oracle 13 4 4 Oracle 14 4 4 Oracle 15 3 3 Oracle 16 3 3 Oracle 17 3 3 Oracle 18 3 3 Oracle 19 3 3 Oracle 20 3 3 Oracle 21 3 3 Oracle 22 3 3 Oracle 23 3 3 Oracle 24 3 3 Oracle 25 3 3 Oracle 26 3 3 Oracle 27 3 3 Oracle 28 3 3 Oracle 29 3 3 Oracle 30 3 3 Oracle 31 3 3 Oracle 32 3 3 Oracle 33 3 3 Oracle 34 3 3 Oracle 35 3 3 Oracle 36 3 3 Oracle 37 3 3 Oracle 38 3 3 Oracle 39 3 3 Oracle 40 3 3 Oracle 41 3 3 Oracle 42 3 3 Oracle 43 3 3 Oracle 44 3 3 Oracle 45 3 3 Oracle 46 3 3 Oracle 47 3 3 Oracle 48 3 3 Oracle 49 3 3 Oracle 50 3 3 Total 158 158 Average 3.16 3.16 D.7. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL 279
Cost Results of configuration Structural
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 11.05 5.53 5.53 5.53 5.53 5.53 0.00 1.00 1.00 Oracle 2 15.14 7.57 7.57 7.57 7.57 7.57 0.00 1.00 1.00 Oracle 3 13.86 6.93 6.93 6.93 6.93 6.93 0.00 1.00 1.00 Oracle 4 13.86 6.93 6.93 6.93 6.93 6.93 0.00 1.00 1.00 Oracle 5 14.83 7.41 7.41 7.41 7.41 7.41 0.00 1.00 1.00 Oracle 6 12.01 6.01 6.01 6.01 6.01 6.01 0.00 1.00 1.00 Oracle 7 15.96 7.98 7.98 7.98 7.98 7.98 0.00 1.00 1.00 Oracle 8 15.96 7.98 7.98 7.98 7.98 7.98 0.00 1.00 1.00 Oracle 9 15.96 7.98 7.98 7.98 7.98 7.98 0.00 1.00 1.00 Oracle 10 15.96 7.98 7.98 7.98 7.98 7.98 0.00 1.00 1.00 Oracle 11 12.12 6.06 6.06 6.06 6.06 6.06 0.00 1.00 1.00 Oracle 12 10.20 5.09 5.09 5.09 5.09 5.09 0.00 1.00 1.00 Oracle 13 11.49 5.75 5.75 5.75 5.75 5.75 0.00 1.00 1.00 Oracle 14 11.49 5.75 5.75 5.75 5.75 5.75 0.00 1.00 1.00 Oracle 15 9.82 4.91 4.91 4.91 4.91 4.91 0.00 1.00 1.00 Oracle 16 12.63 6.31 6.31 6.31 6.31 6.31 0.00 1.00 1.00 Oracle 17 12.63 6.31 6.31 6.31 6.31 6.31 0.00 1.00 1.00 Oracle 18 12.63 6.31 6.31 6.31 6.31 6.31 0.00 1.00 1.00 Oracle 19 14.81 7.41 7.41 7.41 7.41 7.41 0.00 1.00 1.00 Oracle 20 14.81 7.41 7.41 7.41 7.41 7.41 0.00 1.00 1.00 Oracle 21 12.05 6.02 6.02 6.02 6.02 6.02 0.00 1.00 1.00 Oracle 22 12.05 6.02 6.02 6.02 6.02 6.02 0.00 1.00 1.00 Oracle 23 20.20 10.09 10.09 10.09 10.09 10.09 0.00 1.00 1.00 Oracle 24 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 25 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 26 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 27 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 28 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 29 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 30 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 31 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 32 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 33 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 34 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 35 17.71 8.85 8.85 8.85 8.85 8.85 0.00 1.00 1.00 Oracle 36 17.71 8.85 8.85 8.85 8.85 8.85 0.00 1.00 1.00 Oracle 37 17.71 8.85 8.85 8.85 8.85 8.85 0.00 1.00 1.00 Oracle 38 17.71 8.85 8.85 8.85 8.85 8.85 0.00 1.00 1.00 Oracle 39 17.71 8.85 8.85 8.85 8.85 8.85 0.00 1.00 1.00 Oracle 40 18.21 9.10 9.10 9.10 9.10 9.10 0.00 1.00 1.00 Oracle 41 18.21 9.10 9.10 9.10 9.10 9.10 0.00 1.00 1.00 Oracle 42 17.02 8.51 8.51 8.51 8.51 8.51 0.00 1.00 1.00 Oracle 43 17.02 8.51 8.51 8.51 8.51 8.51 0.00 1.00 1.00 Oracle 44 17.02 8.51 8.51 8.51 8.51 8.51 0.00 1.00 1.00 Oracle 45 17.02 8.51 8.51 8.51 8.51 8.51 0.00 1.00 1.00 Oracle 46 21.02 10.52 10.52 10.52 10.52 10.52 0.00 1.00 1.00 Oracle 47 21.02 10.52 10.52 10.52 10.52 10.52 0.00 1.00 1.00 Oracle 48 21.02 10.52 10.52 10.52 10.52 10.52 0.00 1.00 1.00 Oracle 49 21.02 10.52 10.52 10.52 10.52 10.52 0.00 1.00 1.00 Oracle 50 21.02 10.52 10.52 10.52 10.52 10.52 0.00 1.00 1.00 Total 914.80 457.40 457.40 457.40 457.40 457.40 0.00 50.00 50.00 Average 18.30 9.15 9.15 9.15 9.15 9.15 0.00 1.00 1.00 Micro avg ····················· 1.00 1.00 280 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Structural
1 2 Oracle 1 5.53 5.53 Oracle 2 7.57 7.57 Oracle 3 6.93 6.93 Oracle 4 6.93 6.93 Oracle 5 7.41 7.41 Oracle 6 6.01 6.01 Oracle 7 7.98 7.98 Oracle 8 7.98 7.98 Oracle 9 7.98 7.98 Oracle 10 7.98 7.98 Oracle 11 6.06 6.06 Oracle 12 5.09 5.09 Oracle 13 5.75 5.75 Oracle 14 5.75 5.75 Oracle 15 4.91 4.91 Oracle 16 6.31 6.31 Oracle 17 6.31 6.31 Oracle 18 6.31 6.31 Oracle 19 7.41 7.41 Oracle 20 7.41 7.41 Oracle 21 6.02 6.02 Oracle 22 6.02 6.02 Oracle 23 10.09 10.09 Oracle 24 13.87 13.87 Oracle 25 13.87 13.87 Oracle 26 13.87 13.87 Oracle 27 13.87 13.87 Oracle 28 13.87 13.87 Oracle 29 13.87 13.87 Oracle 30 13.87 13.87 Oracle 31 13.87 13.87 Oracle 32 13.87 13.87 Oracle 33 13.87 13.87 Oracle 34 13.87 13.87 Oracle 35 8.85 8.85 Oracle 36 8.85 8.85 Oracle 37 8.85 8.85 Oracle 38 8.85 8.85 Oracle 39 8.85 8.85 Oracle 40 9.10 9.10 Oracle 41 9.10 9.10 Oracle 42 8.51 8.51 Oracle 43 8.51 8.51 Oracle 44 8.51 8.51 Oracle 45 8.51 8.51 Oracle 46 10.52 10.52 Oracle 47 10.52 10.52 Oracle 48 10.52 10.52 Oracle 49 10.52 10.52 Oracle 50 10.52 10.52 Total 457 457 Average 9.15 9.15 D.8. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH HIGH MAXIMUM COST281 D.8 Complete results of configuration Shattered Document with high maximum cost
The Shattered Documents approach, with Spreading Activation, and a Legacy Docu- ment (the ARM). Using functions Super Page, Choose, Recentre from Kasim2.Activation. Using the model of the ARM on Kasim2.Documents. Nominal parameters.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 15 Maximum session cost = 15.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 1.00
Size0 = 6.08
Size1 = 3.30 Size gain = 1.84
Cost0 = 6.08
Cost1 = 3.30 Cost gain = 1.84 282 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with high max- imum cost
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 0 0 0 9 0.00 1.00 Oracle 2 30 6 0 0 0 0 6 0.00 1.00 Oracle 3 40 8 0 0 0 0 8 0.00 1.00 Oracle 4 46 7 0 0 0 0 7 0.00 1.00 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 0 0 0 0 7 0.00 1.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 0 0 0 8 0.00 1.00 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 0 0 0 9 0.00 1.00 Oracle 18 269 11 0 0 0 0 11 0.00 1.00 Oracle 19 290 6 0 0 0 0 6 0.00 1.00 Oracle 20 307 6 0 0 0 0 6 0.00 1.00 Oracle 21 366 10 0 0 0 0 10 0.00 1.00 Oracle 22 410 10 0 0 0 0 10 0.00 1.00 Oracle 23 455 15 0 0 0 0 15 0.00 1.00 Oracle 24 508 11 0 0 0 0 11 0.00 1.00 Oracle 25 512 9 0 0 0 0 9 0.00 1.00 Oracle 26 513 15 0 0 0 0 15 0.00 1.00 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 0 0 0 7 0.00 1.00 Oracle 29 542 5 0 0 0 0 5 0.00 1.00 Oracle 30 554 13 0 0 0 0 13 0.00 1.00 Oracle 31 601 11 0 0 0 0 11 0.00 1.00 Oracle 32 606 14 0 0 0 0 14 0.00 1.00 Oracle 33 616 12 0 0 0 0 12 0.00 1.00 Oracle 34 649 8 0 0 0 0 8 0.00 1.00 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 0 0 0 11 0.00 1.00 Oracle 39 786 7 0 0 0 0 7 0.00 1.00 Oracle 40 840 14 0 0 0 0 14 0.00 1.00 Oracle 41 846 8 0 0 0 0 8 0.00 1.00 Oracle 42 883 1 0 0 0 0 1 0.00 1.00 Oracle 43 890 10 0 0 0 0 10 0.00 1.00 Oracle 44 897 6 0 0 0 0 6 0.00 1.00 Oracle 45 899 9 0 0 0 0 9 0.00 1.00 Oracle 46 929 10 0 0 0 0 10 0.00 1.00 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 0 0 0 0 13 0.00 1.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 0 0 0 9 0.00 1.00 Total ··· 500 0 0 0 0 500 0.00 50.00 Average ··· 10.00 0.00 0.00 0.00 0.00 10.00 0.00 1.00 Micro avg ····················· 0.00 1.00 D.8. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH HIGH MAXIMUM COST283
Duration Results of configuration Shattered Document with high maximum cost
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 6.70 0.63 1.16 0.74 1.16 0.65 0.52 0.55 1.80 Oracle 2 4.84 0.59 1.36 0.80 1.36 0.66 0.70 0.48 2.05 Oracle 3 6.32 0.63 1.47 0.79 1.47 0.67 0.79 0.46 2.17 Oracle 4 5.26 0.60 1.39 0.75 1.39 0.70 0.70 0.50 2.01 Oracle 5 4.58 0.50 0.78 0.57 0.78 0.52 0.26 0.67 1.49 Oracle 6 7.42 0.60 1.06 0.67 1.06 0.60 0.46 0.57 1.76 Oracle 7 7.07 0.37 1.59 0.64 1.27 0.38 0.90 0.30 3.41 Oracle 8 12.40 0.94 1.36 1.13 0.94 1.20 -0.27 1.29 0.77 Oracle 9 4.39 0.37 1.57 0.63 1.57 0.38 1.20 0.23 4.23 Oracle 10 9.30 0.35 0.88 0.47 0.78 0.37 0.41 0.47 2.12 Oracle 11 7.94 0.59 0.93 0.66 0.93 0.62 0.31 0.66 1.51 Oracle 12 8.45 0.60 0.95 0.65 0.95 0.61 0.34 0.65 1.55 Oracle 13 11.32 0.60 1.04 0.76 0.98 0.60 0.38 0.62 1.63 Oracle 14 7.16 0.55 0.98 0.65 0.98 0.55 0.42 0.57 1.75 Oracle 15 3.31 0.36 0.59 0.41 0.59 0.41 0.18 0.70 1.43 Oracle 16 4.54 0.60 0.98 0.76 0.98 0.63 0.34 0.65 1.55 Oracle 17 6.69 0.57 1.23 0.74 1.23 0.61 0.63 0.49 2.03 Oracle 18 8.08 0.61 1.03 0.73 1.03 0.78 0.24 0.77 1.31 Oracle 19 5.48 0.88 0.93 0.91 0.90 0.88 0.02 0.98 1.02 Oracle 20 6.28 0.84 1.93 1.05 1.93 0.85 1.08 0.45 2.26 Oracle 21 10.52 0.86 2.40 1.05 2.40 0.86 1.54 0.36 2.79 Oracle 22 9.91 0.83 1.89 0.99 1.89 0.91 0.98 0.48 2.08 Oracle 23 10.96 0.59 1.54 0.73 1.33 0.64 0.69 0.48 2.07 Oracle 24 7.91 0.59 1.44 0.72 1.44 0.65 0.79 0.45 2.22 Oracle 25 8.84 0.59 2.46 0.98 1.99 0.60 1.39 0.30 3.31 Oracle 26 9.86 0.35 2.16 0.66 2.16 0.40 1.76 0.19 5.37 Oracle 27 7.70 0.38 0.67 0.59 0.67 0.39 0.28 0.59 1.71 Oracle 28 7.44 0.68 1.70 1.06 1.31 0.68 0.63 0.52 1.94 Oracle 29 4.96 0.61 2.39 0.99 2.39 0.61 1.78 0.26 3.93 Oracle 30 7.01 0.36 1.01 0.54 1.01 0.55 0.45 0.55 1.81 Oracle 31 7.94 0.57 1.61 0.72 1.61 0.60 1.02 0.38 2.70 Oracle 32 9.90 0.59 1.48 0.70 1.48 0.62 0.86 0.42 2.39 Oracle 33 9.32 0.60 2.37 0.77 2.37 0.61 1.75 0.26 3.87 Oracle 34 6.93 0.59 2.29 0.87 2.29 0.63 1.66 0.27 3.63 Oracle 35 6.45 0.37 1.70 0.65 1.70 0.38 1.32 0.22 4.54 Oracle 36 8.55 0.60 0.88 0.66 0.61 0.62 -0.01 1.02 0.98 Oracle 37 6.57 0.36 1.34 0.59 1.34 0.36 0.98 0.27 3.73 Oracle 38 6.49 0.36 1.41 0.59 1.41 0.42 0.99 0.30 3.35 Oracle 39 5.72 0.59 1.90 0.82 1.90 0.63 1.27 0.34 3.00 Oracle 40 13.59 0.84 2.06 0.97 2.06 0.87 1.20 0.42 2.38 Oracle 41 7.88 0.88 1.30 0.98 1.30 1.03 0.27 0.80 1.26 Oracle 42 1.61 1.61 1.61 1.61 1.61 1.61 0.00 1.00 1.00 Oracle 43 11.63 0.90 2.43 1.16 2.43 1.20 1.23 0.49 2.04 Oracle 44 6.76 0.83 1.39 1.13 1.39 1.25 0.14 0.90 1.11 Oracle 45 8.87 0.84 1.23 0.98 0.93 1.15 -0.22 1.23 0.81 Oracle 46 7.13 0.36 1.80 0.71 1.80 0.84 0.96 0.47 2.15 Oracle 47 7.84 0.38 1.51 0.78 1.51 0.87 0.63 0.58 1.73 Oracle 48 10.38 0.60 1.41 0.80 1.28 0.87 0.41 0.68 1.48 Oracle 49 7.06 0.59 0.85 0.70 0.62 0.85 -0.23 1.38 0.73 Oracle 50 19.46 0.63 13.44 2.16 13.44 0.83 12.60 0.06 16.17 Total 392.71 30.71 84.88 41.23 81.95 35.23 46.72 27.69 124.08 Average 7.85 0.62 1.70 0.83 1.64 0.70 0.94 0.55 2.48 Micro avg ····················· 0.43 2.33 284 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with high maximum cost
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 1.16 0.69 0.73 0.90 0.63 0.63 0.66 0.64 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 Oracle 2 1.36 0.91 0.67 0.59 0.64 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 Oracle 3 1.47 0.95 0.63 0.63 0.68 0.66 0.63 0.67 0.67 0.67 0.67 0.67 0.67 0.67 0.67 0.67 0.67 0.67 0.67 0.67 Oracle 4 1.39 0.66 0.61 0.60 0.63 0.66 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70 Oracle 5 0.78 0.68 0.53 0.53 0.50 0.51 0.52 0.52 0.52 0.52 0.52 0.52 0.52 0.52 0.52 0.52 0.52 0.52 0.52 0.52 Oracle 6 1.06 0.67 0.63 0.63 0.62 0.66 0.63 0.66 0.63 0.62 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 Oracle 7 1.27 1.59 0.67 0.66 0.64 0.38 0.38 0.37 0.39 0.37 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 Oracle 8 0.94 1.36 1.07 1.09 1.12 1.13 1.05 1.11 1.19 1.16 1.20 1.20 1.20 1.20 1.20 1.20 1.20 1.20 1.20 1.20 Oracle 9 1.57 0.57 0.37 0.38 0.57 0.55 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 Oracle 10 0.78 0.61 0.67 0.88 0.41 0.35 0.37 0.47 0.37 0.37 0.38 0.38 0.38 0.37 0.38 0.53 0.35 0.46 0.43 0.37 Oracle 11 0.93 0.63 0.60 0.64 0.64 0.72 0.68 0.65 0.61 0.61 0.59 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 12 0.95 0.61 0.63 0.62 0.60 0.62 0.62 0.62 0.67 0.66 0.61 0.65 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 13 0.98 1.04 0.94 0.88 0.91 0.63 0.68 0.70 0.63 0.65 0.73 0.69 0.62 0.64 0.60 0.60 0.60 0.60 0.60 0.60 Oracle 14 0.98 0.88 0.59 0.57 0.64 0.56 0.72 0.58 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 15 0.59 0.38 0.38 0.41 0.36 0.38 0.38 0.41 0.41 0.41 0.41 0.41 0.41 0.41 0.41 0.41 0.41 0.41 0.41 0.41 Oracle 16 0.98 0.84 0.67 0.60 0.81 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 Oracle 17 1.23 0.87 0.61 0.68 0.57 0.69 0.63 0.80 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 18 1.03 0.84 0.64 0.64 0.72 0.89 0.68 0.63 0.63 0.61 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 0.78 Oracle 19 0.90 0.91 0.92 0.93 0.93 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 Oracle 20 1.93 0.91 0.88 0.87 0.84 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 Oracle 21 2.40 0.91 0.92 0.91 0.91 0.89 0.95 0.88 0.88 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 Oracle 22 1.89 0.89 0.86 0.94 0.83 0.90 0.88 0.86 0.95 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 23 1.33 1.54 0.63 0.59 0.65 0.67 0.59 0.62 0.63 0.63 0.59 0.60 0.63 0.61 0.64 0.64 0.64 0.64 0.64 0.64 Oracle 24 1.44 0.69 0.66 0.59 0.66 0.63 0.63 0.63 0.64 0.70 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 Oracle 25 1.99 2.46 0.67 0.65 0.63 0.63 0.59 0.61 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 Oracle 26 2.16 0.64 0.67 0.36 0.61 0.60 0.63 0.64 0.76 0.69 0.36 0.35 0.36 0.63 0.40 0.40 0.40 0.40 0.40 0.40 Oracle 27 0.67 0.66 0.38 0.63 0.65 0.63 0.63 0.59 0.62 0.60 0.61 0.63 0.39 0.39 0.39 0.39 0.39 0.39 0.39 0.39 Oracle 28 1.31 1.24 0.90 0.89 1.70 0.72 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 Oracle 29 2.39 0.63 0.65 0.69 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 30 1.01 0.51 0.52 0.38 0.63 0.55 0.36 0.55 0.41 0.49 0.52 0.53 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 31 1.61 0.61 0.58 0.61 0.66 0.62 0.88 0.57 0.59 0.59 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 Oracle 32 1.48 0.88 0.63 0.63 0.62 0.66 0.63 0.65 0.66 0.59 0.63 0.61 0.63 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 33 2.37 0.67 0.63 0.63 0.62 0.62 0.69 0.60 0.61 0.66 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 34 2.29 0.88 0.63 0.68 0.59 0.62 0.61 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 Oracle 35 1.70 0.63 0.69 0.40 0.66 0.60 0.41 0.37 0.65 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 0.38 Oracle 36 0.61 0.63 0.60 0.63 0.88 0.86 0.60 0.62 0.61 0.61 0.62 0.66 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 37 1.34 0.39 0.71 0.59 0.64 0.39 0.67 0.71 0.40 0.37 0.36 0.36 0.36 0.36 0.36 0.36 0.36 0.36 0.36 0.36 Oracle 38 1.41 0.65 0.38 0.39 0.59 0.57 0.36 0.59 0.57 0.58 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.42 0.42 Oracle 39 1.90 0.59 0.61 0.70 0.59 0.68 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 Oracle 40 2.06 0.86 0.86 0.88 0.91 0.84 0.98 0.88 0.87 0.91 0.84 0.96 0.88 0.87 0.87 0.87 0.87 0.87 0.87 0.87 Oracle 41 1.30 0.93 0.88 0.98 0.94 0.89 0.91 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 42 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 Oracle 43 2.43 1.17 0.91 0.91 0.90 1.27 0.95 0.94 0.96 1.20 1.20 1.20 1.20 1.20 1.20 1.20 1.20 1.20 1.20 1.20 Oracle 44 1.39 1.23 0.88 0.83 1.17 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 1.25 Oracle 45 0.93 0.86 1.23 0.87 0.87 0.95 0.84 1.18 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 1.15 Oracle 46 1.80 0.44 0.36 0.64 0.69 0.64 0.39 0.38 0.96 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 0.84 Oracle 47 1.51 0.66 0.66 0.38 0.62 0.38 0.88 0.94 0.95 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 Oracle 48 1.28 1.41 0.69 0.62 0.62 0.62 0.60 0.70 0.84 0.61 0.93 0.60 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 Oracle 49 0.62 0.62 0.67 0.60 0.84 0.81 0.84 0.63 0.59 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 Oracle 50 13.44 0.63 0.63 0.68 0.66 0.86 0.88 0.88 0.83 0.83 0.83 0.83 0.83 0.83 0.83 0.83 0.83 0.83 0.83 0.83 Total 82 43 35 35 37 36 35 35 36 36 35 35 35 35 35 35 35 35 35 35 Average 1.64 0.86 0.70 0.70 0.74 0.71 0.70 0.71 0.72 0.71 0.71 0.70 0.70 0.71 0.70 0.71 0.70 0.70 0.70 0.70 D.8. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH HIGH MAXIMUM COST285
Size Results of configuration Shattered Document with high maximum cost
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 30 3 5 3.34 5 3 2 0.60 1.66 Oracle 2 22 3 6 3.66 6 3 3 0.50 2.00 Oracle 3 27 3 5 3.38 5 3 2 0.60 1.66 Oracle 4 24 3 6 3.43 6 3 3 0.50 2.00 Oracle 5 25 3 4 3.13 4 3 1 0.75 1.34 Oracle 6 35 3 5 3.18 5 3 2 0.60 1.66 Oracle 7 32 2 6 2.91 5 2 3 0.40 2.50 Oracle 8 55 4 6 5.00 4 5 -1 1.25 0.80 Oracle 9 22 2 7 3.14 7 2 5 0.29 3.50 Oracle 10 47 2 4 2.35 4 2 2 0.50 2.00 Oracle 11 37 3 4 3.09 4 3 1 0.75 1.34 Oracle 12 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 13 50 3 4 3.34 4 3 1 0.75 1.34 Oracle 14 35 3 4 3.18 4 3 1 0.75 1.34 Oracle 15 17 2 3 2.13 3 2 1 0.66 1.50 Oracle 16 21 3 4 3.50 4 3 1 0.75 1.34 Oracle 17 31 3 5 3.45 5 3 2 0.60 1.66 Oracle 18 38 3 5 3.45 5 4 1 0.80 1.25 Oracle 19 24 4 4 4.00 4 4 0 1.00 1.00 Oracle 20 28 4 8 4.66 8 4 4 0.50 2.00 Oracle 21 45 4 9 4.50 9 4 5 0.45 2.25 Oracle 22 44 4 8 4.40 8 4 4 0.50 2.00 Oracle 23 52 3 7 3.47 6 3 3 0.50 2.00 Oracle 24 36 3 6 3.27 6 3 3 0.50 2.00 Oracle 25 40 3 10 4.45 9 3 6 0.34 3.00 Oracle 26 46 2 9 3.07 9 2 7 0.22 4.50 Oracle 27 37 2 3 2.84 3 2 1 0.66 1.50 Oracle 28 33 3 7 4.71 6 3 3 0.50 2.00 Oracle 29 22 3 10 4.40 10 3 7 0.30 3.34 Oracle 30 38 2 5 2.92 5 3 2 0.60 1.66 Oracle 31 38 3 7 3.45 7 3 4 0.43 2.34 Oracle 32 47 3 7 3.36 7 3 4 0.43 2.34 Oracle 33 43 3 10 3.59 10 3 7 0.30 3.34 Oracle 34 32 3 10 4.00 10 3 7 0.30 3.34 Oracle 35 30 2 7 3.00 7 2 5 0.29 3.50 Oracle 36 41 3 4 3.16 3 3 0 1.00 1.00 Oracle 37 31 2 6 2.82 6 2 4 0.34 3.00 Oracle 38 32 2 6 2.91 6 2 4 0.34 3.00 Oracle 39 26 3 8 3.71 8 3 5 0.38 2.66 Oracle 40 61 4 9 4.36 9 4 5 0.45 2.25 Oracle 41 34 4 6 4.25 6 4 2 0.66 1.50 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 48 4 9 4.80 9 5 4 0.55 1.80 Oracle 44 29 4 6 4.84 6 5 1 0.84 1.20 Oracle 45 39 4 5 4.34 4 5 -1 1.25 0.80 Oracle 46 32 2 7 3.20 7 4 3 0.57 1.75 Oracle 47 35 2 6 3.50 6 4 2 0.66 1.50 Oracle 48 48 3 6 3.70 6 4 2 0.66 1.50 Oracle 49 34 3 4 3.40 3 4 -1 1.34 0.75 Oracle 50 38 3 10 4.22 10 4 6 0.40 2.50 Total 1758 152 313 182.99 304 165 139 30.04 98.52 Average 35.16 3.04 6.26 3.66 6.08 3.30 2.78 0.60 1.97 Micro avg ····················· 0.54 1.84 286 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with high maximum cost
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 5 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 2 6 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 3 5 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 4 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 5 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 6 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 7 5 6 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 8 4 6 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 9 7 3 2 2 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 10 4 3 3 4 2 2 2 2 2 2 2 2 2 2 2 3 2 2 2 2 Oracle 11 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 12 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 13 4 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 14 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 15 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 16 4 4 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 17 5 4 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 18 5 4 3 3 3 4 3 3 3 3 4 4 4 4 4 4 4 4 4 4 Oracle 19 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 20 8 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 21 9 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 22 8 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 23 6 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 24 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 25 9 10 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 26 9 3 3 2 3 3 3 3 3 3 2 2 2 3 2 2 2 2 2 2 Oracle 27 3 3 2 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 Oracle 28 6 6 4 4 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 29 10 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 30 5 3 3 2 3 3 2 3 2 3 3 3 3 3 3 3 3 3 3 3 Oracle 31 7 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 32 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 33 10 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 34 10 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 35 7 3 3 2 3 3 2 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 36 3 3 3 3 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 37 6 2 3 3 3 2 3 3 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 38 6 3 2 2 3 3 2 3 3 3 2 2 2 2 2 2 2 2 2 2 Oracle 39 8 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 40 9 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 41 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 9 5 4 4 4 5 4 4 4 5 5 5 5 5 5 5 5 5 5 5 Oracle 44 6 5 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 45 4 4 5 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 46 7 2 2 3 3 3 2 2 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 47 6 3 3 2 3 2 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 48 6 6 3 3 3 3 3 3 4 3 4 3 4 4 4 4 4 4 4 4 Oracle 49 3 3 3 3 4 4 4 3 3 4 4 4 4 4 4 4 4 4 4 4 Oracle 50 10 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Total 304 194 163 162 172 166 162 164 165 166 166 165 165 166 165 166 165 165 165 165 Average 6.08 3.88 3.26 3.24 3.44 3.32 3.24 3.28 3.30 3.32 3.32 3.30 3.30 3.32 3.30 3.32 3.30 3.30 3.30 3.30 D.8. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH HIGH MAXIMUM COST287
Cost Results of configuration Shattered Document with high maximum cost
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 30.00 3.00 5.00 3.34 5.00 3.00 2.00 0.60 1.66 Oracle 2 22.00 3.00 6.00 3.66 6.00 3.00 3.00 0.50 2.00 Oracle 3 27.00 3.00 5.00 3.38 5.00 3.00 2.00 0.60 1.66 Oracle 4 24.00 3.00 6.00 3.43 6.00 3.00 3.00 0.50 2.00 Oracle 5 25.00 3.00 4.00 3.13 4.00 3.00 1.00 0.75 1.34 Oracle 6 35.00 3.00 5.00 3.18 5.00 3.00 2.00 0.60 1.66 Oracle 7 32.00 2.00 6.00 2.91 5.00 2.00 3.00 0.40 2.50 Oracle 8 55.00 4.00 6.00 5.00 4.00 5.00 -1.00 1.25 0.80 Oracle 9 22.00 2.00 7.00 3.14 7.00 2.00 5.00 0.29 3.50 Oracle 10 47.00 2.00 4.00 2.35 4.00 2.00 2.00 0.50 2.00 Oracle 11 37.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 12 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 13 50.00 3.00 4.00 3.34 4.00 3.00 1.00 0.75 1.34 Oracle 14 35.00 3.00 4.00 3.18 4.00 3.00 1.00 0.75 1.34 Oracle 15 17.00 2.00 3.00 2.13 3.00 2.00 1.00 0.66 1.50 Oracle 16 21.00 3.00 4.00 3.50 4.00 3.00 1.00 0.75 1.34 Oracle 17 31.00 3.00 5.00 3.45 5.00 3.00 2.00 0.60 1.66 Oracle 18 38.00 3.00 5.00 3.45 5.00 4.00 1.00 0.80 1.25 Oracle 19 24.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 20 28.00 4.00 8.00 4.66 8.00 4.00 4.00 0.50 2.00 Oracle 21 45.00 4.00 9.00 4.50 9.00 4.00 5.00 0.45 2.25 Oracle 22 44.00 4.00 8.00 4.40 8.00 4.00 4.00 0.50 2.00 Oracle 23 52.00 3.00 7.00 3.47 6.00 3.00 3.00 0.50 2.00 Oracle 24 36.00 3.00 6.00 3.27 6.00 3.00 3.00 0.50 2.00 Oracle 25 40.00 3.00 10.00 4.45 9.00 3.00 6.00 0.34 3.00 Oracle 26 46.00 2.00 9.00 3.07 9.00 2.00 7.00 0.22 4.50 Oracle 27 37.00 2.00 3.00 2.84 3.00 2.00 1.00 0.66 1.50 Oracle 28 33.00 3.00 7.00 4.71 6.00 3.00 3.00 0.50 2.00 Oracle 29 22.00 3.00 10.00 4.40 10.00 3.00 7.00 0.30 3.34 Oracle 30 38.00 2.00 5.00 2.92 5.00 3.00 2.00 0.60 1.66 Oracle 31 38.00 3.00 7.00 3.45 7.00 3.00 4.00 0.43 2.34 Oracle 32 47.00 3.00 7.00 3.36 7.00 3.00 4.00 0.43 2.34 Oracle 33 43.00 3.00 10.00 3.59 10.00 3.00 7.00 0.30 3.34 Oracle 34 32.00 3.00 10.00 4.00 10.00 3.00 7.00 0.30 3.34 Oracle 35 30.00 2.00 7.00 3.00 7.00 2.00 5.00 0.29 3.50 Oracle 36 41.00 3.00 4.00 3.16 3.00 3.00 0.00 1.00 1.00 Oracle 37 31.00 2.00 6.00 2.82 6.00 2.00 4.00 0.34 3.00 Oracle 38 32.00 2.00 6.00 2.91 6.00 2.00 4.00 0.34 3.00 Oracle 39 26.00 3.00 8.00 3.71 8.00 3.00 5.00 0.38 2.66 Oracle 40 61.00 4.00 9.00 4.36 9.00 4.00 5.00 0.45 2.25 Oracle 41 34.00 4.00 6.00 4.25 6.00 4.00 2.00 0.66 1.50 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 48.00 4.00 9.00 4.80 9.00 5.00 4.00 0.55 1.80 Oracle 44 29.00 4.00 6.00 4.84 6.00 5.00 1.00 0.84 1.20 Oracle 45 39.00 4.00 5.00 4.34 4.00 5.00 -1.00 1.25 0.80 Oracle 46 32.00 2.00 7.00 3.20 7.00 4.00 3.00 0.57 1.75 Oracle 47 35.00 2.00 6.00 3.50 6.00 4.00 2.00 0.66 1.50 Oracle 48 48.00 3.00 6.00 3.70 6.00 4.00 2.00 0.66 1.50 Oracle 49 34.00 3.00 4.00 3.40 3.00 4.00 -1.00 1.34 0.75 Oracle 50 38.00 3.00 10.00 4.22 10.00 4.00 6.00 0.40 2.50 Total 1758.00 152.00 313.00 182.99 304.00 165.00 139.00 30.04 98.52 Average 35.16 3.04 6.26 3.66 6.08 3.30 2.78 0.60 1.97 Micro avg ····················· 0.54 1.84 288 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with high maximum cost
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 5.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 2 6.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 3 5.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 4 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 5 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 6 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 7 5.00 6.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 8 4.00 6.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 9 7.00 3.00 2.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 10 4.00 3.00 3.00 4.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 Oracle 11 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 12 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 13 4.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 14 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 15 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 16 4.00 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 17 5.00 4.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 18 5.00 4.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 19 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 20 8.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 21 9.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 22 8.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 23 6.00 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 24 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 25 9.00 10.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 26 9.00 3.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 27 3.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 28 6.00 6.00 4.00 4.00 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 29 10.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 30 5.00 3.00 3.00 2.00 3.00 3.00 2.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 31 7.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 32 7.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 33 10.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 34 10.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 35 7.00 3.00 3.00 2.00 3.00 3.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 36 3.00 3.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 37 6.00 2.00 3.00 3.00 3.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 38 6.00 3.00 2.00 2.00 3.00 3.00 2.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 39 8.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 40 9.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 41 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 9.00 5.00 4.00 4.00 4.00 5.00 4.00 4.00 4.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 44 6.00 5.00 4.00 4.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 45 4.00 4.00 5.00 4.00 4.00 4.00 4.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 46 7.00 2.00 2.00 3.00 3.00 3.00 2.00 2.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 47 6.00 3.00 3.00 2.00 3.00 2.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 48 6.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 49 3.00 3.00 3.00 3.00 4.00 4.00 4.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 50 10.00 3.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Total 304 194 163 162 172 166 162 164 165 166 166 165 165 166 165 166 165 165 165 165 Average 6.08 3.88 3.26 3.24 3.44 3.32 3.24 3.28 3.30 3.32 3.32 3.30 3.30 3.32 3.30 3.32 3.30 3.30 3.30 3.30 D.9. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL WITH HIGH MAXIMUM COST289 D.9 Complete results of configuration Structural with high maximum cost
The legacy document in its original structure, without adaptation. Using special functions Start, Choose, Recentre from Kasim2.Structural. Using the model of the ARM on Kasim2.Documents.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 100 Page size (number of items) = 10 Quantity of expanded items = 0 Update travel matrix strategy = False Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 15.00 Weight of Next links = 0.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 1.00
Size0 = 3.48
Size1 = 3.48 Size gain = 1.00
Cost0 = 10.30
Cost1 = 10.30 Cost gain = 1.00 290 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Structural with high maximum cost
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 2 0 0 0 0 2 0.00 1.00 Oracle 2 30 2 0 0 0 0 2 0.00 1.00 Oracle 3 40 2 0 0 0 0 2 0.00 1.00 Oracle 4 46 2 0 0 0 0 2 0.00 1.00 Oracle 5 56 2 0 0 0 0 2 0.00 1.00 Oracle 6 85 2 0 0 0 0 2 0.00 1.00 Oracle 7 108 2 0 0 0 0 2 0.00 1.00 Oracle 8 125 2 0 0 0 0 2 0.00 1.00 Oracle 9 128 2 0 0 0 0 2 0.00 1.00 Oracle 10 130 2 0 0 0 0 2 0.00 1.00 Oracle 11 138 2 0 0 0 0 2 0.00 1.00 Oracle 12 157 2 0 0 0 0 2 0.00 1.00 Oracle 13 171 2 0 0 0 0 2 0.00 1.00 Oracle 14 173 2 0 0 0 0 2 0.00 1.00 Oracle 15 241 2 0 0 0 0 2 0.00 1.00 Oracle 16 265 2 0 0 0 0 2 0.00 1.00 Oracle 17 266 2 0 0 0 0 2 0.00 1.00 Oracle 18 269 2 0 0 0 0 2 0.00 1.00 Oracle 19 290 2 0 0 0 0 2 0.00 1.00 Oracle 20 307 2 0 0 0 0 2 0.00 1.00 Oracle 21 366 2 0 0 0 0 2 0.00 1.00 Oracle 22 410 2 0 0 0 0 2 0.00 1.00 Oracle 23 455 2 0 0 0 0 2 0.00 1.00 Oracle 24 508 2 0 0 0 0 2 0.00 1.00 Oracle 25 512 2 0 0 0 0 2 0.00 1.00 Oracle 26 513 2 0 0 0 0 2 0.00 1.00 Oracle 27 516 2 0 0 0 0 2 0.00 1.00 Oracle 28 521 2 0 0 0 0 2 0.00 1.00 Oracle 29 542 2 0 0 0 0 2 0.00 1.00 Oracle 30 554 2 0 0 0 0 2 0.00 1.00 Oracle 31 601 2 0 0 0 0 2 0.00 1.00 Oracle 32 606 2 0 0 0 0 2 0.00 1.00 Oracle 33 616 2 0 0 0 0 2 0.00 1.00 Oracle 34 649 2 0 0 0 0 2 0.00 1.00 Oracle 35 723 2 0 0 0 0 2 0.00 1.00 Oracle 36 725 2 0 0 0 0 2 0.00 1.00 Oracle 37 738 2 0 0 0 0 2 0.00 1.00 Oracle 38 741 2 0 0 0 0 2 0.00 1.00 Oracle 39 786 2 0 0 0 0 2 0.00 1.00 Oracle 40 840 2 0 0 0 0 2 0.00 1.00 Oracle 41 846 2 0 0 0 0 2 0.00 1.00 Oracle 42 883 2 0 0 0 0 2 0.00 1.00 Oracle 43 890 2 0 0 0 0 2 0.00 1.00 Oracle 44 897 2 0 0 0 0 2 0.00 1.00 Oracle 45 899 2 0 0 0 0 2 0.00 1.00 Oracle 46 929 2 0 0 0 0 2 0.00 1.00 Oracle 47 936 2 0 0 0 0 2 0.00 1.00 Oracle 48 939 2 0 0 0 0 2 0.00 1.00 Oracle 49 944 2 0 0 0 0 2 0.00 1.00 Oracle 50 999 2 0 0 0 0 2 0.00 1.00 Total ··· 100 0 0 0 0 100 0.00 50.00 Average ··· 2.00 0.00 0.00 0.00 0.00 2.00 0.00 1.00 Micro avg ····················· 0.00 1.00 D.9. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL WITH HIGH MAXIMUM COST291
Duration Results of configuration Structural with high maxi- mum cost
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 0.62 0.30 0.31 0.30 0.31 0.30 0.01 0.97 1.04 Oracle 2 0.63 0.30 0.32 0.31 0.30 0.32 -0.02 1.07 0.94 Oracle 3 0.61 0.30 0.30 0.30 0.30 0.30 0.00 0.99 1.01 Oracle 4 0.61 0.30 0.30 0.30 0.30 0.30 -0.01 1.02 0.98 Oracle 5 0.41 0.20 0.21 0.21 0.20 0.21 -0.01 1.05 0.95 Oracle 6 0.63 0.31 0.31 0.31 0.31 0.31 -0.01 1.02 0.98 Oracle 7 0.39 0.20 0.20 0.20 0.20 0.20 0.00 1.01 0.99 Oracle 8 0.43 0.20 0.23 0.21 0.23 0.20 0.03 0.85 1.17 Oracle 9 0.38 0.20 0.20 0.20 0.20 0.20 0.00 1.00 1.00 Oracle 10 0.38 0.20 0.20 0.20 0.20 0.20 0.00 1.01 0.99 Oracle 11 0.40 0.20 0.21 0.20 0.20 0.21 -0.02 1.08 0.93 Oracle 12 0.41 0.20 0.21 0.20 0.21 0.20 0.02 0.92 1.09 Oracle 13 0.63 0.30 0.32 0.31 0.32 0.30 0.02 0.95 1.05 Oracle 14 0.65 0.31 0.34 0.32 0.34 0.31 0.02 0.92 1.08 Oracle 15 0.39 0.19 0.20 0.20 0.20 0.19 0.01 0.98 1.03 Oracle 16 0.40 0.20 0.20 0.20 0.20 0.20 0.01 0.95 1.05 Oracle 17 0.38 0.20 0.20 0.20 0.20 0.20 0.00 1.01 0.99 Oracle 18 0.40 0.20 0.20 0.20 0.20 0.20 -0.01 1.02 0.98 Oracle 19 0.39 0.19 0.20 0.20 0.19 0.20 -0.02 1.08 0.93 Oracle 20 0.63 0.30 0.32 0.31 0.30 0.32 -0.01 1.04 0.96 Oracle 21 0.63 0.30 0.33 0.31 0.33 0.30 0.02 0.95 1.06 Oracle 22 0.63 0.30 0.32 0.31 0.30 0.32 -0.01 1.03 0.97 Oracle 23 0.39 0.20 0.20 0.20 0.20 0.20 0.00 0.98 1.02 Oracle 24 0.40 0.20 0.20 0.20 0.20 0.20 0.02 0.95 1.06 Oracle 25 0.44 0.20 0.23 0.22 0.23 0.20 0.04 0.84 1.19 Oracle 26 0.41 0.20 0.21 0.20 0.21 0.20 0.00 0.98 1.02 Oracle 27 0.39 0.20 0.20 0.20 0.20 0.20 0.00 0.98 1.02 Oracle 28 0.41 0.20 0.21 0.21 0.21 0.20 0.01 0.98 1.02 Oracle 29 0.63 0.31 0.31 0.31 0.31 0.31 0.00 1.01 0.99 Oracle 30 0.42 0.20 0.23 0.21 0.23 0.20 0.02 0.88 1.13 Oracle 31 0.66 0.31 0.34 0.33 0.34 0.31 0.02 0.94 1.06 Oracle 32 0.66 0.30 0.36 0.33 0.36 0.30 0.05 0.85 1.17 Oracle 33 0.62 0.31 0.31 0.31 0.31 0.31 0.00 1.00 1.00 Oracle 34 0.65 0.30 0.34 0.33 0.34 0.30 0.04 0.89 1.13 Oracle 35 0.41 0.20 0.21 0.20 0.21 0.20 0.02 0.91 1.09 Oracle 36 0.41 0.20 0.21 0.20 0.21 0.20 0.02 0.91 1.11 Oracle 37 0.40 0.20 0.20 0.20 0.20 0.20 -0.01 1.04 0.96 Oracle 38 0.41 0.20 0.21 0.20 0.21 0.20 0.02 0.91 1.09 Oracle 39 0.62 0.30 0.30 0.30 0.30 0.30 0.00 1.00 1.00 Oracle 40 0.62 0.30 0.31 0.30 0.30 0.31 -0.01 1.02 0.98 Oracle 41 0.66 0.31 0.34 0.33 0.31 0.34 -0.03 1.11 0.90 Oracle 42 0.43 0.21 0.21 0.21 0.21 0.21 0.00 1.02 0.98 Oracle 43 0.63 0.30 0.34 0.31 0.30 0.34 -0.03 1.11 0.90 Oracle 44 0.62 0.31 0.31 0.31 0.31 0.31 0.00 1.00 1.00 Oracle 45 0.62 0.30 0.32 0.30 0.30 0.32 -0.02 1.05 0.95 Oracle 46 0.41 0.20 0.21 0.21 0.21 0.20 0.02 0.94 1.06 Oracle 47 0.39 0.20 0.20 0.20 0.20 0.20 0.00 0.99 1.01 Oracle 48 0.41 0.20 0.21 0.20 0.21 0.20 0.02 0.92 1.09 Oracle 49 0.41 0.20 0.21 0.20 0.21 0.20 0.02 0.90 1.12 Oracle 50 0.62 0.30 0.31 0.31 0.30 0.31 -0.01 1.02 0.98 Total 25.10 12.22 12.88 12.55 12.67 12.43 0.24 49.03 51.21 Average 0.50 0.24 0.26 0.25 0.25 0.25 0.01 0.98 1.02 Micro avg ····················· 0.98 1.02 292 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Structural with high max- imum cost
1 2 Oracle 1 0.31 0.30 Oracle 2 0.30 0.32 Oracle 3 0.30 0.30 Oracle 4 0.30 0.30 Oracle 5 0.20 0.21 Oracle 6 0.31 0.31 Oracle 7 0.20 0.20 Oracle 8 0.23 0.20 Oracle 9 0.20 0.20 Oracle 10 0.20 0.20 Oracle 11 0.20 0.21 Oracle 12 0.21 0.20 Oracle 13 0.32 0.30 Oracle 14 0.34 0.31 Oracle 15 0.20 0.19 Oracle 16 0.20 0.20 Oracle 17 0.20 0.20 Oracle 18 0.20 0.20 Oracle 19 0.19 0.20 Oracle 20 0.30 0.32 Oracle 21 0.33 0.30 Oracle 22 0.30 0.32 Oracle 23 0.20 0.20 Oracle 24 0.20 0.20 Oracle 25 0.23 0.20 Oracle 26 0.21 0.20 Oracle 27 0.20 0.20 Oracle 28 0.21 0.20 Oracle 29 0.31 0.31 Oracle 30 0.23 0.20 Oracle 31 0.34 0.31 Oracle 32 0.36 0.30 Oracle 33 0.31 0.31 Oracle 34 0.34 0.30 Oracle 35 0.21 0.20 Oracle 36 0.21 0.20 Oracle 37 0.20 0.20 Oracle 38 0.21 0.20 Oracle 39 0.30 0.30 Oracle 40 0.30 0.31 Oracle 41 0.31 0.34 Oracle 42 0.21 0.21 Oracle 43 0.30 0.34 Oracle 44 0.31 0.31 Oracle 45 0.30 0.32 Oracle 46 0.21 0.20 Oracle 47 0.20 0.20 Oracle 48 0.21 0.20 Oracle 49 0.21 0.20 Oracle 50 0.30 0.31 Total 13 12 Average 0.25 0.25 D.9. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL WITH HIGH MAXIMUM COST293
Size Results of configuration Structural with high maximum cost
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 2 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 3 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 4 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 5 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 6 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 7 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 8 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 9 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 10 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 11 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 12 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 13 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 14 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 15 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 16 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 17 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 18 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 19 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 20 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 21 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 22 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 23 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 24 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 25 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 26 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 27 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 28 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 29 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 30 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 31 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 32 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 33 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 34 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 35 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 36 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 37 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 38 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 39 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 40 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 41 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 42 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 43 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 44 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 45 8 4 4 4.00 4 4 0 1.00 1.00 Oracle 46 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 47 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 48 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 49 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 50 8 4 4 4.00 4 4 0 1.00 1.00 Total 348 174 174 174.00 174 174 0 50.00 50.00 Average 6.96 3.48 3.48 3.48 3.48 3.48 0.00 1.00 1.00 Micro avg ····················· 1.00 1.00 294 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Structural with high maximum cost
1 2 Oracle 1 4 4 Oracle 2 4 4 Oracle 3 4 4 Oracle 4 4 4 Oracle 5 4 4 Oracle 6 4 4 Oracle 7 3 3 Oracle 8 3 3 Oracle 9 3 3 Oracle 10 3 3 Oracle 11 3 3 Oracle 12 3 3 Oracle 13 4 4 Oracle 14 4 4 Oracle 15 3 3 Oracle 16 3 3 Oracle 17 3 3 Oracle 18 3 3 Oracle 19 3 3 Oracle 20 4 4 Oracle 21 4 4 Oracle 22 4 4 Oracle 23 3 3 Oracle 24 3 3 Oracle 25 3 3 Oracle 26 3 3 Oracle 27 3 3 Oracle 28 3 3 Oracle 29 4 4 Oracle 30 4 4 Oracle 31 4 4 Oracle 32 4 4 Oracle 33 4 4 Oracle 34 4 4 Oracle 35 3 3 Oracle 36 3 3 Oracle 37 3 3 Oracle 38 3 3 Oracle 39 4 4 Oracle 40 4 4 Oracle 41 4 4 Oracle 42 3 3 Oracle 43 4 4 Oracle 44 4 4 Oracle 45 4 4 Oracle 46 3 3 Oracle 47 3 3 Oracle 48 3 3 Oracle 49 3 3 Oracle 50 4 4 Total 174 174 Average 3.48 3.48 D.9. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL WITH HIGH MAXIMUM COST295
Cost Results of configuration Structural with high maximum cost
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 11.05 5.53 5.53 5.53 5.53 5.53 0.00 1.00 1.00 Oracle 2 15.14 7.57 7.57 7.57 7.57 7.57 0.00 1.00 1.00 Oracle 3 13.86 6.93 6.93 6.93 6.93 6.93 0.00 1.00 1.00 Oracle 4 13.86 6.93 6.93 6.93 6.93 6.93 0.00 1.00 1.00 Oracle 5 14.83 7.41 7.41 7.41 7.41 7.41 0.00 1.00 1.00 Oracle 6 12.01 6.01 6.01 6.01 6.01 6.01 0.00 1.00 1.00 Oracle 7 15.96 7.98 7.98 7.98 7.98 7.98 0.00 1.00 1.00 Oracle 8 15.96 7.98 7.98 7.98 7.98 7.98 0.00 1.00 1.00 Oracle 9 15.96 7.98 7.98 7.98 7.98 7.98 0.00 1.00 1.00 Oracle 10 15.96 7.98 7.98 7.98 7.98 7.98 0.00 1.00 1.00 Oracle 11 12.12 6.06 6.06 6.06 6.06 6.06 0.00 1.00 1.00 Oracle 12 10.20 5.09 5.09 5.09 5.09 5.09 0.00 1.00 1.00 Oracle 13 11.49 5.75 5.75 5.75 5.75 5.75 0.00 1.00 1.00 Oracle 14 11.49 5.75 5.75 5.75 5.75 5.75 0.00 1.00 1.00 Oracle 15 9.82 4.91 4.91 4.91 4.91 4.91 0.00 1.00 1.00 Oracle 16 12.63 6.31 6.31 6.31 6.31 6.31 0.00 1.00 1.00 Oracle 17 12.63 6.31 6.31 6.31 6.31 6.31 0.00 1.00 1.00 Oracle 18 12.63 6.31 6.31 6.31 6.31 6.31 0.00 1.00 1.00 Oracle 19 14.81 7.41 7.41 7.41 7.41 7.41 0.00 1.00 1.00 Oracle 20 21.36 10.68 10.68 10.68 10.68 10.68 0.00 1.00 1.00 Oracle 21 24.71 12.35 12.35 12.35 12.35 12.35 0.00 1.00 1.00 Oracle 22 15.15 7.57 7.57 7.57 7.57 7.57 0.00 1.00 1.00 Oracle 23 20.20 10.09 10.09 10.09 10.09 10.09 0.00 1.00 1.00 Oracle 24 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 25 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 26 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 27 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 28 27.74 13.87 13.87 13.87 13.87 13.87 0.00 1.00 1.00 Oracle 29 32.70 16.35 16.35 16.35 16.35 16.35 0.00 1.00 1.00 Oracle 30 31.72 15.86 15.86 15.86 15.86 15.86 0.00 1.00 1.00 Oracle 31 40.07 20.03 20.03 20.03 20.03 20.03 0.00 1.00 1.00 Oracle 32 40.07 20.03 20.03 20.03 20.03 20.03 0.00 1.00 1.00 Oracle 33 32.39 16.20 16.20 16.20 16.20 16.20 0.00 1.00 1.00 Oracle 34 34.69 17.34 17.34 17.34 17.34 17.34 0.00 1.00 1.00 Oracle 35 17.71 8.85 8.85 8.85 8.85 8.85 0.00 1.00 1.00 Oracle 36 17.71 8.85 8.85 8.85 8.85 8.85 0.00 1.00 1.00 Oracle 37 17.71 8.85 8.85 8.85 8.85 8.85 0.00 1.00 1.00 Oracle 38 17.71 8.85 8.85 8.85 8.85 8.85 0.00 1.00 1.00 Oracle 39 21.08 10.54 10.54 10.54 10.54 10.54 0.00 1.00 1.00 Oracle 40 24.19 12.09 12.09 12.09 12.09 12.09 0.00 1.00 1.00 Oracle 41 24.19 12.09 12.09 12.09 12.09 12.09 0.00 1.00 1.00 Oracle 42 17.02 8.51 8.51 8.51 8.51 8.51 0.00 1.00 1.00 Oracle 43 24.91 12.45 12.45 12.45 12.45 12.45 0.00 1.00 1.00 Oracle 44 24.91 12.45 12.45 12.45 12.45 12.45 0.00 1.00 1.00 Oracle 45 24.91 12.45 12.45 12.45 12.45 12.45 0.00 1.00 1.00 Oracle 46 21.02 10.52 10.52 10.52 10.52 10.52 0.00 1.00 1.00 Oracle 47 21.02 10.52 10.52 10.52 10.52 10.52 0.00 1.00 1.00 Oracle 48 21.02 10.52 10.52 10.52 10.52 10.52 0.00 1.00 1.00 Oracle 49 21.02 10.52 10.52 10.52 10.52 10.52 0.00 1.00 1.00 Oracle 50 29.77 14.88 14.88 14.88 14.88 14.88 0.00 1.00 1.00 Total 1030.05 515.02 515.02 515.02 515.02 515.02 0.00 50.00 50.00 Average 20.60 10.30 10.30 10.30 10.30 10.30 0.00 1.00 1.00 Micro avg ····················· 1.00 1.00 296 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Structural with high maximum cost
1 2 Oracle 1 5.53 5.53 Oracle 2 7.57 7.57 Oracle 3 6.93 6.93 Oracle 4 6.93 6.93 Oracle 5 7.41 7.41 Oracle 6 6.01 6.01 Oracle 7 7.98 7.98 Oracle 8 7.98 7.98 Oracle 9 7.98 7.98 Oracle 10 7.98 7.98 Oracle 11 6.06 6.06 Oracle 12 5.09 5.09 Oracle 13 5.75 5.75 Oracle 14 5.75 5.75 Oracle 15 4.91 4.91 Oracle 16 6.31 6.31 Oracle 17 6.31 6.31 Oracle 18 6.31 6.31 Oracle 19 7.41 7.41 Oracle 20 10.68 10.68 Oracle 21 12.35 12.35 Oracle 22 7.57 7.57 Oracle 23 10.09 10.09 Oracle 24 13.87 13.87 Oracle 25 13.87 13.87 Oracle 26 13.87 13.87 Oracle 27 13.87 13.87 Oracle 28 13.87 13.87 Oracle 29 16.35 16.35 Oracle 30 15.86 15.86 Oracle 31 20.03 20.03 Oracle 32 20.03 20.03 Oracle 33 16.20 16.20 Oracle 34 17.34 17.34 Oracle 35 8.85 8.85 Oracle 36 8.85 8.85 Oracle 37 8.85 8.85 Oracle 38 8.85 8.85 Oracle 39 10.54 10.54 Oracle 40 12.09 12.09 Oracle 41 12.09 12.09 Oracle 42 8.51 8.51 Oracle 43 12.45 12.45 Oracle 44 12.45 12.45 Oracle 45 12.45 12.45 Oracle 46 10.52 10.52 Oracle 47 10.52 10.52 Oracle 48 10.52 10.52 Oracle 49 10.52 10.52 Oracle 50 14.88 14.88 Total 515 515 Average 10.30 10.30 D.10. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH CHILD LINKS ONLY297 D.10 Complete results of configuration Shattered Document with Child links only
Like Shattered Document, but with only Child connections for starting point.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 0.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.89
Size0 = 5.28
Size1 = 3.32 Size gain = 1.59
Cost0 = 5.22
Cost1 = 3.28 Cost gain = 1.59 298 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Child links only
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 0 0 0 9 0.00 1.00 Oracle 2 30 6 0 1 0 1 5 0.16 0.84 Oracle 3 40 8 1 0 0 1 7 0.13 0.88 Oracle 4 46 7 0 0 0 0 7 0.00 1.00 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 7 0 0 7 0 1.00 0.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 0 0 0 8 0.00 1.00 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 0 0 0 9 0.00 1.00 Oracle 18 269 11 0 0 0 0 11 0.00 1.00 Oracle 19 290 6 0 0 0 0 6 0.00 1.00 Oracle 20 307 6 0 0 0 0 6 0.00 1.00 Oracle 21 366 10 0 0 0 0 10 0.00 1.00 Oracle 22 410 10 0 0 0 0 10 0.00 1.00 Oracle 23 455 15 0 0 0 0 15 0.00 1.00 Oracle 24 508 11 0 0 0 0 11 0.00 1.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 0 0 0 15 0.00 1.00 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 2 0 2 5 0.29 0.71 Oracle 29 542 5 0 3 0 3 2 0.60 0.40 Oracle 30 554 13 0 0 0 0 13 0.00 1.00 Oracle 31 601 11 0 0 0 0 11 0.00 1.00 Oracle 32 606 14 0 0 0 0 14 0.00 1.00 Oracle 33 616 12 12 0 0 12 0 1.00 0.00 Oracle 34 649 8 0 8 0 8 0 1.00 0.00 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 0 0 0 11 0.00 1.00 Oracle 39 786 7 0 0 0 0 7 0.00 1.00 Oracle 40 840 14 0 2 0 2 12 0.14 0.86 Oracle 41 846 8 0 0 0 0 8 0.00 1.00 Oracle 42 883 1 0 0 0 0 1 0.00 1.00 Oracle 43 890 10 0 1 0 1 9 0.10 0.90 Oracle 44 897 6 0 0 0 0 6 0.00 1.00 Oracle 45 899 9 0 0 0 0 9 0.00 1.00 Oracle 46 929 10 0 0 0 0 10 0.00 1.00 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 0 0 0 0 13 0.00 1.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 20 35 0 55 445 6.42 43.58 Average ··· 10.00 0.40 0.70 0.00 1.10 8.90 0.13 0.88 Micro avg ····················· 0.11 0.89 D.10. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH CHILD LINKS ONLY299
Duration Results of configuration Shattered Document with Child links only
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 90.76 9.92 10.72 10.09 10.72 9.97 0.75 0.93 1.08 Oracle 2 53.25 2.79 10.82 8.88 2.79 9.88 -7.10 3.55 0.28 Oracle 3 91.43 1.45 30.27 11.43 30.27 9.68 20.59 0.32 3.13 Oracle 4 69.77 9.71 10.40 9.97 10.40 9.73 0.66 0.94 1.07 Oracle 5 10.29 1.27 1.33 1.29 1.33 1.30 0.02 0.98 1.02 Oracle 6 26.34 1.45 11.38 2.40 11.38 1.48 9.89 0.13 7.67 Oracle 7 17.02 0.61 10.85 1.55 10.85 0.61 10.24 0.05 17.73 Oracle 8 11.37 1.02 1.07 1.03 1.02 1.02 0.00 1.00 1.00 Oracle 9 211.88 30.14 30.37 30.27 30.37 30.25 0.13 0.99 1.01 Oracle 10 21.26 0.61 9.55 1.06 9.55 0.61 8.94 0.06 15.59 Oracle 11 21.31 1.02 9.98 1.77 9.98 1.05 8.93 0.10 9.53 Oracle 12 22.51 1.02 9.95 1.73 9.95 1.02 8.92 0.10 9.70 Oracle 13 52.52 1.03 10.43 3.50 10.43 1.06 9.37 0.10 9.80 Oracle 14 29.64 1.03 10.09 2.70 10.05 1.03 9.02 0.10 9.78 Oracle 15 17.23 1.02 10.00 2.16 10.00 1.08 8.91 0.11 9.25 Oracle 16 14.73 1.02 9.56 2.45 9.56 1.03 8.52 0.11 9.25 Oracle 17 13.37 1.44 1.77 1.48 1.77 1.45 0.33 0.81 1.23 Oracle 18 95.79 0.61 10.02 8.71 10.02 0.61 9.41 0.06 16.41 Oracle 19 7.15 1.03 1.48 1.19 1.03 1.03 0.01 0.99 1.01 Oracle 20 7.95 1.02 2.30 1.33 2.30 1.05 1.24 0.46 2.17 Oracle 21 16.56 1.41 2.28 1.66 2.28 1.41 0.86 0.63 1.61 Oracle 22 20.80 1.03 11.31 2.08 11.31 1.04 10.27 0.09 10.87 Oracle 23 24.98 0.61 11.30 1.66 11.30 1.02 10.28 0.09 11.02 Oracle 24 105.35 9.37 10.85 9.58 10.85 9.47 1.38 0.88 1.15 Oracle 25 23.95 2.64 2.70 2.66 2.64 2.65 0.00 1.00 1.00 Oracle 26 107.83 0.61 10.46 7.19 10.46 0.61 9.85 0.05 17.15 Oracle 27 97.58 0.61 9.75 7.51 9.65 9.51 0.14 0.98 1.02 Oracle 28 38.05 1.42 10.98 5.44 2.69 9.44 -6.75 3.51 0.28 Oracle 29 20.44 1.02 11.11 4.09 2.75 1.02 1.72 0.38 2.68 Oracle 30 10.70 0.62 1.73 0.82 1.73 0.85 0.88 0.49 2.03 Oracle 31 21.30 0.61 11.43 1.94 11.43 1.02 10.40 0.09 11.13 Oracle 32 23.02 0.61 11.36 1.64 11.36 1.02 10.34 0.09 11.13 Oracle 33 364.02 30.07 30.56 30.34 30.25 30.47 -0.22 1.01 0.99 Oracle 34 21.83 2.67 2.80 2.73 2.80 2.67 0.13 0.95 1.05 Oracle 35 61.20 0.61 10.98 6.12 10.98 0.62 10.37 0.05 17.90 Oracle 36 62.95 0.60 10.50 4.84 10.50 0.61 9.89 0.05 17.15 Oracle 37 52.79 0.61 10.94 4.80 10.94 0.61 10.33 0.05 18.00 Oracle 38 52.98 0.61 10.93 4.82 10.93 0.62 10.31 0.05 17.61 Oracle 39 24.97 2.23 11.41 3.57 11.41 2.24 9.16 0.20 5.09 Oracle 40 27.45 1.02 10.59 1.96 2.73 1.02 1.70 0.38 2.67 Oracle 41 18.09 1.02 10.85 2.26 10.85 1.02 9.83 0.09 10.59 Oracle 42 1.02 1.02 1.02 1.02 1.02 1.02 0.00 1.00 1.00 Oracle 43 15.70 1.02 3.05 1.57 3.05 1.44 1.61 0.47 2.12 Oracle 44 15.45 0.61 10.73 2.58 10.73 0.61 10.11 0.05 17.56 Oracle 45 19.38 0.61 11.38 2.16 11.38 1.09 10.30 0.09 10.46 Oracle 46 42.73 0.61 10.19 4.27 10.19 0.61 9.59 0.06 16.80 Oracle 47 55.59 0.63 10.13 5.55 2.28 0.63 1.65 0.27 3.61 Oracle 48 53.26 0.61 10.09 4.09 10.09 9.48 0.61 0.94 1.06 Oracle 49 51.07 0.61 9.77 5.11 9.71 9.52 0.19 0.98 1.02 Oracle 50 24.46 2.70 2.74 2.72 2.74 2.70 0.05 0.98 1.02 Total 2361.04 137.62 486.27 241.73 444.80 191.02 253.78 27.91 344.45 Average 47.22 2.75 9.73 4.84 8.90 3.82 5.08 0.55 6.89 Micro avg ····················· 0.43 2.33 300 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Child links only
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 10.72 10.10 10.02 9.98 9.98 9.92 10.00 10.07 9.97 9.97 9.97 9.97 9.97 9.97 9.97 9.97 9.97 9.97 9.97 9.97 Oracle 2 2.79 10.82 10.04 9.94 9.78 9.88 9.88 9.88 9.88 9.88 9.88 9.88 9.88 9.88 9.88 9.88 9.88 9.88 9.88 9.88 Oracle 3 30.27 1.45 10.41 10.05 10.06 9.73 9.77 9.68 9.68 9.68 9.68 9.68 9.68 9.68 9.68 9.68 9.68 9.68 9.68 9.68 Oracle 4 10.40 10.03 9.98 9.71 9.84 10.09 9.73 9.73 9.73 9.73 9.73 9.73 9.73 9.73 9.73 9.73 9.73 9.73 9.73 9.73 Oracle 5 1.33 1.27 1.27 1.29 1.27 1.27 1.27 1.30 1.30 1.30 1.30 1.30 1.30 1.30 1.30 1.30 1.30 1.30 1.30 1.30 Oracle 6 11.38 1.89 1.47 1.46 1.45 1.45 1.45 1.45 1.45 1.45 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 Oracle 7 10.85 0.63 0.62 0.63 0.61 0.61 0.61 0.61 0.61 0.64 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 8 1.02 1.05 1.02 1.03 1.04 1.02 1.05 1.07 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 9 30.37 30.27 30.27 30.33 30.26 30.14 30.25 30.25 30.25 30.25 30.25 30.25 30.25 30.25 30.25 30.25 30.25 30.25 30.25 30.25 Oracle 10 9.55 0.61 0.62 0.61 0.62 0.61 0.62 0.61 0.61 0.61 0.62 0.63 0.62 0.62 0.64 0.61 0.61 0.64 0.61 0.61 Oracle 11 9.98 1.02 1.02 1.05 1.03 1.02 1.05 1.02 1.02 1.02 1.02 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 12 9.95 1.03 1.11 1.05 1.03 1.03 1.08 1.03 1.06 1.03 1.05 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 13 10.43 10.27 10.17 10.17 1.05 1.03 1.03 1.05 1.04 1.03 1.03 1.08 1.04 1.03 1.06 1.06 1.06 1.06 1.06 1.06 Oracle 14 10.05 1.06 10.09 1.04 1.12 1.10 1.03 1.03 1.03 1.06 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 15 10.00 1.03 1.04 1.02 1.02 1.02 1.02 1.08 1.08 1.08 1.08 1.08 1.08 1.08 1.08 1.08 1.08 1.08 1.08 1.08 Oracle 16 9.56 1.04 1.04 1.02 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 17 1.77 1.44 1.48 1.47 1.45 1.45 1.44 1.44 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 Oracle 18 10.02 9.63 9.60 9.59 9.38 9.37 9.45 9.34 9.39 9.39 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 19 1.03 1.45 1.48 1.09 1.06 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 20 2.30 1.47 1.06 1.04 1.02 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 21 2.28 1.88 1.88 1.90 1.45 1.42 1.46 1.42 1.46 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 Oracle 22 11.31 1.04 1.03 1.08 1.05 1.05 1.03 1.07 1.09 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 23 11.30 0.61 0.61 1.03 1.03 1.03 1.03 1.03 1.04 1.08 1.02 1.02 1.02 1.08 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 24 10.85 9.61 9.42 9.43 9.40 9.37 9.41 9.46 9.52 9.42 9.47 9.47 9.47 9.47 9.47 9.47 9.47 9.47 9.47 9.47 Oracle 25 2.64 2.70 2.65 2.66 2.66 2.66 2.69 2.64 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Oracle 26 10.46 9.72 0.62 0.61 9.70 9.41 9.36 9.45 9.38 9.52 9.55 0.63 9.40 9.42 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 27 9.65 0.61 9.67 9.75 0.62 0.62 9.69 9.53 9.55 9.40 9.49 9.50 9.51 9.51 9.51 9.51 9.51 9.51 9.51 9.51 Oracle 28 2.69 2.63 1.42 1.42 10.98 9.48 9.44 9.44 9.44 9.44 9.44 9.44 9.44 9.44 9.44 9.44 9.44 9.44 9.44 9.44 Oracle 29 2.75 2.80 2.76 11.11 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 30 1.73 0.92 0.87 0.62 0.64 0.62 0.62 0.62 0.88 0.62 0.86 0.87 0.85 0.85 0.85 0.85 0.85 0.85 0.85 0.85 Oracle 31 11.43 0.61 1.02 1.04 1.03 1.04 1.02 1.02 1.02 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 32 11.36 0.62 0.61 0.62 0.61 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 33 30.25 30.24 30.15 30.07 30.07 30.53 30.56 30.38 30.47 30.47 30.36 30.47 30.47 30.47 30.47 30.47 30.47 30.47 30.47 30.47 Oracle 34 2.80 2.69 2.73 2.74 2.70 2.80 2.69 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 35 10.98 9.61 9.69 9.66 9.40 0.61 0.61 0.61 9.41 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 36 10.50 9.71 0.61 9.80 9.70 0.63 0.62 0.61 9.48 9.46 0.61 0.60 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 37 10.94 0.61 9.69 9.67 0.63 9.41 9.41 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 38 10.93 0.61 0.61 0.61 9.63 9.70 9.47 0.64 9.55 0.61 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 39 11.41 2.24 2.24 2.25 2.35 2.23 2.24 2.24 2.24 2.24 2.24 2.24 2.24 2.24 2.24 2.24 2.24 2.24 2.24 2.24 Oracle 40 2.73 2.73 10.59 1.07 1.05 1.03 1.02 1.02 1.02 1.02 1.08 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 41 10.85 1.03 1.03 1.03 1.04 1.04 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 42 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 43 3.05 1.47 1.45 1.45 1.44 1.47 1.49 1.43 1.02 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 Oracle 44 10.73 1.05 1.03 1.02 1.02 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 45 11.38 1.03 1.07 1.02 1.02 1.05 0.61 1.08 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 46 10.19 0.62 0.61 9.70 9.68 0.64 0.64 0.61 9.45 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 47 2.28 1.42 1.41 1.45 10.13 9.86 9.44 9.51 9.46 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 Oracle 48 10.09 0.61 9.72 9.44 0.61 0.62 9.65 0.61 0.61 0.61 0.61 0.61 9.48 9.48 9.48 9.48 9.48 9.48 9.48 9.48 Oracle 49 9.71 0.62 9.77 9.51 9.48 0.61 0.63 0.61 0.61 9.52 9.52 9.52 9.52 9.52 9.52 9.52 9.52 9.52 9.52 9.52 Oracle 50 2.74 2.71 2.70 2.74 2.72 2.70 2.74 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Total 445 201 242 249 238 209 226 200 235 208 191 182 200 200 191 191 191 191 191 191 Average 8.90 4.02 4.85 4.98 4.76 4.18 4.52 3.99 4.70 4.16 3.82 3.64 3.99 4.00 3.82 3.82 3.82 3.82 3.82 3.82 D.10. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH CHILD LINKS ONLY301
Size Results of configuration Shattered Document with Child links only
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 37 4 5 4.11 5 4 1 0.80 1.25 Oracle 2 29 4 7 4.84 7 4 3 0.57 1.75 Oracle 3 33 4 5 4.13 4 4 0 1.00 1.00 Oracle 4 29 4 5 4.14 5 4 1 0.80 1.25 Oracle 5 32 4 4 4.00 4 4 0 1.00 1.00 Oracle 6 48 4 7 4.37 7 4 3 0.57 1.75 Oracle 7 26 2 6 2.37 6 2 4 0.34 3.00 Oracle 8 33 3 3 3.00 3 3 0 1.00 1.00 Oracle 9 35 5 5 5.00 5 5 0 1.00 1.00 Oracle 10 41 2 3 2.05 3 2 1 0.66 1.50 Oracle 11 37 3 4 3.09 4 3 1 0.75 1.34 Oracle 12 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 13 49 3 4 3.27 4 3 1 0.75 1.34 Oracle 14 35 3 4 3.18 4 3 1 0.75 1.34 Oracle 15 25 3 4 3.13 4 3 1 0.75 1.34 Oracle 16 18 3 3 3.00 3 3 0 1.00 1.00 Oracle 17 36 4 4 4.00 4 4 0 1.00 1.00 Oracle 18 33 2 4 3.00 4 2 2 0.50 2.00 Oracle 19 20 3 4 3.34 3 3 0 1.00 1.00 Oracle 20 22 3 6 3.66 6 3 3 0.50 2.00 Oracle 21 45 4 6 4.50 6 4 2 0.66 1.50 Oracle 22 34 3 7 3.40 7 3 4 0.43 2.34 Oracle 23 47 2 7 3.13 7 3 4 0.43 2.34 Oracle 24 36 3 6 3.27 6 3 3 0.50 2.00 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 43 2 5 2.87 5 2 3 0.40 2.50 Oracle 27 36 2 3 2.77 3 3 0 1.00 1.00 Oracle 28 34 3 7 4.86 7 3 4 0.43 2.34 Oracle 29 31 3 7 6.20 7 3 4 0.43 2.34 Oracle 30 35 2 5 2.70 5 3 2 0.60 1.66 Oracle 31 36 2 7 3.27 7 3 4 0.43 2.34 Oracle 32 42 2 7 3.00 7 3 4 0.43 2.34 Oracle 33 50 4 6 4.16 6 4 2 0.66 1.50 Oracle 34 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 35 29 2 6 2.90 6 2 4 0.34 3.00 Oracle 36 34 2 5 2.62 5 2 3 0.40 2.50 Oracle 37 30 2 6 2.73 6 2 4 0.34 3.00 Oracle 38 30 2 6 2.73 6 2 4 0.34 3.00 Oracle 39 43 6 7 6.14 7 6 1 0.86 1.16 Oracle 40 52 3 7 3.71 7 3 4 0.43 2.34 Oracle 41 26 3 5 3.25 5 3 2 0.60 1.66 Oracle 42 3 3 3 3.00 3 3 0 1.00 1.00 Oracle 43 42 3 7 4.20 7 4 3 0.57 1.75 Oracle 44 19 2 5 3.16 5 2 3 0.40 2.50 Oracle 45 30 2 7 3.34 7 3 4 0.43 2.34 Oracle 46 25 2 4 2.50 4 2 2 0.50 2.00 Oracle 47 36 2 6 3.60 6 2 4 0.34 3.00 Oracle 48 32 2 4 2.46 4 3 1 0.75 1.34 Oracle 49 25 2 3 2.50 3 3 0 1.00 1.00 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 1765 157 266 184.70 264 166 98 33.16 86.91 Average 35.30 3.14 5.32 3.70 5.28 3.32 1.96 0.66 1.74 Micro avg ····················· 0.63 1.59 302 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Child links only
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 2 7 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 3 4 4 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 4 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 6 7 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 7 6 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 8 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 9 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 10 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 11 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 12 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 13 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 14 4 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 15 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 16 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 17 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 18 4 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 Oracle 19 3 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 20 6 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 21 6 5 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 22 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 23 7 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 24 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 5 3 2 2 3 3 3 3 3 3 3 2 3 3 2 2 2 2 2 2 Oracle 27 3 2 3 3 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 28 7 7 4 4 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 29 7 7 7 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 30 5 3 3 2 2 2 2 2 3 2 3 3 3 3 3 3 3 3 3 3 Oracle 31 7 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 32 7 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 33 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 34 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 35 6 3 3 3 3 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 36 5 3 2 3 3 2 2 2 3 3 2 2 2 2 2 2 2 2 2 2 Oracle 37 6 2 3 3 2 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 38 6 2 2 2 3 3 3 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 39 7 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 Oracle 40 7 7 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 41 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 42 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 43 7 4 4 4 4 4 4 4 3 4 4 4 4 4 4 4 4 4 4 4 Oracle 44 5 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 45 7 3 3 3 3 3 2 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 46 4 2 2 3 3 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 47 6 4 4 4 4 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 48 4 2 3 3 2 2 3 2 2 2 2 2 3 3 3 3 3 3 3 3 Oracle 49 3 2 3 3 3 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 264 186 182 179 174 167 168 166 170 167 166 165 167 167 166 166 166 166 166 166 Average 5.28 3.72 3.64 3.58 3.48 3.34 3.36 3.32 3.40 3.34 3.32 3.30 3.34 3.34 3.32 3.32 3.32 3.32 3.32 3.32 D.10. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH CHILD LINKS ONLY303
Cost Results of configuration Shattered Document with Child links only
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 37.00 4.00 5.00 4.11 5.00 4.00 1.00 0.80 1.25 Oracle 2 29.00 4.00 7.00 4.84 7.00 4.00 3.00 0.57 1.75 Oracle 3 32.00 3.00 5.00 4.00 3.00 4.00 -1.00 1.34 0.75 Oracle 4 29.00 4.00 5.00 4.14 5.00 4.00 1.00 0.80 1.25 Oracle 5 32.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 6 48.00 4.00 7.00 4.37 7.00 4.00 3.00 0.57 1.75 Oracle 7 26.00 2.00 6.00 2.37 6.00 2.00 4.00 0.34 3.00 Oracle 8 33.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 9 28.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 10 41.00 2.00 3.00 2.05 3.00 2.00 1.00 0.66 1.50 Oracle 11 37.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 12 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 13 49.00 3.00 4.00 3.27 4.00 3.00 1.00 0.75 1.34 Oracle 14 35.00 3.00 4.00 3.18 4.00 3.00 1.00 0.75 1.34 Oracle 15 25.00 3.00 4.00 3.13 4.00 3.00 1.00 0.75 1.34 Oracle 16 18.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 17 36.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 18 33.00 2.00 4.00 3.00 4.00 2.00 2.00 0.50 2.00 Oracle 19 20.00 3.00 4.00 3.34 3.00 3.00 0.00 1.00 1.00 Oracle 20 22.00 3.00 6.00 3.66 6.00 3.00 3.00 0.50 2.00 Oracle 21 45.00 4.00 6.00 4.50 6.00 4.00 2.00 0.66 1.50 Oracle 22 34.00 3.00 7.00 3.40 7.00 3.00 4.00 0.43 2.34 Oracle 23 47.00 2.00 7.00 3.13 7.00 3.00 4.00 0.43 2.34 Oracle 24 36.00 3.00 6.00 3.27 6.00 3.00 3.00 0.50 2.00 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 43.00 2.00 5.00 2.87 5.00 2.00 3.00 0.40 2.50 Oracle 27 36.00 2.00 3.00 2.77 3.00 3.00 0.00 1.00 1.00 Oracle 28 34.00 3.00 7.00 4.86 7.00 3.00 4.00 0.43 2.34 Oracle 29 31.00 3.00 7.00 6.20 7.00 3.00 4.00 0.43 2.34 Oracle 30 35.00 2.00 5.00 2.70 5.00 3.00 2.00 0.60 1.66 Oracle 31 36.00 2.00 7.00 3.27 7.00 3.00 4.00 0.43 2.34 Oracle 32 42.00 2.00 7.00 3.00 7.00 3.00 4.00 0.43 2.34 Oracle 33 38.00 3.00 5.00 3.16 5.00 3.00 2.00 0.60 1.66 Oracle 34 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 35 29.00 2.00 6.00 2.90 6.00 2.00 4.00 0.34 3.00 Oracle 36 34.00 2.00 5.00 2.62 5.00 2.00 3.00 0.40 2.50 Oracle 37 30.00 2.00 6.00 2.73 6.00 2.00 4.00 0.34 3.00 Oracle 38 30.00 2.00 6.00 2.73 6.00 2.00 4.00 0.34 3.00 Oracle 39 43.00 6.00 7.00 6.14 7.00 6.00 1.00 0.86 1.16 Oracle 40 52.00 3.00 7.00 3.71 7.00 3.00 4.00 0.43 2.34 Oracle 41 26.00 3.00 5.00 3.25 5.00 3.00 2.00 0.60 1.66 Oracle 42 3.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 43 42.00 3.00 7.00 4.20 7.00 4.00 3.00 0.57 1.75 Oracle 44 19.00 2.00 5.00 3.16 5.00 2.00 3.00 0.40 2.50 Oracle 45 30.00 2.00 7.00 3.34 7.00 3.00 4.00 0.43 2.34 Oracle 46 25.00 2.00 4.00 2.50 4.00 2.00 2.00 0.50 2.00 Oracle 47 36.00 2.00 6.00 3.60 6.00 2.00 4.00 0.34 3.00 Oracle 48 32.00 2.00 4.00 2.46 4.00 3.00 1.00 0.75 1.34 Oracle 49 25.00 2.00 3.00 2.50 3.00 3.00 0.00 1.00 1.00 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 1745.00 154.00 264.00 182.57 261.00 164.00 97.00 33.43 86.84 Average 34.90 3.08 5.28 3.65 5.22 3.28 1.94 0.67 1.73 Micro avg ····················· 0.63 1.59 304 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Child links only
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 2 7.00 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 3 3.00 4.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 4 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 5 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 6 7.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 7 6.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 8 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 9 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 10 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 11 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 12 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 13 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 14 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 15 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 16 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 17 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 18 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 19 3.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 20 6.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 21 6.00 5.00 5.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 22 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 23 7.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 24 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 5.00 3.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 27 3.00 2.00 3.00 3.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 28 7.00 7.00 4.00 4.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 29 7.00 7.00 7.00 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 30 5.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 31 7.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 32 7.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 33 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 34 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 35 6.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 36 5.00 3.00 2.00 3.00 3.00 2.00 2.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 37 6.00 2.00 3.00 3.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 38 6.00 2.00 2.00 2.00 3.00 3.00 3.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 39 7.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 40 7.00 7.00 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 41 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 42 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 43 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 44 5.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 45 7.00 3.00 3.00 3.00 3.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 46 4.00 2.00 2.00 3.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 47 6.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 48 4.00 2.00 3.00 3.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 49 3.00 2.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 261 184 180 177 172 165 166 164 168 165 164 163 165 165 164 164 164 164 164 164 Average 5.22 3.68 3.60 3.54 3.44 3.30 3.32 3.28 3.36 3.30 3.28 3.26 3.30 3.30 3.28 3.28 3.28 3.28 3.28 3.28 D.11. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, NO NEXT LINKS, WEAK RESIDUAL305 D.11 Complete results of configuration Shattered Document, no Next links, weak Residual
Like Shattered Document, but with null Next weight and a small Residual.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 0.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.10
Principal results: Cost evolution:
Success Rate = 0.52
Size0 = 6.12
Size1 = 4.70 Size gain = 1.30
Cost0 = 6.12
Cost1 = 4.70 Cost gain = 1.30 306 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document, no Next links, weak Residual
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 9 0 9 0 1.00 0.00 Oracle 2 30 6 0 6 0 6 0 1.00 0.00 Oracle 3 40 8 0 8 0 8 0 1.00 0.00 Oracle 4 46 7 0 7 0 7 0 1.00 0.00 Oracle 5 56 8 0 8 0 8 0 1.00 0.00 Oracle 6 85 11 0 11 0 11 0 1.00 0.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 0 0 0 0 7 0.00 1.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 12 0 12 0 1.00 0.00 Oracle 12 157 13 0 13 0 13 0 1.00 0.00 Oracle 13 171 15 0 15 0 15 0 1.00 0.00 Oracle 14 173 11 0 11 0 11 0 1.00 0.00 Oracle 15 241 8 0 8 0 8 0 1.00 0.00 Oracle 16 265 6 0 6 0 6 0 1.00 0.00 Oracle 17 266 9 0 9 0 9 0 1.00 0.00 Oracle 18 269 11 0 11 0 11 0 1.00 0.00 Oracle 19 290 6 0 6 0 6 0 1.00 0.00 Oracle 20 307 6 0 6 0 6 0 1.00 0.00 Oracle 21 366 10 0 0 0 0 10 0.00 1.00 Oracle 22 410 10 0 10 0 10 0 1.00 0.00 Oracle 23 455 15 0 0 0 0 15 0.00 1.00 Oracle 24 508 11 0 0 0 0 11 0.00 1.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 1 0 1 14 0.07 0.93 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 2 0 2 5 0.29 0.71 Oracle 29 542 5 0 1 0 1 4 0.20 0.80 Oracle 30 554 13 0 0 0 0 13 0.00 1.00 Oracle 31 601 11 0 0 0 0 11 0.00 1.00 Oracle 32 606 14 0 0 0 0 14 0.00 1.00 Oracle 33 616 12 0 4 0 4 8 0.34 0.66 Oracle 34 649 8 0 1 0 1 7 0.13 0.88 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 0 0 0 11 0.00 1.00 Oracle 39 786 7 0 7 0 7 0 1.00 0.00 Oracle 40 840 14 0 14 0 14 0 1.00 0.00 Oracle 41 846 8 0 8 0 8 0 1.00 0.00 Oracle 42 883 1 0 0 0 0 1 0.00 1.00 Oracle 43 890 10 0 10 0 10 0 1.00 0.00 Oracle 44 897 6 0 6 0 6 0 1.00 0.00 Oracle 45 899 9 0 9 0 9 0 1.00 0.00 Oracle 46 929 10 0 0 0 0 10 0.00 1.00 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 0 0 0 0 13 0.00 1.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 237 0 237 263 26.01 23.99 Average ··· 10.00 0.00 4.74 0.00 4.74 5.26 0.52 0.48 Micro avg ····················· 0.48 0.52 D.11. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, NO NEXT LINKS, WEAK RESIDUAL307
Duration Results of configuration Shattered Document, no Next links, weak Residual
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 24.09 2.66 2.72 2.68 2.72 2.66 0.06 0.98 1.02 Oracle 2 16.03 2.66 2.71 2.67 2.71 2.67 0.04 0.98 1.02 Oracle 3 21.59 2.65 2.73 2.70 2.73 2.66 0.07 0.98 1.02 Oracle 4 18.77 2.66 2.72 2.68 2.72 2.69 0.03 0.99 1.01 Oracle 5 21.37 2.65 2.71 2.67 2.66 2.67 -0.02 1.01 0.99 Oracle 6 29.55 2.65 2.72 2.69 2.65 2.70 -0.05 1.02 0.98 Oracle 7 8.46 0.61 0.92 0.77 0.91 0.62 0.29 0.68 1.47 Oracle 8 8.30 0.61 1.35 0.76 1.35 0.61 0.74 0.45 2.22 Oracle 9 6.83 0.61 2.11 0.98 2.11 0.65 1.46 0.30 3.25 Oracle 10 13.55 0.61 0.88 0.68 0.88 0.61 0.27 0.69 1.45 Oracle 11 32.25 2.65 2.80 2.69 2.71 2.80 -0.09 1.03 0.97 Oracle 12 34.95 2.65 2.78 2.69 2.69 2.78 -0.09 1.03 0.97 Oracle 13 40.14 2.66 2.71 2.68 2.68 2.66 0.02 0.99 1.01 Oracle 14 29.68 2.66 2.73 2.70 2.73 2.71 0.02 0.99 1.01 Oracle 15 21.40 2.65 2.71 2.67 2.68 2.71 -0.03 1.01 0.99 Oracle 16 16.10 2.65 2.77 2.68 2.77 2.66 0.11 0.96 1.04 Oracle 17 24.12 2.65 2.84 2.68 2.84 2.66 0.19 0.94 1.07 Oracle 18 29.65 2.66 2.75 2.70 2.66 2.69 -0.03 1.02 0.98 Oracle 19 16.08 2.66 2.70 2.68 2.66 2.66 0.00 1.00 1.00 Oracle 20 16.17 2.66 2.75 2.70 2.68 2.66 0.02 0.99 1.01 Oracle 21 10.29 1.02 1.05 1.03 1.03 1.02 0.01 1.00 1.00 Oracle 22 26.91 2.66 2.72 2.69 2.71 2.66 0.05 0.98 1.02 Oracle 23 10.77 0.61 2.09 0.72 2.09 0.62 1.48 0.30 3.40 Oracle 24 9.77 0.61 2.11 0.89 2.11 0.61 1.49 0.29 3.45 Oracle 25 24.17 2.65 2.78 2.69 2.71 2.66 0.05 0.98 1.02 Oracle 26 14.85 0.61 2.77 0.99 2.77 0.61 2.16 0.22 4.52 Oracle 27 11.67 0.61 2.52 0.90 2.52 0.61 1.91 0.24 4.12 Oracle 28 13.47 0.87 2.71 1.92 1.03 0.87 0.16 0.84 1.18 Oracle 29 7.09 1.02 2.68 1.42 2.68 1.09 1.59 0.41 2.45 Oracle 30 10.53 0.61 1.77 0.81 1.77 0.87 0.91 0.48 2.05 Oracle 31 8.31 0.61 2.16 0.76 2.16 0.64 1.52 0.30 3.35 Oracle 32 10.17 0.61 2.12 0.73 2.12 0.63 1.49 0.30 3.41 Oracle 33 19.64 1.02 2.70 1.64 2.70 1.02 1.67 0.38 2.63 Oracle 34 10.23 1.02 2.68 1.28 2.68 1.02 1.66 0.38 2.62 Oracle 35 8.95 0.61 2.12 0.90 2.12 0.61 1.51 0.29 3.45 Oracle 36 10.38 0.61 1.70 0.80 1.70 0.62 1.08 0.37 2.75 Oracle 37 9.29 0.61 2.13 0.84 2.13 0.61 1.52 0.29 3.50 Oracle 38 8.90 0.61 1.70 0.81 1.70 0.61 1.09 0.36 2.79 Oracle 39 18.70 2.65 2.72 2.67 2.70 2.66 0.04 0.98 1.02 Oracle 40 37.45 2.65 2.72 2.67 2.72 2.66 0.05 0.98 1.02 Oracle 41 21.41 2.66 2.72 2.68 2.66 2.67 -0.01 1.00 1.00 Oracle 42 1.28 1.28 1.28 1.28 1.28 1.28 0.00 1.00 1.00 Oracle 43 26.73 2.65 2.71 2.67 2.70 2.67 0.02 0.99 1.01 Oracle 44 16.09 2.65 2.72 2.68 2.71 2.66 0.05 0.98 1.02 Oracle 45 24.05 2.66 2.70 2.67 2.70 2.66 0.05 0.98 1.02 Oracle 46 7.88 0.61 1.27 0.79 1.27 0.61 0.66 0.48 2.08 Oracle 47 8.98 0.61 2.11 0.90 2.11 0.62 1.49 0.29 3.43 Oracle 48 10.80 0.61 2.11 0.83 2.11 0.90 1.21 0.43 2.34 Oracle 49 7.70 0.61 0.89 0.77 0.87 0.89 -0.02 1.02 0.98 Oracle 50 24.04 2.65 2.72 2.67 2.66 2.66 0.00 1.00 1.00 Total 859.60 84.13 116.31 91.21 113.74 85.84 27.90 36.59 90.11 Average 17.20 1.68 2.33 1.82 2.27 1.72 0.55 0.73 1.80 Micro avg ····················· 0.76 1.33 308 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document, no Next links, weak Residual
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 2.72 2.66 2.68 2.67 2.69 2.68 2.66 2.67 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 2 2.71 2.66 2.66 2.68 2.66 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 3 2.73 2.71 2.73 2.65 2.70 2.71 2.71 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 4 2.72 2.66 2.69 2.68 2.68 2.66 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 5 2.66 2.66 2.71 2.67 2.66 2.65 2.68 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 6 2.65 2.71 2.72 2.66 2.72 2.69 2.71 2.66 2.67 2.66 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 7 0.91 0.87 0.88 0.87 0.92 0.62 0.64 0.67 0.87 0.61 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 8 1.35 0.88 0.62 0.88 0.61 0.63 0.61 0.61 0.61 0.91 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 9 2.11 0.88 0.87 0.61 0.87 0.86 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 Oracle 10 0.88 0.62 0.62 0.87 0.61 0.61 0.61 0.62 0.87 0.61 0.61 0.61 0.61 0.87 0.64 0.61 0.61 0.87 0.62 0.61 Oracle 11 2.71 2.67 2.72 2.71 2.66 2.65 2.66 2.66 2.66 2.69 2.67 2.80 2.80 2.80 2.80 2.80 2.80 2.80 2.80 2.80 Oracle 12 2.69 2.68 2.70 2.65 2.78 2.66 2.66 2.66 2.66 2.71 2.66 2.65 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 Oracle 13 2.68 2.69 2.69 2.70 2.71 2.66 2.66 2.69 2.66 2.67 2.71 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 14 2.73 2.70 2.71 2.70 2.70 2.66 2.71 2.66 2.69 2.70 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 15 2.68 2.66 2.66 2.66 2.71 2.67 2.65 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 16 2.77 2.65 2.65 2.66 2.72 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 17 2.84 2.68 2.66 2.66 2.66 2.66 2.65 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 18 2.66 2.66 2.75 2.71 2.74 2.68 2.67 2.70 2.72 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 19 2.66 2.70 2.69 2.70 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 20 2.68 2.70 2.71 2.75 2.68 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 21 1.03 1.03 1.04 1.02 1.02 1.02 1.03 1.02 1.05 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 22 2.71 2.69 2.66 2.66 2.72 2.69 2.71 2.72 2.70 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 23 2.09 0.61 0.64 0.64 0.62 0.62 0.61 0.62 0.63 0.61 0.61 0.61 0.64 0.61 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 24 2.11 0.87 0.87 0.88 0.86 0.88 0.62 0.61 0.62 0.87 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 25 2.71 2.66 2.67 2.65 2.68 2.78 2.69 2.67 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 26 2.77 2.53 0.61 0.62 0.87 0.88 0.87 0.62 0.87 0.62 0.86 0.61 0.62 0.91 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 27 2.52 0.61 0.61 0.62 0.88 0.61 0.87 0.88 0.88 0.87 0.87 0.87 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 28 1.03 1.03 2.71 2.66 2.65 2.52 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 Oracle 29 2.68 1.27 1.02 1.02 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 30 1.77 0.90 0.64 0.61 0.63 0.63 0.61 0.62 0.87 0.62 0.89 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 0.87 Oracle 31 2.16 0.62 0.61 0.62 0.61 0.61 0.61 0.61 0.61 0.61 0.64 0.64 0.64 0.64 0.64 0.64 0.64 0.64 0.64 0.64 Oracle 32 2.12 0.61 0.61 0.61 0.64 0.62 0.63 0.62 0.61 0.62 0.61 0.65 0.62 0.63 0.63 0.63 0.63 0.63 0.63 0.63 Oracle 33 2.70 2.67 2.66 2.65 1.73 1.05 1.02 1.04 1.05 1.02 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 34 2.68 1.28 1.02 1.12 1.05 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 35 2.12 0.87 0.87 0.87 0.87 0.63 0.61 0.63 0.88 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 36 1.70 0.87 0.61 0.87 0.89 0.61 0.61 0.61 0.88 0.87 0.63 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 37 2.13 0.61 0.87 0.88 0.61 0.88 0.61 0.61 0.62 0.87 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 38 1.70 0.65 0.62 0.61 0.87 0.87 0.87 0.62 0.87 0.62 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 39 2.70 2.66 2.65 2.72 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 40 2.72 2.66 2.66 2.70 2.71 2.66 2.66 2.66 2.68 2.65 2.70 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 41 2.66 2.66 2.71 2.66 2.72 2.66 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 42 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 Oracle 43 2.70 2.65 2.65 2.68 2.66 2.71 2.67 2.68 2.66 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 44 2.71 2.66 2.69 2.72 2.65 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 45 2.70 2.66 2.66 2.66 2.69 2.66 2.66 2.69 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 46 1.27 0.95 0.61 0.87 0.88 0.61 0.61 0.61 0.87 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 47 2.11 0.62 0.87 0.87 0.61 0.61 0.94 0.87 0.88 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 48 2.11 0.87 0.87 0.88 0.61 0.62 0.87 0.61 0.61 0.63 0.62 0.61 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 49 0.87 0.87 0.87 0.87 0.87 0.63 0.63 0.61 0.62 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 Oracle 50 2.66 2.70 2.65 2.66 2.66 2.72 2.68 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Total 114 92 91 91 90 88 86 85 87 86 86 86 86 86 86 86 86 86 86 86 Average 2.27 1.84 1.81 1.83 1.80 1.76 1.73 1.71 1.75 1.73 1.72 1.71 1.72 1.73 1.72 1.72 1.72 1.72 1.72 1.72 D.11. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, NO NEXT LINKS, WEAK RESIDUAL309
Size Results of configuration Shattered Document, no Next links, weak Residual
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 2 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 3 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 4 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 5 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 6 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 7 28 2 3 2.55 3 2 1 0.66 1.50 Oracle 8 27 2 4 2.45 4 2 2 0.50 2.00 Oracle 9 22 2 6 3.14 6 2 4 0.34 3.00 Oracle 10 45 2 3 2.25 3 2 1 0.66 1.50 Oracle 11 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 12 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 13 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 14 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 15 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 16 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 17 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 18 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 19 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 20 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 21 30 3 3 3.00 3 3 0 1.00 1.00 Oracle 22 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 23 34 2 6 2.27 6 2 4 0.34 3.00 Oracle 24 32 2 6 2.91 6 2 4 0.34 3.00 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 46 2 7 3.07 7 2 5 0.29 3.50 Oracle 27 38 2 7 2.92 7 2 5 0.29 3.50 Oracle 28 37 3 7 5.29 3 3 0 1.00 1.00 Oracle 29 20 3 7 4.00 7 3 4 0.43 2.34 Oracle 30 34 2 5 2.62 5 3 2 0.60 1.66 Oracle 31 26 2 6 2.37 6 2 4 0.34 3.00 Oracle 32 32 2 6 2.29 6 2 4 0.34 3.00 Oracle 33 54 3 7 4.50 7 3 4 0.43 2.34 Oracle 34 29 3 7 3.63 7 3 4 0.43 2.34 Oracle 35 29 2 6 2.90 6 2 4 0.34 3.00 Oracle 36 34 2 5 2.62 5 2 3 0.40 2.50 Oracle 37 30 2 6 2.73 6 2 4 0.34 3.00 Oracle 38 29 2 5 2.63 5 2 3 0.40 2.50 Oracle 39 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 40 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 41 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 42 4 4 4 4.00 4 4 0 1.00 1.00 Oracle 43 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 44 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 45 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 46 26 2 4 2.60 4 2 2 0.50 2.00 Oracle 47 29 2 6 2.90 6 2 4 0.34 3.00 Oracle 48 35 2 6 2.70 6 3 3 0.50 2.00 Oracle 49 26 2 3 2.60 3 3 0 1.00 1.00 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 2372 232 310 249.91 306 235 71 37.76 82.66 Average 47.44 4.64 6.20 5.00 6.12 4.70 1.42 0.76 1.66 Micro avg ····················· 0.77 1.30 310 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document, no Next links, weak Residual
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 2 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 3 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 4 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 5 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 6 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 7 3 3 3 3 3 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 8 4 3 2 3 2 2 2 2 2 3 2 2 2 2 2 2 2 2 2 2 Oracle 9 6 3 3 2 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 10 3 2 2 3 2 2 2 2 3 2 2 2 2 3 2 2 2 3 2 2 Oracle 11 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 12 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 13 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 14 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 15 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 16 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 17 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 18 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 19 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 20 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 21 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 22 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 23 6 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 24 6 3 3 3 3 3 2 2 2 3 2 2 2 2 2 2 2 2 2 2 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 7 7 2 2 3 3 3 2 3 2 3 2 2 3 2 2 2 2 2 2 Oracle 27 7 2 2 2 3 2 3 3 3 3 3 3 2 2 2 2 2 2 2 2 Oracle 28 3 3 7 7 7 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 29 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 30 5 3 2 2 2 2 2 2 3 2 3 3 3 3 3 3 3 3 3 3 Oracle 31 6 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 32 6 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 33 7 7 7 7 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 34 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 35 6 3 3 3 3 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 36 5 3 2 3 3 2 2 2 3 3 2 2 2 2 2 2 2 2 2 2 Oracle 37 6 2 3 3 2 3 2 2 2 3 2 2 2 2 2 2 2 2 2 2 Oracle 38 5 2 2 2 3 3 3 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 39 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 40 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 41 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 42 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 43 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 44 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 45 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 46 4 3 2 3 3 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 47 6 2 3 3 2 2 3 3 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 48 6 3 3 3 2 2 3 2 2 2 2 2 3 3 3 3 3 3 3 3 Oracle 49 3 3 3 3 3 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 306 253 248 251 248 241 237 234 242 238 236 235 235 237 235 235 235 236 235 235 Average 6.12 5.06 4.96 5.02 4.96 4.82 4.74 4.68 4.84 4.76 4.72 4.70 4.70 4.74 4.70 4.70 4.70 4.72 4.70 4.70 D.11. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, NO NEXT LINKS, WEAK RESIDUAL311
Cost Results of configuration Shattered Document, no Next links, weak Residual
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 2 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 3 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 4 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 5 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 6 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 7 28.00 2.00 3.00 2.55 3.00 2.00 1.00 0.66 1.50 Oracle 8 27.00 2.00 4.00 2.45 4.00 2.00 2.00 0.50 2.00 Oracle 9 22.00 2.00 6.00 3.14 6.00 2.00 4.00 0.34 3.00 Oracle 10 45.00 2.00 3.00 2.25 3.00 2.00 1.00 0.66 1.50 Oracle 11 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 12 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 13 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 14 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 15 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 16 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 17 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 18 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 19 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 20 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 21 30.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 22 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 23 34.00 2.00 6.00 2.27 6.00 2.00 4.00 0.34 3.00 Oracle 24 32.00 2.00 6.00 2.91 6.00 2.00 4.00 0.34 3.00 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 46.00 2.00 7.00 3.07 7.00 2.00 5.00 0.29 3.50 Oracle 27 38.00 2.00 7.00 2.92 7.00 2.00 5.00 0.29 3.50 Oracle 28 37.00 3.00 7.00 5.29 3.00 3.00 0.00 1.00 1.00 Oracle 29 20.00 3.00 7.00 4.00 7.00 3.00 4.00 0.43 2.34 Oracle 30 34.00 2.00 5.00 2.62 5.00 3.00 2.00 0.60 1.66 Oracle 31 26.00 2.00 6.00 2.37 6.00 2.00 4.00 0.34 3.00 Oracle 32 32.00 2.00 6.00 2.29 6.00 2.00 4.00 0.34 3.00 Oracle 33 54.00 3.00 7.00 4.50 7.00 3.00 4.00 0.43 2.34 Oracle 34 29.00 3.00 7.00 3.63 7.00 3.00 4.00 0.43 2.34 Oracle 35 29.00 2.00 6.00 2.90 6.00 2.00 4.00 0.34 3.00 Oracle 36 34.00 2.00 5.00 2.62 5.00 2.00 3.00 0.40 2.50 Oracle 37 30.00 2.00 6.00 2.73 6.00 2.00 4.00 0.34 3.00 Oracle 38 29.00 2.00 5.00 2.63 5.00 2.00 3.00 0.40 2.50 Oracle 39 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 40 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 41 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 42 4.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 43 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 44 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 45 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 46 26.00 2.00 4.00 2.60 4.00 2.00 2.00 0.50 2.00 Oracle 47 29.00 2.00 6.00 2.90 6.00 2.00 4.00 0.34 3.00 Oracle 48 35.00 2.00 6.00 2.70 6.00 3.00 3.00 0.50 2.00 Oracle 49 26.00 2.00 3.00 2.60 3.00 3.00 0.00 1.00 1.00 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 2372.00 232.00 310.00 249.91 306.00 235.00 71.00 37.76 82.66 Average 47.44 4.64 6.20 5.00 6.12 4.70 1.42 0.76 1.66 Micro avg ····················· 0.77 1.30 312 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document, no Next links, weak Residual
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 2 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 3 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 4 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 5 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 6 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 7 3.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 8 4.00 3.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 9 6.00 3.00 3.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 10 3.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 Oracle 11 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 12 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 13 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 14 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 15 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 16 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 17 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 18 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 19 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 20 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 21 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 22 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 23 6.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 24 6.00 3.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 7.00 7.00 2.00 2.00 3.00 3.00 3.00 2.00 3.00 2.00 3.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 27 7.00 2.00 2.00 2.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 28 3.00 3.00 7.00 7.00 7.00 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 29 7.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 30 5.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 31 6.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 32 6.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 33 7.00 7.00 7.00 7.00 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 34 7.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 35 6.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 36 5.00 3.00 2.00 3.00 3.00 2.00 2.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 37 6.00 2.00 3.00 3.00 2.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 38 5.00 2.00 2.00 2.00 3.00 3.00 3.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 39 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 40 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 41 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 42 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 43 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 44 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 45 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 46 4.00 3.00 2.00 3.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 47 6.00 2.00 3.00 3.00 2.00 2.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 48 6.00 3.00 3.00 3.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 49 3.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 306 253 248 251 248 241 237 234 242 238 236 235 235 237 235 235 235 236 235 235 Average 6.12 5.06 4.96 5.02 4.96 4.82 4.74 4.68 4.84 4.76 4.72 4.70 4.70 4.74 4.70 4.70 4.70 4.72 4.70 4.70 D.12. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, WEAK NEXT LINKS313 D.12 Complete results of configuration Shattered Document, weak Next links
Like Shattered Document, but with smaller weight for Next links.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 0.10 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.92
Size0 = 5.46
Size1 = 3.36 Size gain = 1.63
Cost0 = 5.46
Cost1 = 3.36 Cost gain = 1.63 314 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document, weak Next links
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 0 0 0 9 0.00 1.00 Oracle 2 30 6 0 0 0 0 6 0.00 1.00 Oracle 3 40 8 0 0 0 0 8 0.00 1.00 Oracle 4 46 7 0 0 0 0 7 0.00 1.00 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 0 0 0 0 7 0.00 1.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 0 0 0 8 0.00 1.00 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 0 0 0 9 0.00 1.00 Oracle 18 269 11 0 0 0 0 11 0.00 1.00 Oracle 19 290 6 0 0 0 0 6 0.00 1.00 Oracle 20 307 6 0 1 0 1 5 0.16 0.84 Oracle 21 366 10 0 1 0 1 9 0.10 0.90 Oracle 22 410 10 0 0 0 0 10 0.00 1.00 Oracle 23 455 15 0 0 0 0 15 0.00 1.00 Oracle 24 508 11 0 2 0 2 9 0.18 0.82 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 0 0 0 15 0.00 1.00 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 0 0 0 7 0.00 1.00 Oracle 29 542 5 0 3 0 3 2 0.60 0.40 Oracle 30 554 13 0 10 0 10 3 0.77 0.23 Oracle 31 601 11 0 0 0 0 11 0.00 1.00 Oracle 32 606 14 0 0 0 0 14 0.00 1.00 Oracle 33 616 12 0 1 0 1 11 0.09 0.91 Oracle 34 649 8 0 0 0 0 8 0.00 1.00 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 0 0 0 11 0.00 1.00 Oracle 39 786 7 0 1 0 1 6 0.14 0.86 Oracle 40 840 14 0 1 0 1 13 0.07 0.93 Oracle 41 846 8 0 0 0 0 8 0.00 1.00 Oracle 42 883 1 0 0 0 0 1 0.00 1.00 Oracle 43 890 10 0 1 0 1 9 0.10 0.90 Oracle 44 897 6 0 0 0 0 6 0.00 1.00 Oracle 45 899 9 0 0 0 0 9 0.00 1.00 Oracle 46 929 10 0 0 0 0 10 0.00 1.00 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 0 0 0 0 13 0.00 1.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 39 0 39 461 4.22 45.78 Average ··· 10.00 0.00 0.78 0.00 0.78 9.22 0.09 0.91 Micro avg ····················· 0.08 0.92 D.12. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, WEAK NEXT LINKS315
Duration Results of configuration Shattered Document, weak Next links
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 10.46 1.03 1.78 1.16 1.78 1.04 0.74 0.59 1.71 Oracle 2 7.29 1.02 2.16 1.21 2.16 1.02 1.15 0.47 2.13 Oracle 3 9.09 0.92 1.88 1.14 1.88 0.95 0.94 0.51 1.98 Oracle 4 7.98 1.02 1.79 1.14 1.79 1.02 0.77 0.57 1.76 Oracle 5 7.73 0.85 1.29 0.97 1.29 0.86 0.43 0.66 1.50 Oracle 6 12.04 0.88 2.19 1.09 2.19 0.88 1.30 0.41 2.47 Oracle 7 9.23 0.60 1.90 0.84 1.84 0.60 1.23 0.33 3.05 Oracle 8 11.19 1.00 1.09 1.02 1.02 1.09 -0.07 1.07 0.94 Oracle 9 13.29 1.82 2.26 1.90 2.26 1.87 0.39 0.83 1.21 Oracle 10 12.51 0.60 0.88 0.63 0.88 0.60 0.28 0.68 1.47 Oracle 11 12.64 1.02 1.38 1.05 1.38 1.02 0.36 0.74 1.35 Oracle 12 13.53 1.01 1.34 1.04 1.34 1.02 0.32 0.76 1.32 Oracle 13 17.04 1.02 1.50 1.13 1.50 1.05 0.45 0.70 1.44 Oracle 14 12.03 1.02 1.34 1.09 1.34 1.05 0.30 0.78 1.28 Oracle 15 8.46 1.02 1.33 1.05 1.33 1.02 0.30 0.77 1.30 Oracle 16 6.65 0.92 1.33 1.11 0.92 1.02 -0.09 1.10 0.91 Oracle 17 10.65 1.02 1.86 1.18 1.86 1.02 0.84 0.55 1.83 Oracle 18 12.82 1.02 1.73 1.16 1.73 1.30 0.42 0.76 1.32 Oracle 19 8.56 1.41 1.45 1.43 1.42 1.41 0.01 0.99 1.01 Oracle 20 10.25 1.43 2.73 1.71 2.73 1.44 1.29 0.52 1.90 Oracle 21 17.02 1.44 2.74 1.70 2.67 1.44 1.23 0.54 1.86 Oracle 22 15.70 1.43 2.61 1.57 2.61 1.46 1.15 0.56 1.78 Oracle 23 16.90 1.01 2.55 1.13 2.55 1.02 1.53 0.40 2.50 Oracle 24 14.65 0.93 2.66 1.33 2.62 0.94 1.68 0.36 2.80 Oracle 25 23.77 2.61 2.70 2.64 2.64 2.69 -0.05 1.02 0.98 Oracle 26 13.72 0.60 1.80 0.91 1.80 0.61 1.19 0.34 2.95 Oracle 27 11.51 0.60 0.95 0.88 0.95 0.95 -0.02 1.02 0.98 Oracle 28 12.05 0.93 2.30 1.72 2.23 0.93 1.30 0.41 2.41 Oracle 29 11.45 1.05 2.80 2.29 2.80 1.05 1.75 0.38 2.66 Oracle 30 32.04 1.80 2.73 2.46 2.67 1.88 0.79 0.70 1.42 Oracle 31 12.47 0.65 2.57 1.13 2.57 1.02 1.55 0.40 2.52 Oracle 32 15.82 1.01 2.49 1.13 2.49 1.02 1.47 0.41 2.43 Oracle 33 14.70 1.02 2.70 1.23 2.70 1.02 1.69 0.38 2.66 Oracle 34 8.90 1.02 1.74 1.11 1.74 1.03 0.71 0.59 1.69 Oracle 35 9.94 0.60 2.70 0.99 2.70 0.61 2.08 0.23 4.40 Oracle 36 13.26 1.00 1.07 1.02 1.04 1.03 0.01 0.99 1.01 Oracle 37 9.55 0.60 2.20 0.87 2.20 0.60 1.59 0.27 3.63 Oracle 38 10.15 0.60 2.18 0.92 2.18 0.61 1.58 0.28 3.60 Oracle 39 8.81 1.02 2.68 1.26 2.68 1.02 1.66 0.38 2.64 Oracle 40 22.17 1.43 2.70 1.59 2.70 1.44 1.26 0.53 1.88 Oracle 41 12.36 1.43 2.11 1.55 2.11 1.46 0.65 0.69 1.45 Oracle 42 2.10 2.10 2.10 2.10 2.10 2.10 0.00 1.00 1.00 Oracle 43 17.76 1.41 2.77 1.77 2.77 1.84 0.93 0.66 1.51 Oracle 44 9.09 1.02 2.24 1.52 2.24 1.48 0.77 0.66 1.52 Oracle 45 12.66 1.02 1.51 1.41 1.42 1.49 -0.07 1.05 0.95 Oracle 46 8.84 0.60 2.31 0.88 2.31 0.63 1.69 0.27 3.70 Oracle 47 9.22 0.60 1.80 0.92 1.80 0.60 1.20 0.34 2.99 Oracle 48 12.64 0.60 2.26 0.97 2.26 0.95 1.30 0.42 2.38 Oracle 49 7.80 0.60 0.96 0.78 0.94 0.96 -0.02 1.02 0.98 Oracle 50 24.37 2.67 2.80 2.71 2.68 2.70 -0.02 1.01 0.99 Total 624.84 54.02 100.95 65.58 99.80 57.84 41.97 31.07 96.13 Average 12.50 1.08 2.02 1.31 2.00 1.16 0.84 0.63 1.92 Micro avg ····················· 0.58 1.73 316 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document, weak Next links
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 1.78 1.36 1.05 1.03 1.07 1.05 1.03 1.03 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 2 2.16 1.04 1.02 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 3 1.88 1.41 1.02 0.94 1.04 0.93 0.92 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 4 1.79 1.05 1.03 1.05 1.02 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 5 1.29 1.28 0.87 0.86 0.85 0.89 0.85 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 0.86 Oracle 6 2.19 1.34 1.02 1.02 1.02 0.94 0.95 0.90 0.89 0.89 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.88 Oracle 7 1.84 1.90 0.61 0.63 0.63 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 Oracle 8 1.02 1.01 1.01 1.00 1.05 1.01 1.01 1.01 1.01 1.00 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 9 2.26 1.82 1.86 1.82 1.83 1.83 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 Oracle 10 0.88 0.61 0.61 0.60 0.61 0.63 0.60 0.60 0.60 0.60 0.64 0.65 0.60 0.60 0.60 0.61 0.60 0.60 0.63 0.60 Oracle 11 1.38 1.02 1.02 1.02 1.04 1.02 1.04 1.02 1.02 1.04 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 12 1.34 1.02 1.02 1.02 1.02 1.02 1.01 1.02 1.01 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 13 1.50 1.50 1.37 1.33 1.03 1.02 1.02 1.05 1.02 1.02 1.06 1.02 1.02 1.02 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 14 1.34 1.09 1.34 1.02 1.03 1.05 1.04 1.02 1.02 1.03 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 15 1.33 1.02 1.02 1.04 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 16 0.92 1.32 1.02 1.05 1.33 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 17 1.86 1.03 1.32 1.02 1.02 1.04 1.02 1.34 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 18 1.73 1.29 1.02 1.05 1.07 1.30 1.02 1.02 1.02 1.02 1.30 1.30 1.30 1.30 1.30 1.30 1.30 1.30 1.30 1.30 Oracle 19 1.42 1.41 1.45 1.45 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 Oracle 20 2.73 1.78 1.44 1.44 1.43 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 Oracle 21 2.67 2.74 1.44 1.46 1.45 1.45 1.44 1.47 1.47 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 1.44 Oracle 22 2.61 1.50 1.43 1.51 1.45 1.44 1.43 1.44 1.44 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 23 2.55 1.03 1.03 1.05 1.01 1.02 1.02 1.02 1.02 1.02 1.05 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 24 2.62 2.66 1.79 0.95 0.95 0.94 0.93 0.95 0.96 0.98 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 Oracle 25 2.64 2.61 2.65 2.62 2.63 2.70 2.62 2.63 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 26 1.80 0.99 0.60 0.60 0.94 0.94 0.94 0.94 0.98 0.95 0.94 0.60 0.94 0.96 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 27 0.95 0.60 0.60 0.95 0.93 0.93 0.94 0.94 0.93 0.93 0.93 0.93 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 28 2.23 2.30 1.41 1.50 1.41 2.27 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 Oracle 29 2.80 2.73 2.68 2.21 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 30 2.67 2.61 2.62 2.68 2.62 2.68 2.64 2.65 2.73 2.65 1.80 1.81 1.88 1.88 1.88 1.88 1.88 1.88 1.88 1.88 Oracle 31 2.57 0.65 1.05 1.02 1.02 1.02 1.04 1.02 1.02 1.07 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 32 2.49 1.06 1.02 1.02 1.04 1.02 1.04 1.01 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 33 2.70 1.78 1.02 1.02 1.02 1.02 1.04 1.02 1.02 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 34 1.74 1.02 1.03 1.02 1.02 1.02 1.02 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 35 2.70 0.94 0.98 0.64 0.93 0.94 0.60 0.63 0.98 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 36 1.04 1.01 1.03 1.03 1.01 1.01 1.01 1.01 1.01 1.01 1.00 1.07 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 37 2.20 0.60 0.92 0.93 0.63 0.60 0.92 0.93 0.61 0.61 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 Oracle 38 2.18 1.04 0.61 0.60 0.91 0.92 0.60 0.89 0.90 0.89 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 39 2.68 1.02 1.02 1.05 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 40 2.70 2.16 1.44 1.45 1.43 1.44 1.45 1.43 1.48 1.46 1.44 1.44 1.43 1.44 1.44 1.44 1.44 1.44 1.44 1.44 Oracle 41 2.11 1.44 1.47 1.43 1.49 1.50 1.47 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 42 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 2.10 Oracle 43 2.77 1.43 1.41 1.42 1.41 1.83 1.84 1.83 1.95 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 Oracle 44 2.24 1.02 1.48 1.45 1.44 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 Oracle 45 1.42 1.02 1.45 1.44 1.45 1.44 1.43 1.51 1.49 1.49 1.49 1.49 1.49 1.49 1.49 1.49 1.49 1.49 1.49 1.49 Oracle 46 2.31 0.61 0.63 0.95 0.95 0.61 0.60 0.60 0.95 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 Oracle 47 1.80 1.05 0.63 0.95 0.66 0.60 0.95 0.95 1.02 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 0.60 Oracle 48 2.26 2.23 1.01 0.98 0.60 0.61 0.95 0.61 0.60 0.60 0.61 0.63 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 49 0.94 0.60 0.96 0.93 0.94 0.64 0.61 0.60 0.63 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 Oracle 50 2.68 2.80 2.73 2.68 2.70 2.68 2.74 2.67 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Total 100 71 62 61 60 60 59 59 59 59 58 57 58 58 58 58 58 58 58 58 Average 2.00 1.41 1.25 1.22 1.20 1.20 1.17 1.18 1.19 1.17 1.16 1.15 1.16 1.16 1.16 1.16 1.16 1.16 1.16 1.16 D.12. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, WEAK NEXT LINKS317
Size Results of configuration Shattered Document, weak Next links
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 30 3 5 3.34 5 3 2 0.60 1.66 Oracle 2 21 3 6 3.50 6 3 3 0.50 2.00 Oracle 3 27 3 5 3.38 5 3 2 0.60 1.66 Oracle 4 23 3 5 3.29 5 3 2 0.60 1.66 Oracle 5 26 3 4 3.25 4 3 1 0.75 1.34 Oracle 6 37 3 6 3.37 6 3 3 0.50 2.00 Oracle 7 28 2 5 2.55 5 2 3 0.40 2.50 Oracle 8 33 3 3 3.00 3 3 0 1.00 1.00 Oracle 9 36 5 6 5.14 6 5 1 0.84 1.20 Oracle 10 41 2 3 2.05 3 2 1 0.66 1.50 Oracle 11 37 3 4 3.09 4 3 1 0.75 1.34 Oracle 12 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 13 49 3 4 3.27 4 3 1 0.75 1.34 Oracle 14 35 3 4 3.18 4 3 1 0.75 1.34 Oracle 15 25 3 4 3.13 4 3 1 0.75 1.34 Oracle 16 20 3 4 3.34 3 3 0 1.00 1.00 Oracle 17 31 3 5 3.45 5 3 2 0.60 1.66 Oracle 18 38 3 5 3.45 5 4 1 0.80 1.25 Oracle 19 24 4 4 4.00 4 4 0 1.00 1.00 Oracle 20 28 4 7 4.66 7 4 3 0.57 1.75 Oracle 21 46 4 7 4.60 7 4 3 0.57 1.75 Oracle 22 43 4 7 4.30 7 4 3 0.57 1.75 Oracle 23 49 3 7 3.27 7 3 4 0.43 2.34 Oracle 24 43 3 7 3.91 7 3 4 0.43 2.34 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 43 2 5 2.87 5 2 3 0.40 2.50 Oracle 27 37 2 3 2.84 3 3 0 1.00 1.00 Oracle 28 33 3 6 4.71 6 3 3 0.50 2.00 Oracle 29 30 3 7 6.00 7 3 4 0.43 2.34 Oracle 30 85 5 7 6.54 7 5 2 0.71 1.40 Oracle 31 36 2 7 3.27 7 3 4 0.43 2.34 Oracle 32 46 3 7 3.29 7 3 4 0.43 2.34 Oracle 33 42 3 7 3.50 7 3 4 0.43 2.34 Oracle 34 26 3 5 3.25 5 3 2 0.60 1.66 Oracle 35 30 2 7 3.00 7 2 5 0.29 3.50 Oracle 36 39 3 3 3.00 3 3 0 1.00 1.00 Oracle 37 30 2 6 2.73 6 2 4 0.34 3.00 Oracle 38 32 2 6 2.91 6 2 4 0.34 3.00 Oracle 39 25 3 7 3.57 7 3 4 0.43 2.34 Oracle 40 61 4 7 4.36 7 4 3 0.57 1.75 Oracle 41 34 4 6 4.25 6 4 2 0.66 1.50 Oracle 42 6 6 6 6.00 6 6 0 1.00 1.00 Oracle 43 48 4 7 4.80 7 5 2 0.71 1.40 Oracle 44 25 3 6 4.16 6 4 2 0.66 1.50 Oracle 45 35 3 4 3.89 4 4 0 1.00 1.00 Oracle 46 27 2 6 2.70 6 2 4 0.34 3.00 Oracle 47 28 2 5 2.80 5 2 3 0.40 2.50 Oracle 48 38 2 6 2.92 6 3 3 0.50 2.00 Oracle 49 25 2 3 2.50 3 3 0 1.00 1.00 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 1797 160 274 187.42 273 168 105 32.34 87.41 Average 35.94 3.20 5.48 3.75 5.46 3.36 2.10 0.65 1.75 Micro avg ····················· 0.62 1.63 318 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document, weak Next links
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 5 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 2 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 3 5 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 4 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 5 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 6 6 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 7 5 5 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 8 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 9 6 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 10 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 11 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 12 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 13 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 14 4 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 15 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 16 3 4 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 17 5 3 4 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 18 5 4 3 3 3 4 3 3 3 3 4 4 4 4 4 4 4 4 4 4 Oracle 19 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 20 7 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 21 7 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 22 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 23 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 24 7 7 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 5 3 2 2 3 3 3 3 3 3 3 2 3 3 2 2 2 2 2 2 Oracle 27 3 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 28 6 6 4 4 4 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 29 7 7 7 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 30 7 7 7 7 7 7 7 7 7 7 5 5 5 5 5 5 5 5 5 5 Oracle 31 7 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 32 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 33 7 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 34 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 35 7 3 3 2 3 3 2 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 36 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 37 6 2 3 3 2 2 3 3 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 38 6 3 2 2 3 3 2 3 3 3 2 2 2 2 2 2 2 2 2 2 Oracle 39 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 40 7 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 41 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 42 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 Oracle 43 7 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 44 6 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 45 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 46 6 2 2 3 3 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 47 5 3 2 3 2 2 3 3 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 48 6 6 3 3 2 2 3 2 2 2 2 2 3 3 3 3 3 3 3 3 Oracle 49 3 2 3 3 3 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 273 199 179 176 173 174 171 172 172 170 168 167 169 169 168 168 168 168 168 168 Average 5.46 3.98 3.58 3.52 3.46 3.48 3.42 3.44 3.44 3.40 3.36 3.34 3.38 3.38 3.36 3.36 3.36 3.36 3.36 3.36 D.12. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, WEAK NEXT LINKS319
Cost Results of configuration Shattered Document, weak Next links
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 30.00 3.00 5.00 3.34 5.00 3.00 2.00 0.60 1.66 Oracle 2 21.00 3.00 6.00 3.50 6.00 3.00 3.00 0.50 2.00 Oracle 3 27.00 3.00 5.00 3.38 5.00 3.00 2.00 0.60 1.66 Oracle 4 23.00 3.00 5.00 3.29 5.00 3.00 2.00 0.60 1.66 Oracle 5 26.00 3.00 4.00 3.25 4.00 3.00 1.00 0.75 1.34 Oracle 6 37.00 3.00 6.00 3.37 6.00 3.00 3.00 0.50 2.00 Oracle 7 28.00 2.00 5.00 2.55 5.00 2.00 3.00 0.40 2.50 Oracle 8 33.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 9 36.00 5.00 6.00 5.14 6.00 5.00 1.00 0.84 1.20 Oracle 10 41.00 2.00 3.00 2.05 3.00 2.00 1.00 0.66 1.50 Oracle 11 37.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 12 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 13 49.00 3.00 4.00 3.27 4.00 3.00 1.00 0.75 1.34 Oracle 14 35.00 3.00 4.00 3.18 4.00 3.00 1.00 0.75 1.34 Oracle 15 25.00 3.00 4.00 3.13 4.00 3.00 1.00 0.75 1.34 Oracle 16 20.00 3.00 4.00 3.34 3.00 3.00 0.00 1.00 1.00 Oracle 17 31.00 3.00 5.00 3.45 5.00 3.00 2.00 0.60 1.66 Oracle 18 38.00 3.00 5.00 3.45 5.00 4.00 1.00 0.80 1.25 Oracle 19 24.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 20 28.00 4.00 7.00 4.66 7.00 4.00 3.00 0.57 1.75 Oracle 21 46.00 4.00 7.00 4.60 7.00 4.00 3.00 0.57 1.75 Oracle 22 43.00 4.00 7.00 4.30 7.00 4.00 3.00 0.57 1.75 Oracle 23 49.00 3.00 7.00 3.27 7.00 3.00 4.00 0.43 2.34 Oracle 24 43.00 3.00 7.00 3.91 7.00 3.00 4.00 0.43 2.34 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 43.00 2.00 5.00 2.87 5.00 2.00 3.00 0.40 2.50 Oracle 27 37.00 2.00 3.00 2.84 3.00 3.00 0.00 1.00 1.00 Oracle 28 33.00 3.00 6.00 4.71 6.00 3.00 3.00 0.50 2.00 Oracle 29 30.00 3.00 7.00 6.00 7.00 3.00 4.00 0.43 2.34 Oracle 30 85.00 5.00 7.00 6.54 7.00 5.00 2.00 0.71 1.40 Oracle 31 36.00 2.00 7.00 3.27 7.00 3.00 4.00 0.43 2.34 Oracle 32 46.00 3.00 7.00 3.29 7.00 3.00 4.00 0.43 2.34 Oracle 33 42.00 3.00 7.00 3.50 7.00 3.00 4.00 0.43 2.34 Oracle 34 26.00 3.00 5.00 3.25 5.00 3.00 2.00 0.60 1.66 Oracle 35 30.00 2.00 7.00 3.00 7.00 2.00 5.00 0.29 3.50 Oracle 36 39.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 37 30.00 2.00 6.00 2.73 6.00 2.00 4.00 0.34 3.00 Oracle 38 32.00 2.00 6.00 2.91 6.00 2.00 4.00 0.34 3.00 Oracle 39 25.00 3.00 7.00 3.57 7.00 3.00 4.00 0.43 2.34 Oracle 40 61.00 4.00 7.00 4.36 7.00 4.00 3.00 0.57 1.75 Oracle 41 34.00 4.00 6.00 4.25 6.00 4.00 2.00 0.66 1.50 Oracle 42 6.00 6.00 6.00 6.00 6.00 6.00 0.00 1.00 1.00 Oracle 43 48.00 4.00 7.00 4.80 7.00 5.00 2.00 0.71 1.40 Oracle 44 25.00 3.00 6.00 4.16 6.00 4.00 2.00 0.66 1.50 Oracle 45 35.00 3.00 4.00 3.89 4.00 4.00 0.00 1.00 1.00 Oracle 46 27.00 2.00 6.00 2.70 6.00 2.00 4.00 0.34 3.00 Oracle 47 28.00 2.00 5.00 2.80 5.00 2.00 3.00 0.40 2.50 Oracle 48 38.00 2.00 6.00 2.92 6.00 3.00 3.00 0.50 2.00 Oracle 49 25.00 2.00 3.00 2.50 3.00 3.00 0.00 1.00 1.00 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 1797.00 160.00 274.00 187.42 273.00 168.00 105.00 32.34 87.41 Average 35.94 3.20 5.48 3.75 5.46 3.36 2.10 0.65 1.75 Micro avg ····················· 0.62 1.63 320 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document, weak Next links
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 5.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 2 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 3 5.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 4 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 5 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 6 6.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 7 5.00 5.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 8 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 9 6.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 10 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 11 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 12 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 13 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 14 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 15 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 16 3.00 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 17 5.00 3.00 4.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 18 5.00 4.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 19 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 20 7.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 21 7.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 22 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 23 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 24 7.00 7.00 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 5.00 3.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 27 3.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 28 6.00 6.00 4.00 4.00 4.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 29 7.00 7.00 7.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 30 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 31 7.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 32 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 33 7.00 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 34 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 35 7.00 3.00 3.00 2.00 3.00 3.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 36 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 37 6.00 2.00 3.00 3.00 2.00 2.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 38 6.00 3.00 2.00 2.00 3.00 3.00 2.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 39 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 40 7.00 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 41 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 42 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 43 7.00 4.00 4.00 4.00 4.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 44 6.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 45 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 46 6.00 2.00 2.00 3.00 3.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 47 5.00 3.00 2.00 3.00 2.00 2.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 48 6.00 6.00 3.00 3.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 49 3.00 2.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 273 199 179 176 173 174 171 172 172 170 168 167 169 169 168 168 168 168 168 168 Average 5.46 3.98 3.58 3.52 3.46 3.48 3.42 3.44 3.44 3.40 3.36 3.34 3.38 3.38 3.36 3.36 3.36 3.36 3.36 3.36 D.13. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH STRONG LEGACY321 D.13 Complete results of configuration Shattered Document with Strong Legacy
The Shattered Documents approach, but with original links weighting more. Using functions Super Page, Choose, Recentre from Kasim2.Activation. Using the model of the ARM on Kasim2.Documents.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 10.00 Weight of heading-to-paragraph Child links = 10.00 Weight of heading-to-heading Child links = 10.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.92
Size0 = 5.70
Size1 = 3.68 Size gain = 1.55
Cost0 = 5.70
Cost1 = 3.68 Cost gain = 1.55 322 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Strong Legacy
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 2 0 2 7 0.22 0.78 Oracle 2 30 6 0 0 0 0 6 0.00 1.00 Oracle 3 40 8 0 0 0 0 8 0.00 1.00 Oracle 4 46 7 0 1 0 1 6 0.14 0.86 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 0 0 0 0 7 0.00 1.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 0 0 0 8 0.00 1.00 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 0 0 0 9 0.00 1.00 Oracle 18 269 11 0 0 0 0 11 0.00 1.00 Oracle 19 290 6 0 0 0 0 6 0.00 1.00 Oracle 20 307 6 0 0 0 0 6 0.00 1.00 Oracle 21 366 10 0 1 0 1 9 0.10 0.90 Oracle 22 410 10 0 2 0 2 8 0.20 0.80 Oracle 23 455 15 0 0 0 0 15 0.00 1.00 Oracle 24 508 11 0 0 0 0 11 0.00 1.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 6 0 6 9 0.40 0.60 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 0 0 0 7 0.00 1.00 Oracle 29 542 5 0 1 0 1 4 0.20 0.80 Oracle 30 554 13 0 0 0 0 13 0.00 1.00 Oracle 31 601 11 0 0 0 0 11 0.00 1.00 Oracle 32 606 14 0 0 0 0 14 0.00 1.00 Oracle 33 616 12 0 0 0 0 12 0.00 1.00 Oracle 34 649 8 0 1 0 1 7 0.13 0.88 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 0 0 0 11 0.00 1.00 Oracle 39 786 7 0 7 0 7 0 1.00 0.00 Oracle 40 840 14 0 1 0 1 13 0.07 0.93 Oracle 41 846 8 0 0 0 0 8 0.00 1.00 Oracle 42 883 1 0 0 0 0 1 0.00 1.00 Oracle 43 890 10 0 1 0 1 9 0.10 0.90 Oracle 44 897 6 0 0 0 0 6 0.00 1.00 Oracle 45 899 9 0 0 0 0 9 0.00 1.00 Oracle 46 929 10 0 0 0 0 10 0.00 1.00 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 0 0 0 0 13 0.00 1.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 41 0 41 459 4.56 45.44 Average ··· 10.00 0.00 0.82 0.00 0.82 9.18 0.09 0.91 Micro avg ····················· 0.08 0.92 D.13. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH STRONG LEGACY323
Duration Results of configuration Shattered Document with Strong Legacy
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 15.17 1.36 2.82 1.69 2.77 1.36 1.41 0.49 2.04 Oracle 2 8.77 1.05 2.20 1.46 2.20 1.07 1.13 0.49 2.05 Oracle 3 11.37 1.04 1.95 1.42 1.95 1.04 0.91 0.53 1.88 Oracle 4 11.54 1.04 2.80 1.65 2.80 1.04 1.76 0.38 2.70 Oracle 5 8.91 0.88 1.48 1.12 1.48 1.28 0.20 0.87 1.16 Oracle 6 17.65 1.33 2.67 1.60 1.80 1.33 0.47 0.73 1.36 Oracle 7 15.69 1.02 2.34 1.43 1.85 1.03 0.82 0.55 1.80 Oracle 8 26.12 1.45 2.76 2.38 1.45 2.30 -0.85 1.59 0.63 Oracle 9 11.45 1.33 2.57 1.63 2.57 1.34 1.23 0.52 1.92 Oracle 10 24.72 1.03 1.76 1.23 1.33 1.03 0.30 0.77 1.29 Oracle 11 13.24 1.02 1.46 1.10 1.41 1.03 0.38 0.73 1.37 Oracle 12 15.54 1.02 1.47 1.20 1.36 1.02 0.34 0.75 1.33 Oracle 13 21.46 1.03 2.65 1.43 2.65 1.03 1.62 0.39 2.56 Oracle 14 17.19 1.32 2.32 1.56 1.53 1.37 0.16 0.89 1.12 Oracle 15 5.25 0.61 0.94 0.66 0.94 0.61 0.33 0.66 1.53 Oracle 16 8.15 1.34 1.38 1.36 1.34 1.37 -0.02 1.02 0.98 Oracle 17 12.31 1.03 2.13 1.37 2.13 1.34 0.80 0.63 1.60 Oracle 18 13.82 1.04 1.73 1.26 1.73 1.04 0.70 0.60 1.67 Oracle 19 8.70 1.44 1.48 1.45 1.45 1.45 -0.01 1.01 0.99 Oracle 20 9.98 1.45 2.66 1.66 2.66 1.46 1.20 0.55 1.81 Oracle 21 17.39 1.45 2.80 1.74 2.80 1.47 1.34 0.52 1.91 Oracle 22 18.38 1.45 2.74 1.84 2.73 1.48 1.26 0.54 1.85 Oracle 23 30.59 1.03 2.54 2.04 2.25 1.05 1.20 0.46 2.15 Oracle 24 11.94 0.94 2.27 1.09 2.27 0.95 1.31 0.42 2.38 Oracle 25 23.98 2.65 2.70 2.66 2.66 2.65 0.02 0.99 1.01 Oracle 26 26.52 1.02 2.70 1.77 2.66 1.03 1.63 0.39 2.58 Oracle 27 12.74 0.93 1.38 0.98 1.38 0.95 0.43 0.69 1.46 Oracle 28 9.88 0.95 2.23 1.41 1.45 0.95 0.50 0.66 1.53 Oracle 29 8.09 1.03 2.71 1.62 2.71 1.03 1.68 0.38 2.63 Oracle 30 11.91 0.62 1.70 0.91 1.70 0.62 1.07 0.37 2.73 Oracle 31 14.26 1.03 2.59 1.30 2.59 1.34 1.24 0.52 1.93 Oracle 32 17.09 1.02 2.52 1.22 2.52 1.02 1.50 0.41 2.47 Oracle 33 14.02 1.02 2.63 1.17 2.63 1.03 1.60 0.39 2.55 Oracle 34 10.59 1.02 2.89 1.32 2.89 1.03 1.86 0.36 2.81 Oracle 35 13.59 0.61 2.73 1.36 2.73 0.61 2.12 0.23 4.47 Oracle 36 17.15 1.02 1.47 1.32 1.44 1.02 0.43 0.70 1.42 Oracle 37 14.60 1.03 2.58 1.33 2.58 1.03 1.55 0.40 2.49 Oracle 38 13.43 0.62 2.15 1.22 2.15 0.62 1.53 0.29 3.48 Oracle 39 19.35 2.72 2.87 2.77 2.87 2.73 0.14 0.95 1.05 Oracle 40 24.27 1.45 2.75 1.73 2.75 1.46 1.29 0.53 1.88 Oracle 41 13.56 1.46 2.19 1.70 2.19 1.48 0.71 0.67 1.48 Oracle 42 2.28 2.28 2.28 2.28 2.28 2.28 0.00 1.00 1.00 Oracle 43 19.80 1.87 2.87 1.98 2.87 1.87 1.00 0.65 1.53 Oracle 44 11.94 1.84 2.69 1.99 2.69 1.85 0.84 0.69 1.45 Oracle 45 15.45 1.45 1.86 1.72 1.84 1.86 -0.02 1.02 0.98 Oracle 46 10.93 0.63 2.77 1.09 2.77 0.95 1.81 0.34 2.90 Oracle 47 10.73 0.62 1.81 1.07 1.81 0.96 0.85 0.53 1.89 Oracle 48 15.73 0.61 2.68 1.21 2.27 0.96 1.30 0.43 2.34 Oracle 49 9.53 0.94 0.99 0.95 0.95 0.94 0.01 0.99 1.01 Oracle 50 24.59 2.69 2.85 2.73 2.72 2.70 0.02 0.99 1.01 Total 741.37 60.82 113.48 76.17 107.51 64.43 43.09 31.64 92.19 Average 14.83 1.22 2.27 1.52 2.15 1.29 0.86 0.63 1.84 Micro avg ····················· 0.60 1.67 324 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Strong Legacy
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 2.77 1.39 2.82 1.36 1.36 1.37 1.37 1.37 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 Oracle 2 2.20 1.48 1.48 1.48 1.05 1.07 1.07 1.07 1.07 1.07 1.07 1.07 1.07 1.07 1.07 1.07 1.07 1.07 1.07 1.07 Oracle 3 1.95 1.39 1.37 1.38 1.38 1.45 1.42 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 4 2.80 1.80 1.98 1.49 1.05 1.38 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 5 1.48 1.31 0.89 1.30 0.88 0.88 0.88 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 1.28 Oracle 6 1.80 1.33 2.67 1.46 1.91 1.79 1.34 1.34 1.35 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 Oracle 7 1.85 2.34 1.40 1.47 1.41 1.39 1.38 1.03 1.02 1.38 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 8 1.45 2.29 2.28 2.71 2.76 2.71 2.70 2.35 2.29 2.29 2.30 2.30 2.30 2.30 2.30 2.30 2.30 2.30 2.30 2.30 Oracle 9 2.57 1.33 1.76 1.75 1.34 1.37 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 Oracle 10 1.33 1.04 1.03 1.03 1.03 1.03 1.03 1.03 1.73 1.45 1.49 1.45 1.73 1.76 1.05 1.32 1.05 1.04 1.05 1.03 Oracle 11 1.41 1.04 1.02 1.46 1.04 1.04 1.06 1.04 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 12 1.36 1.03 1.04 1.46 1.47 1.45 1.45 1.09 1.05 1.04 1.04 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 13 2.65 1.38 1.37 1.36 1.34 1.35 1.35 1.78 1.36 1.36 1.37 1.36 1.38 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 14 1.53 2.32 1.36 1.75 1.77 1.77 1.34 1.34 1.33 1.32 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 Oracle 15 0.94 0.62 0.61 0.62 0.62 0.62 0.62 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 16 1.34 1.36 1.37 1.34 1.38 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 Oracle 17 2.13 1.38 1.35 1.03 1.34 1.08 1.34 1.33 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 Oracle 18 1.73 1.30 1.31 1.38 1.32 1.04 1.05 1.30 1.05 1.30 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 19 1.45 1.45 1.45 1.44 1.48 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 Oracle 20 2.66 1.46 1.45 1.50 1.45 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 21 2.80 2.79 1.45 1.53 1.47 1.48 1.48 1.45 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 Oracle 22 2.73 2.74 2.60 1.48 1.45 1.48 1.47 1.47 1.47 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 Oracle 23 2.25 2.27 2.25 2.26 2.28 2.26 2.25 2.30 2.29 2.26 2.27 2.54 1.04 1.03 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 24 2.27 0.96 0.96 0.96 0.95 0.95 0.95 1.09 0.98 0.94 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 25 2.66 2.67 2.66 2.66 2.70 2.70 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Oracle 26 2.66 2.65 2.70 2.66 2.66 2.70 2.29 1.03 1.03 1.02 1.02 1.02 1.03 1.02 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 27 1.38 0.94 0.95 0.95 0.95 0.95 0.95 0.96 0.95 0.94 0.93 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 28 1.45 1.44 1.43 1.45 2.23 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 29 2.71 2.25 1.04 1.05 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 30 1.70 0.88 0.88 0.88 0.88 0.87 0.88 0.91 0.87 0.87 0.86 0.86 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 31 2.59 1.03 1.04 1.03 1.04 1.07 1.36 1.36 1.04 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 Oracle 32 2.52 1.32 1.04 1.03 1.04 1.32 1.32 1.05 1.05 1.03 1.30 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 33 2.63 1.03 1.03 1.04 1.04 1.04 1.04 1.05 1.02 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 34 2.89 1.38 1.05 1.09 1.05 1.09 1.02 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 35 2.73 1.38 1.38 1.39 1.38 1.38 1.38 1.03 0.94 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 36 1.44 1.45 1.45 1.47 1.44 1.43 1.45 1.47 1.44 1.02 1.02 1.05 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 37 2.58 1.04 1.36 1.36 1.04 1.41 1.37 1.05 1.04 1.35 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 38 2.15 1.04 1.03 1.33 1.04 1.32 1.34 1.34 1.33 0.91 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 39 2.87 2.75 2.75 2.76 2.78 2.72 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 Oracle 40 2.75 2.20 1.80 1.80 1.45 1.85 1.47 1.48 1.46 1.81 1.47 1.47 1.79 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 41 2.19 1.73 1.46 1.47 1.73 1.76 1.75 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 Oracle 42 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 2.28 Oracle 43 2.87 1.87 1.87 1.87 1.88 1.88 1.88 1.95 1.88 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 1.87 Oracle 44 2.69 1.85 1.85 1.85 1.84 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 Oracle 45 1.84 1.46 1.84 1.85 1.84 1.48 1.45 1.84 1.86 1.86 1.86 1.86 1.86 1.86 1.86 1.86 1.86 1.86 1.86 1.86 Oracle 46 2.77 1.38 1.03 0.97 0.97 0.63 0.63 0.63 0.98 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 47 1.81 1.03 1.39 1.38 0.62 0.65 0.98 0.98 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 Oracle 48 2.27 2.68 0.97 0.98 0.98 0.98 1.39 1.05 0.96 0.96 0.61 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 Oracle 49 0.95 0.95 0.95 0.98 0.95 0.99 0.95 0.95 0.95 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 Oracle 50 2.72 2.73 2.71 2.72 2.85 2.77 2.70 2.69 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Total 108 81 77 75 73 72 71 69 68 68 67 67 66 65 64 65 64 64 64 64 Average 2.15 1.63 1.55 1.51 1.46 1.45 1.42 1.38 1.37 1.37 1.34 1.34 1.31 1.30 1.29 1.30 1.29 1.29 1.29 1.29 D.13. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH STRONG LEGACY325
Size Results of configuration Shattered Document with Strong Legacy
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 42 4 7 4.66 7 4 3 0.57 1.75 Oracle 2 24 3 6 4.00 6 3 3 0.50 2.00 Oracle 3 32 3 5 4.00 5 3 2 0.60 1.66 Oracle 4 31 3 7 4.43 7 3 4 0.43 2.34 Oracle 5 28 3 4 3.50 4 4 0 1.00 1.00 Oracle 6 50 4 7 4.55 5 4 1 0.80 1.25 Oracle 7 44 3 6 4.00 5 3 2 0.60 1.66 Oracle 8 68 4 7 6.18 4 6 -2 1.50 0.66 Oracle 9 33 4 7 4.71 7 4 3 0.57 1.75 Oracle 10 71 3 5 3.55 4 3 1 0.75 1.34 Oracle 11 38 3 4 3.16 4 3 1 0.75 1.34 Oracle 12 44 3 4 3.38 4 3 1 0.75 1.34 Oracle 13 61 3 6 4.07 6 3 3 0.50 2.00 Oracle 14 49 4 6 4.45 4 4 0 1.00 1.00 Oracle 15 17 2 3 2.13 3 2 1 0.66 1.50 Oracle 16 24 4 4 4.00 4 4 0 1.00 1.00 Oracle 17 35 3 5 3.89 5 4 1 0.80 1.25 Oracle 18 41 3 5 3.73 5 3 2 0.60 1.66 Oracle 19 24 4 4 4.00 4 4 0 1.00 1.00 Oracle 20 27 4 7 4.50 7 4 3 0.57 1.75 Oracle 21 46 4 7 4.60 7 4 3 0.57 1.75 Oracle 22 49 4 7 4.90 7 4 3 0.57 1.75 Oracle 23 82 3 7 5.47 6 3 3 0.50 2.00 Oracle 24 36 3 6 3.27 6 3 3 0.50 2.00 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 72 3 7 4.80 7 3 4 0.43 2.34 Oracle 27 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 28 28 3 6 4.00 4 3 1 0.75 1.34 Oracle 29 22 3 7 4.40 7 3 4 0.43 2.34 Oracle 30 40 2 5 3.08 5 2 3 0.40 2.50 Oracle 31 41 3 7 3.73 7 4 3 0.57 1.75 Oracle 32 50 3 7 3.57 7 3 4 0.43 2.34 Oracle 33 40 3 7 3.34 7 3 4 0.43 2.34 Oracle 34 29 3 7 3.63 7 3 4 0.43 2.34 Oracle 35 39 2 7 3.90 7 2 5 0.29 3.50 Oracle 36 48 3 4 3.70 4 3 1 0.75 1.34 Oracle 37 42 3 7 3.82 7 3 4 0.43 2.34 Oracle 38 40 2 6 3.63 6 2 4 0.34 3.00 Oracle 39 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 40 66 4 7 4.71 7 4 3 0.57 1.75 Oracle 41 38 4 6 4.75 6 4 2 0.66 1.50 Oracle 42 6 6 6 6.00 6 6 0 1.00 1.00 Oracle 43 52 5 7 5.20 7 5 2 0.71 1.40 Oracle 44 32 5 7 5.34 7 5 2 0.71 1.40 Oracle 45 42 4 5 4.66 5 5 0 1.00 1.00 Oracle 46 32 2 7 3.20 7 3 4 0.43 2.34 Oracle 47 32 2 5 3.20 5 3 2 0.60 1.66 Oracle 48 46 2 7 3.54 6 3 3 0.50 2.00 Oracle 49 30 3 3 3.00 3 3 0 1.00 1.00 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 2078 175 298 212.40 285 184 101 33.71 83.55 Average 41.56 3.50 5.96 4.25 5.70 3.68 2.02 0.67 1.67 Micro avg ····················· 0.65 1.55 326 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Strong Legacy
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7 4 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 2 6 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 3 5 4 4 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 4 7 5 5 4 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 5 4 4 3 4 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 6 5 4 7 4 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 7 5 6 4 4 4 4 4 3 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 8 4 6 6 7 7 7 7 6 6 6 6 6 6 6 6 6 6 6 6 6 Oracle 9 7 4 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 10 4 3 3 3 3 3 3 3 5 4 4 4 5 5 3 4 3 3 3 3 Oracle 11 4 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 12 4 3 3 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 13 6 4 4 4 4 4 4 5 4 4 4 4 4 3 3 3 3 3 3 3 Oracle 14 4 6 4 5 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 15 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 16 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 17 5 4 4 3 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 18 5 4 4 4 4 3 3 4 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 19 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 20 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 21 7 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 22 7 7 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 23 6 6 6 6 6 6 6 6 6 6 6 7 3 3 3 3 3 3 3 3 Oracle 24 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 7 7 7 7 7 7 6 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 27 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 28 4 4 4 4 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 29 7 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 30 5 3 3 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 Oracle 31 7 3 3 3 3 3 4 4 3 4 4 4 4 4 4 4 4 4 4 4 Oracle 32 7 4 3 3 3 4 4 3 3 3 4 3 3 3 3 3 3 3 3 3 Oracle 33 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 34 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 35 7 4 4 4 4 4 4 3 3 2 2 2 2 2 2 2 2 2 2 2 Oracle 36 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 Oracle 37 7 3 4 4 3 4 4 3 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 38 6 3 3 4 3 4 4 4 4 3 2 2 2 2 2 2 2 2 2 2 Oracle 39 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 40 7 6 5 5 4 5 4 4 4 5 4 4 5 4 4 4 4 4 4 4 Oracle 41 6 5 4 4 5 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 42 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 Oracle 43 7 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 44 7 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 45 5 4 5 5 5 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 46 7 4 3 3 3 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 47 5 3 4 4 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 48 6 7 3 3 3 3 4 3 3 3 2 3 3 3 3 3 3 3 3 3 Oracle 49 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 285 225 215 210 205 203 202 194 194 195 190 191 188 186 184 185 184 184 184 184 Average 5.70 4.50 4.30 4.20 4.10 4.06 4.04 3.88 3.88 3.90 3.80 3.82 3.76 3.72 3.68 3.70 3.68 3.68 3.68 3.68 D.13. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH STRONG LEGACY327
Cost Results of configuration Shattered Document with Strong Legacy
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 42.00 4.00 7.00 4.66 7.00 4.00 3.00 0.57 1.75 Oracle 2 24.00 3.00 6.00 4.00 6.00 3.00 3.00 0.50 2.00 Oracle 3 32.00 3.00 5.00 4.00 5.00 3.00 2.00 0.60 1.66 Oracle 4 31.00 3.00 7.00 4.43 7.00 3.00 4.00 0.43 2.34 Oracle 5 28.00 3.00 4.00 3.50 4.00 4.00 0.00 1.00 1.00 Oracle 6 50.00 4.00 7.00 4.55 5.00 4.00 1.00 0.80 1.25 Oracle 7 44.00 3.00 6.00 4.00 5.00 3.00 2.00 0.60 1.66 Oracle 8 68.00 4.00 7.00 6.18 4.00 6.00 -2.00 1.50 0.66 Oracle 9 33.00 4.00 7.00 4.71 7.00 4.00 3.00 0.57 1.75 Oracle 10 71.00 3.00 5.00 3.55 4.00 3.00 1.00 0.75 1.34 Oracle 11 38.00 3.00 4.00 3.16 4.00 3.00 1.00 0.75 1.34 Oracle 12 44.00 3.00 4.00 3.38 4.00 3.00 1.00 0.75 1.34 Oracle 13 61.00 3.00 6.00 4.07 6.00 3.00 3.00 0.50 2.00 Oracle 14 49.00 4.00 6.00 4.45 4.00 4.00 0.00 1.00 1.00 Oracle 15 17.00 2.00 3.00 2.13 3.00 2.00 1.00 0.66 1.50 Oracle 16 24.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 17 35.00 3.00 5.00 3.89 5.00 4.00 1.00 0.80 1.25 Oracle 18 41.00 3.00 5.00 3.73 5.00 3.00 2.00 0.60 1.66 Oracle 19 24.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 20 27.00 4.00 7.00 4.50 7.00 4.00 3.00 0.57 1.75 Oracle 21 46.00 4.00 7.00 4.60 7.00 4.00 3.00 0.57 1.75 Oracle 22 49.00 4.00 7.00 4.90 7.00 4.00 3.00 0.57 1.75 Oracle 23 82.00 3.00 7.00 5.47 6.00 3.00 3.00 0.50 2.00 Oracle 24 36.00 3.00 6.00 3.27 6.00 3.00 3.00 0.50 2.00 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 72.00 3.00 7.00 4.80 7.00 3.00 4.00 0.43 2.34 Oracle 27 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 28 28.00 3.00 6.00 4.00 4.00 3.00 1.00 0.75 1.34 Oracle 29 22.00 3.00 7.00 4.40 7.00 3.00 4.00 0.43 2.34 Oracle 30 40.00 2.00 5.00 3.08 5.00 2.00 3.00 0.40 2.50 Oracle 31 41.00 3.00 7.00 3.73 7.00 4.00 3.00 0.57 1.75 Oracle 32 50.00 3.00 7.00 3.57 7.00 3.00 4.00 0.43 2.34 Oracle 33 40.00 3.00 7.00 3.34 7.00 3.00 4.00 0.43 2.34 Oracle 34 29.00 3.00 7.00 3.63 7.00 3.00 4.00 0.43 2.34 Oracle 35 39.00 2.00 7.00 3.90 7.00 2.00 5.00 0.29 3.50 Oracle 36 48.00 3.00 4.00 3.70 4.00 3.00 1.00 0.75 1.34 Oracle 37 42.00 3.00 7.00 3.82 7.00 3.00 4.00 0.43 2.34 Oracle 38 40.00 2.00 6.00 3.63 6.00 2.00 4.00 0.34 3.00 Oracle 39 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 40 66.00 4.00 7.00 4.71 7.00 4.00 3.00 0.57 1.75 Oracle 41 38.00 4.00 6.00 4.75 6.00 4.00 2.00 0.66 1.50 Oracle 42 6.00 6.00 6.00 6.00 6.00 6.00 0.00 1.00 1.00 Oracle 43 52.00 5.00 7.00 5.20 7.00 5.00 2.00 0.71 1.40 Oracle 44 32.00 5.00 7.00 5.34 7.00 5.00 2.00 0.71 1.40 Oracle 45 42.00 4.00 5.00 4.66 5.00 5.00 0.00 1.00 1.00 Oracle 46 32.00 2.00 7.00 3.20 7.00 3.00 4.00 0.43 2.34 Oracle 47 32.00 2.00 5.00 3.20 5.00 3.00 2.00 0.60 1.66 Oracle 48 46.00 2.00 7.00 3.54 6.00 3.00 3.00 0.50 2.00 Oracle 49 30.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 2078.00 175.00 298.00 212.40 285.00 184.00 101.00 33.71 83.55 Average 41.56 3.50 5.96 4.25 5.70 3.68 2.02 0.67 1.67 Micro avg ····················· 0.65 1.55 328 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Strong Legacy
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7.00 4.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 2 6.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 3 5.00 4.00 4.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 4 7.00 5.00 5.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 5 4.00 4.00 3.00 4.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 6 5.00 4.00 7.00 4.00 5.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 7 5.00 6.00 4.00 4.00 4.00 4.00 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 8 4.00 6.00 6.00 7.00 7.00 7.00 7.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 9 7.00 4.00 5.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 10 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 5.00 4.00 4.00 4.00 5.00 5.00 3.00 4.00 3.00 3.00 3.00 3.00 Oracle 11 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 12 4.00 3.00 3.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 13 6.00 4.00 4.00 4.00 4.00 4.00 4.00 5.00 4.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 14 4.00 6.00 4.00 5.00 5.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 15 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 16 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 17 5.00 4.00 4.00 3.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 18 5.00 4.00 4.00 4.00 4.00 3.00 3.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 19 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 20 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 21 7.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 22 7.00 7.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 23 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 24 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 7.00 7.00 7.00 7.00 7.00 7.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 27 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 28 4.00 4.00 4.00 4.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 29 7.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 30 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 31 7.00 3.00 3.00 3.00 3.00 3.00 4.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 32 7.00 4.00 3.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 33 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 34 7.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 35 7.00 4.00 4.00 4.00 4.00 4.00 4.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 36 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 37 7.00 3.00 4.00 4.00 3.00 4.00 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 38 6.00 3.00 3.00 4.00 3.00 4.00 4.00 4.00 4.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 39 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 40 7.00 6.00 5.00 5.00 4.00 5.00 4.00 4.00 4.00 5.00 4.00 4.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 41 6.00 5.00 4.00 4.00 5.00 5.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 42 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 43 7.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 44 7.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 45 5.00 4.00 5.00 5.00 5.00 4.00 4.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 46 7.00 4.00 3.00 3.00 3.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 47 5.00 3.00 4.00 4.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 48 6.00 7.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 49 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 285 225 215 210 205 203 202 194 194 195 190 191 188 186 184 185 184 184 184 184 Average 5.70 4.50 4.30 4.20 4.10 4.06 4.04 3.88 3.88 3.90 3.80 3.82 3.76 3.72 3.68 3.70 3.68 3.68 3.68 3.68 D.14. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, DENSE START329 D.14 Complete results of configuration Shattered Document, dense start
Like Shattered Document, but with all atoms connected for starting state.Each atoms connects to every other atom.Uniform connection weight.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 0.00 Weight of heading-to-paragraph Child links = 0.00 Weight of heading-to-heading Child links = 0.00 Residual weight (all links) = 1.00
Principal results: Cost evolution:
Success Rate = 0.05
Size0 = 6.86
Size1 = 6.70 Size gain = 1.02
Cost0 = 6.86
Cost1 = 6.70 Cost gain = 1.02 330 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document, dense start
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 9 0 9 0 1.00 0.00 Oracle 2 30 6 0 6 0 6 0 1.00 0.00 Oracle 3 40 8 0 0 0 0 8 0.00 1.00 Oracle 4 46 7 0 7 0 7 0 1.00 0.00 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 11 0 11 0 1.00 0.00 Oracle 7 108 11 0 11 0 11 0 1.00 0.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 0 7 0 7 0 1.00 0.00 Oracle 10 130 20 0 20 0 20 0 1.00 0.00 Oracle 11 138 12 0 12 0 12 0 1.00 0.00 Oracle 12 157 13 0 13 0 13 0 1.00 0.00 Oracle 13 171 15 0 15 0 15 0 1.00 0.00 Oracle 14 173 11 0 11 0 11 0 1.00 0.00 Oracle 15 241 8 0 8 0 8 0 1.00 0.00 Oracle 16 265 6 0 6 0 6 0 1.00 0.00 Oracle 17 266 9 0 9 0 9 0 1.00 0.00 Oracle 18 269 11 0 11 0 11 0 1.00 0.00 Oracle 19 290 6 0 6 0 6 0 1.00 0.00 Oracle 20 307 6 0 6 0 6 0 1.00 0.00 Oracle 21 366 10 0 10 0 10 0 1.00 0.00 Oracle 22 410 10 0 10 0 10 0 1.00 0.00 Oracle 23 455 15 0 15 0 15 0 1.00 0.00 Oracle 24 508 11 0 11 0 11 0 1.00 0.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 15 0 15 0 1.00 0.00 Oracle 27 516 13 0 13 0 13 0 1.00 0.00 Oracle 28 521 7 0 7 0 7 0 1.00 0.00 Oracle 29 542 5 0 5 0 5 0 1.00 0.00 Oracle 30 554 13 0 13 0 13 0 1.00 0.00 Oracle 31 601 11 0 11 0 11 0 1.00 0.00 Oracle 32 606 14 0 14 0 14 0 1.00 0.00 Oracle 33 616 12 0 12 0 12 0 1.00 0.00 Oracle 34 649 8 0 8 0 8 0 1.00 0.00 Oracle 35 723 10 0 10 0 10 0 1.00 0.00 Oracle 36 725 13 0 13 0 13 0 1.00 0.00 Oracle 37 738 11 0 11 0 11 0 1.00 0.00 Oracle 38 741 11 0 11 0 11 0 1.00 0.00 Oracle 39 786 7 0 7 0 7 0 1.00 0.00 Oracle 40 840 14 0 14 0 14 0 1.00 0.00 Oracle 41 846 8 0 8 0 8 0 1.00 0.00 Oracle 42 883 1 0 1 0 1 0 1.00 0.00 Oracle 43 890 10 0 10 0 10 0 1.00 0.00 Oracle 44 897 6 0 6 0 6 0 1.00 0.00 Oracle 45 899 9 0 9 0 9 0 1.00 0.00 Oracle 46 929 10 0 10 0 10 0 1.00 0.00 Oracle 47 936 10 0 10 0 10 0 1.00 0.00 Oracle 48 939 13 0 13 0 13 0 1.00 0.00 Oracle 49 944 10 0 10 0 10 0 1.00 0.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 473 0 473 27 47.00 3.00 Average ··· 10.00 0.00 9.46 0.00 9.46 0.54 0.94 0.06 Micro avg ····················· 0.95 0.05 D.14. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, DENSE START331
Duration Results of configuration Shattered Document, dense start
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 24.16 2.66 2.73 2.69 2.70 2.68 0.02 0.99 1.01 Oracle 2 16.13 2.66 2.75 2.69 2.66 2.67 -0.02 1.01 0.99 Oracle 3 5.86 0.61 1.30 0.73 1.30 0.62 0.68 0.48 2.10 Oracle 4 18.69 2.65 2.70 2.67 2.66 2.70 -0.04 1.02 0.98 Oracle 5 6.12 0.61 1.30 0.77 1.30 0.63 0.66 0.48 2.05 Oracle 6 29.41 2.65 2.71 2.67 2.66 2.70 -0.04 1.02 0.98 Oracle 7 29.38 2.65 2.70 2.67 2.68 2.66 0.02 0.99 1.01 Oracle 8 8.53 0.61 2.11 0.77 2.11 0.61 1.50 0.29 3.45 Oracle 9 18.71 2.65 2.72 2.67 2.66 2.66 -0.01 1.00 1.00 Oracle 10 53.65 2.65 2.80 2.68 2.69 2.66 0.02 0.99 1.01 Oracle 11 32.30 2.65 2.76 2.70 2.68 2.66 0.02 0.99 1.01 Oracle 12 34.87 2.65 2.74 2.68 2.74 2.67 0.07 0.98 1.02 Oracle 13 40.31 2.65 2.77 2.69 2.68 2.69 -0.01 1.00 1.00 Oracle 14 29.51 2.65 2.80 2.68 2.80 2.65 0.15 0.95 1.05 Oracle 15 21.46 2.66 2.71 2.68 2.71 2.69 0.02 0.99 1.01 Oracle 16 16.02 2.65 2.71 2.67 2.71 2.65 0.05 0.98 1.02 Oracle 17 24.33 2.65 2.84 2.70 2.84 2.67 0.16 0.95 1.06 Oracle 18 29.45 2.65 2.72 2.68 2.72 2.65 0.06 0.98 1.02 Oracle 19 16.20 2.65 2.77 2.70 2.65 2.77 -0.13 1.05 0.95 Oracle 20 16.25 2.65 2.77 2.71 2.68 2.65 0.03 0.99 1.01 Oracle 21 26.72 2.65 2.70 2.67 2.66 2.66 -0.01 1.00 1.00 Oracle 22 26.71 2.65 2.70 2.67 2.68 2.69 -0.01 1.00 1.00 Oracle 23 40.09 2.65 2.73 2.67 2.65 2.66 -0.01 1.00 1.00 Oracle 24 29.23 2.64 2.68 2.66 2.66 2.66 0.00 1.00 1.00 Oracle 25 24.07 2.65 2.72 2.67 2.66 2.66 0.01 1.00 1.00 Oracle 26 40.16 2.64 2.72 2.68 2.70 2.64 0.06 0.98 1.02 Oracle 27 35.05 2.66 2.78 2.70 2.70 2.66 0.03 0.98 1.02 Oracle 28 18.83 2.66 2.73 2.69 2.68 2.66 0.02 0.99 1.01 Oracle 29 13.61 2.66 2.77 2.72 2.77 2.67 0.09 0.97 1.04 Oracle 30 34.84 2.65 2.73 2.68 2.66 2.71 -0.05 1.02 0.98 Oracle 31 29.52 2.64 2.77 2.68 2.70 2.64 0.05 0.98 1.02 Oracle 32 37.56 2.65 2.77 2.68 2.66 2.66 0.00 1.00 1.00 Oracle 33 32.22 2.64 2.72 2.69 2.67 2.69 -0.02 1.01 0.99 Oracle 34 21.59 2.65 2.75 2.70 2.72 2.65 0.07 0.98 1.02 Oracle 35 26.69 2.65 2.71 2.67 2.65 2.70 -0.05 1.02 0.98 Oracle 36 34.72 2.66 2.71 2.67 2.71 2.70 0.02 0.99 1.01 Oracle 37 29.55 2.65 2.78 2.69 2.67 2.65 0.02 0.99 1.01 Oracle 38 29.52 2.66 2.75 2.68 2.70 2.70 0.01 0.99 1.01 Oracle 39 18.67 2.65 2.70 2.66 2.69 2.66 0.03 0.99 1.01 Oracle 40 37.36 2.66 2.71 2.67 2.66 2.66 -0.01 1.00 1.00 Oracle 41 21.39 2.65 2.71 2.67 2.71 2.66 0.05 0.98 1.02 Oracle 42 2.71 2.71 2.71 2.71 2.71 2.71 0.00 1.00 1.00 Oracle 43 26.88 2.65 2.77 2.69 2.77 2.72 0.05 0.98 1.02 Oracle 44 16.02 2.65 2.71 2.67 2.70 2.66 0.04 0.98 1.02 Oracle 45 24.13 2.65 2.73 2.68 2.73 2.70 0.03 0.99 1.01 Oracle 46 26.78 2.65 2.79 2.68 2.79 2.66 0.12 0.95 1.05 Oracle 47 26.97 2.66 2.80 2.70 2.71 2.66 0.05 0.98 1.02 Oracle 48 34.83 2.65 2.72 2.68 2.71 2.70 0.02 0.99 1.01 Oracle 49 26.92 2.65 2.76 2.70 2.70 2.66 0.04 0.98 1.02 Oracle 50 24.07 2.65 2.70 2.67 2.70 2.65 0.05 0.98 1.02 Total 1288.73 126.44 133.43 128.35 131.48 127.54 3.95 47.86 55.02 Average 25.77 2.53 2.67 2.57 2.63 2.55 0.08 0.96 1.10 Micro avg ····················· 0.97 1.03 332 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document, dense start
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 2.70 2.71 2.73 2.66 2.66 2.70 2.66 2.66 2.68 2.68 2.68 2.68 2.68 2.68 2.68 2.68 2.68 2.68 2.68 2.68 Oracle 2 2.66 2.75 2.66 2.69 2.68 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 3 1.30 0.61 0.61 0.62 0.62 0.87 0.64 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 4 2.66 2.65 2.65 2.70 2.66 2.66 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 5 1.30 0.87 0.61 0.61 0.88 0.61 0.62 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 0.63 Oracle 6 2.66 2.67 2.65 2.66 2.69 2.71 2.66 2.66 2.69 2.66 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 7 2.68 2.66 2.67 2.68 2.68 2.70 2.65 2.66 2.69 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 8 2.11 0.62 0.61 0.62 0.62 0.62 0.61 0.61 0.65 0.87 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 9 2.66 2.65 2.72 2.66 2.65 2.71 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 10 2.69 2.80 2.68 2.66 2.65 2.65 2.67 2.66 2.70 2.67 2.71 2.66 2.77 2.67 2.66 2.66 2.69 2.71 2.65 2.66 Oracle 11 2.68 2.75 2.68 2.75 2.65 2.67 2.68 2.66 2.70 2.66 2.76 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 12 2.74 2.66 2.70 2.65 2.70 2.66 2.66 2.73 2.68 2.69 2.66 2.69 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 13 2.68 2.69 2.67 2.71 2.66 2.65 2.66 2.71 2.66 2.73 2.73 2.77 2.66 2.66 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 14 2.80 2.65 2.80 2.69 2.66 2.65 2.66 2.66 2.66 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Oracle 15 2.71 2.71 2.67 2.66 2.70 2.66 2.66 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 16 2.71 2.65 2.66 2.68 2.66 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Oracle 17 2.84 2.67 2.71 2.65 2.66 2.68 2.69 2.75 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 18 2.72 2.71 2.66 2.68 2.66 2.70 2.66 2.66 2.70 2.66 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Oracle 19 2.65 2.73 2.66 2.66 2.72 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 Oracle 20 2.68 2.73 2.71 2.70 2.77 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Oracle 21 2.66 2.70 2.67 2.69 2.66 2.65 2.69 2.70 2.65 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 22 2.68 2.69 2.66 2.66 2.65 2.67 2.70 2.66 2.65 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 23 2.65 2.73 2.66 2.68 2.66 2.66 2.66 2.66 2.65 2.67 2.71 2.66 2.66 2.72 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 24 2.66 2.65 2.66 2.66 2.66 2.66 2.64 2.65 2.66 2.68 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 25 2.66 2.67 2.67 2.65 2.72 2.72 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 26 2.70 2.70 2.66 2.66 2.68 2.69 2.68 2.66 2.70 2.66 2.72 2.70 2.66 2.66 2.64 2.64 2.64 2.64 2.64 2.64 Oracle 27 2.70 2.66 2.72 2.73 2.68 2.78 2.66 2.70 2.68 2.66 2.71 2.70 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 28 2.68 2.73 2.68 2.73 2.66 2.70 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 29 2.77 2.76 2.66 2.75 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 30 2.66 2.65 2.65 2.66 2.66 2.66 2.72 2.65 2.71 2.72 2.73 2.66 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 31 2.70 2.66 2.71 2.66 2.66 2.68 2.66 2.72 2.77 2.66 2.64 2.64 2.64 2.64 2.64 2.64 2.64 2.64 2.64 2.64 Oracle 32 2.66 2.66 2.68 2.77 2.65 2.77 2.66 2.68 2.69 2.69 2.69 2.65 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 33 2.67 2.67 2.71 2.67 2.64 2.65 2.72 2.71 2.69 2.69 2.70 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 34 2.72 2.72 2.66 2.73 2.66 2.75 2.71 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Oracle 35 2.65 2.71 2.69 2.65 2.66 2.68 2.66 2.66 2.65 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 36 2.71 2.66 2.66 2.66 2.66 2.66 2.66 2.69 2.66 2.66 2.70 2.66 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 37 2.67 2.71 2.65 2.68 2.70 2.73 2.66 2.66 2.78 2.66 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Oracle 38 2.70 2.68 2.66 2.69 2.66 2.66 2.72 2.66 2.75 2.66 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 39 2.69 2.65 2.67 2.66 2.65 2.70 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 40 2.66 2.67 2.66 2.66 2.66 2.66 2.66 2.71 2.70 2.69 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 41 2.71 2.71 2.66 2.68 2.66 2.65 2.65 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 42 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 43 2.77 2.67 2.66 2.67 2.70 2.69 2.68 2.65 2.66 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 2.72 Oracle 44 2.70 2.71 2.66 2.65 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 45 2.73 2.65 2.72 2.66 2.66 2.65 2.71 2.66 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 46 2.79 2.67 2.68 2.66 2.66 2.66 2.68 2.65 2.67 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 47 2.71 2.80 2.69 2.69 2.71 2.71 2.67 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 48 2.71 2.66 2.66 2.70 2.68 2.65 2.66 2.72 2.70 2.65 2.66 2.69 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 49 2.70 2.73 2.72 2.66 2.65 2.70 2.67 2.76 2.69 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 50 2.70 2.66 2.69 2.65 2.65 2.66 2.70 2.70 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Total 131 129 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 Average 2.63 2.57 2.55 2.55 2.55 2.56 2.55 2.55 2.56 2.55 2.55 2.55 2.55 2.55 2.55 2.55 2.55 2.55 2.55 2.55 D.14. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, DENSE START333
Size Results of configuration Shattered Document, dense start
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 2 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 3 19 2 4 2.38 4 2 2 0.50 2.00 Oracle 4 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 5 20 2 4 2.50 4 2 2 0.50 2.00 Oracle 6 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 7 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 8 27 2 6 2.45 6 2 4 0.34 3.00 Oracle 9 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 10 140 7 7 7.00 7 7 0 1.00 1.00 Oracle 11 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 12 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 13 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 14 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 15 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 16 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 17 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 18 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 19 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 20 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 21 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 22 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 23 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 24 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 27 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 28 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 29 35 7 7 7.00 7 7 0 1.00 1.00 Oracle 30 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 31 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 32 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 33 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 34 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 35 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 36 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 37 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 38 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 39 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 40 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 41 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 44 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 45 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 46 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 47 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 48 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 49 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 3377 335 343 336.33 343 335 8 48.34 54.00 Average 67.54 6.70 6.86 6.73 6.86 6.70 0.16 0.97 1.08 Micro avg ····················· 0.98 1.02 334 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document, dense start
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 2 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 3 4 2 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 4 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 5 4 3 2 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 6 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 8 6 2 2 2 2 2 2 2 2 3 2 2 2 2 2 2 2 2 2 2 Oracle 9 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 10 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 11 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 12 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 13 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 14 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 15 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 16 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 17 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 18 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 19 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 20 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 21 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 22 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 23 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 24 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 27 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 28 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 29 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 30 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 31 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 32 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 33 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 34 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 35 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 36 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 37 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 38 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 39 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 40 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 41 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 44 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 45 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 46 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 47 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 48 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 49 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 343 336 335 335 336 336 335 335 335 336 335 335 335 335 335 335 335 335 335 335 Average 6.86 6.72 6.70 6.70 6.72 6.72 6.70 6.70 6.70 6.72 6.70 6.70 6.70 6.70 6.70 6.70 6.70 6.70 6.70 6.70 D.14. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, DENSE START335
Cost Results of configuration Shattered Document, dense start
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 2 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 3 19.00 2.00 4.00 2.38 4.00 2.00 2.00 0.50 2.00 Oracle 4 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 5 20.00 2.00 4.00 2.50 4.00 2.00 2.00 0.50 2.00 Oracle 6 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 7 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 8 27.00 2.00 6.00 2.45 6.00 2.00 4.00 0.34 3.00 Oracle 9 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 10 140.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 11 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 12 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 13 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 14 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 15 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 16 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 17 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 18 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 19 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 20 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 21 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 22 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 23 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 24 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 27 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 28 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 29 35.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 30 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 31 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 32 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 33 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 34 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 35 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 36 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 37 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 38 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 39 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 40 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 41 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 44 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 45 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 46 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 47 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 48 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 49 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 3377.00 335.00 343.00 336.33 343.00 335.00 8.00 48.34 54.00 Average 67.54 6.70 6.86 6.73 6.86 6.70 0.16 0.97 1.08 Micro avg ····················· 0.98 1.02 336 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document, dense start
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 2 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 3 4.00 2.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 4 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 5 4.00 3.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 6 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 7 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 8 6.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 9 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 10 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 11 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 12 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 13 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 14 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 15 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 16 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 17 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 18 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 19 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 20 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 21 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 22 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 23 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 24 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 27 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 28 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 29 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 30 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 31 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 32 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 33 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 34 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 35 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 36 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 37 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 38 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 39 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 40 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 41 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 44 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 45 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 46 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 47 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 48 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 49 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 343 336 335 335 336 336 335 335 335 336 335 335 335 335 335 335 335 335 335 335 Average 6.86 6.72 6.70 6.70 6.72 6.72 6.70 6.70 6.70 6.72 6.70 6.70 6.70 6.70 6.70 6.70 6.70 6.70 6.70 6.70 D.15. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS337 D.15 Complete results of configuration Shattered Document with Markov Chains
The Shattered Documents format, using Markov Chains (1st order) for adaptation. Using special functions Start, Choose, Recentre from Kasim2.Markov. Using the model of the ARM on Kasim2.Documents.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.30
Size0 = 6.34
Size1 = 6.06 Size gain = 1.05
Cost0 = 6.34
Cost1 = 6.06 Cost gain = 1.05 338 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Markov Chains
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 0 0 0 9 0.00 1.00 Oracle 2 30 6 0 6 0 6 0 1.00 0.00 Oracle 3 40 8 0 8 0 8 0 1.00 0.00 Oracle 4 46 7 0 7 0 7 0 1.00 0.00 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 11 0 11 0 1.00 0.00 Oracle 8 125 11 0 11 0 11 0 1.00 0.00 Oracle 9 128 7 0 7 0 7 0 1.00 0.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 8 0 8 0 1.00 0.00 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 1 0 1 8 0.11 0.89 Oracle 18 269 11 0 11 0 11 0 1.00 0.00 Oracle 19 290 6 0 6 0 6 0 1.00 0.00 Oracle 20 307 6 0 6 0 6 0 1.00 0.00 Oracle 21 366 10 0 10 0 10 0 1.00 0.00 Oracle 22 410 10 0 10 0 10 0 1.00 0.00 Oracle 23 455 15 0 15 0 15 0 1.00 0.00 Oracle 24 508 11 0 11 0 11 0 1.00 0.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 15 0 15 0 1.00 0.00 Oracle 27 516 13 0 13 0 13 0 1.00 0.00 Oracle 28 521 7 0 7 0 7 0 1.00 0.00 Oracle 29 542 5 0 5 0 5 0 1.00 0.00 Oracle 30 554 13 0 13 0 13 0 1.00 0.00 Oracle 31 601 11 0 11 0 11 0 1.00 0.00 Oracle 32 606 14 0 14 0 14 0 1.00 0.00 Oracle 33 616 12 0 12 0 12 0 1.00 0.00 Oracle 34 649 8 0 8 0 8 0 1.00 0.00 Oracle 35 723 10 0 10 0 10 0 1.00 0.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 11 0 11 0 1.00 0.00 Oracle 39 786 7 0 7 0 7 0 1.00 0.00 Oracle 40 840 14 0 14 0 14 0 1.00 0.00 Oracle 41 846 8 0 8 0 8 0 1.00 0.00 Oracle 42 883 1 0 1 0 1 0 1.00 0.00 Oracle 43 890 10 0 10 0 10 0 1.00 0.00 Oracle 44 897 6 0 6 0 6 0 1.00 0.00 Oracle 45 899 9 0 9 0 9 0 1.00 0.00 Oracle 46 929 10 0 10 0 10 0 1.00 0.00 Oracle 47 936 10 0 10 0 10 0 1.00 0.00 Oracle 48 939 13 0 13 0 13 0 1.00 0.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 353 0 353 147 37.11 12.89 Average ··· 10.00 0.00 7.06 0.00 7.06 2.94 0.74 0.26 Micro avg ····················· 0.70 0.30 D.15. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS339
Duration Results of configuration Shattered Document with Markov Chains
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 13.13 1.45 1.48 1.46 1.45 1.48 -0.03 1.02 0.98 Oracle 2 15.85 2.63 2.67 2.64 2.63 2.67 -0.04 1.02 0.98 Oracle 3 21.19 2.63 2.67 2.65 2.67 2.66 0.02 0.99 1.01 Oracle 4 18.71 2.63 2.79 2.67 2.64 2.71 -0.07 1.02 0.98 Oracle 5 10.38 1.27 1.34 1.30 1.27 1.33 -0.05 1.04 0.96 Oracle 6 16.20 1.44 1.70 1.47 1.70 1.46 0.23 0.86 1.16 Oracle 7 29.19 2.62 2.73 2.66 2.73 2.71 0.02 0.99 1.01 Oracle 8 28.89 2.61 2.69 2.63 2.65 2.69 -0.04 1.02 0.98 Oracle 9 18.45 2.62 2.69 2.63 2.63 2.63 -0.01 1.00 1.00 Oracle 10 21.19 1.03 1.28 1.06 1.28 1.08 0.20 0.84 1.19 Oracle 11 12.68 1.02 1.27 1.05 1.27 1.06 0.21 0.84 1.20 Oracle 12 13.80 1.03 1.28 1.06 1.28 1.06 0.22 0.83 1.21 Oracle 13 22.43 1.44 1.73 1.49 1.73 1.52 0.20 0.88 1.13 Oracle 14 16.52 1.44 1.71 1.50 1.70 1.47 0.24 0.86 1.16 Oracle 15 21.49 2.66 2.72 2.69 2.67 2.71 -0.04 1.02 0.98 Oracle 16 6.85 1.05 1.33 1.14 1.29 1.09 0.20 0.85 1.17 Oracle 17 11.57 1.03 2.77 1.29 2.77 1.04 1.73 0.38 2.67 Oracle 18 29.06 2.62 2.77 2.64 2.63 2.63 0.00 1.00 1.00 Oracle 19 15.73 2.61 2.63 2.63 2.62 2.63 -0.02 1.01 0.99 Oracle 20 16.09 2.66 2.70 2.68 2.70 2.70 0.00 1.00 1.00 Oracle 21 26.44 2.63 2.66 2.64 2.63 2.66 -0.02 1.01 0.99 Oracle 22 26.88 2.66 2.70 2.69 2.70 2.69 0.01 1.00 1.00 Oracle 23 39.68 2.61 2.73 2.65 2.63 2.73 -0.10 1.04 0.96 Oracle 24 29.88 2.65 2.88 2.72 2.65 2.71 -0.05 1.02 0.98 Oracle 25 24.02 2.65 2.69 2.67 2.69 2.68 0.00 1.00 1.00 Oracle 26 40.27 2.65 2.79 2.69 2.66 2.74 -0.09 1.03 0.97 Oracle 27 34.95 2.65 2.75 2.69 2.69 2.70 -0.02 1.01 0.99 Oracle 28 18.84 2.66 2.78 2.69 2.69 2.78 -0.09 1.03 0.97 Oracle 29 14.11 2.72 3.20 2.82 2.73 2.73 -0.01 1.00 1.00 Oracle 30 34.92 2.66 2.82 2.69 2.66 2.71 -0.05 1.02 0.98 Oracle 31 30.17 2.70 2.86 2.74 2.74 2.78 -0.04 1.02 0.98 Oracle 32 38.41 2.70 2.96 2.74 2.70 2.74 -0.04 1.02 0.98 Oracle 33 32.77 2.70 2.80 2.73 2.71 2.76 -0.05 1.02 0.98 Oracle 34 21.98 2.70 2.88 2.75 2.71 2.77 -0.05 1.02 0.98 Oracle 35 26.47 2.61 2.68 2.65 2.63 2.66 -0.02 1.01 0.99 Oracle 36 13.73 1.02 1.28 1.05 1.28 1.04 0.24 0.81 1.23 Oracle 37 11.70 1.03 1.27 1.06 1.27 1.04 0.23 0.82 1.22 Oracle 38 29.18 2.60 2.71 2.66 2.67 2.69 -0.02 1.01 0.99 Oracle 39 18.93 2.67 2.74 2.70 2.67 2.71 -0.04 1.02 0.98 Oracle 40 37.80 2.66 2.83 2.70 2.70 2.74 -0.04 1.02 0.98 Oracle 41 21.69 2.66 2.94 2.71 2.70 2.71 -0.02 1.01 0.99 Oracle 42 2.65 2.65 2.65 2.65 2.65 2.65 0.00 1.00 1.00 Oracle 43 26.96 2.66 2.75 2.70 2.70 2.75 -0.05 1.02 0.98 Oracle 44 16.07 2.66 2.71 2.68 2.66 2.70 -0.03 1.01 0.99 Oracle 45 24.23 2.66 2.74 2.70 2.69 2.74 -0.05 1.02 0.98 Oracle 46 26.38 2.62 2.71 2.64 2.63 2.63 0.00 1.00 1.00 Oracle 47 26.55 2.61 2.77 2.66 2.67 2.66 0.02 0.99 1.01 Oracle 48 34.72 2.61 2.91 2.67 2.64 2.70 -0.06 1.02 0.98 Oracle 49 10.74 1.02 1.28 1.07 1.28 1.04 0.24 0.81 1.23 Oracle 50 24.25 2.66 2.77 2.70 2.66 2.73 -0.07 1.02 0.98 Total 1124.77 113.23 122.17 115.22 118.38 115.70 2.67 48.29 53.09 Average 22.49 2.27 2.45 2.30 2.37 2.31 0.05 0.97 1.06 Micro avg ····················· 0.98 1.02 340 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Markov Chains
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 1.45 1.45 1.45 1.46 1.45 1.47 1.48 1.46 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 Oracle 2 2.63 2.64 2.63 2.63 2.64 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 2.67 Oracle 3 2.67 2.63 2.63 2.63 2.65 2.66 2.65 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 4 2.64 2.63 2.64 2.63 2.79 2.66 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 5 1.27 1.30 1.34 1.27 1.28 1.30 1.29 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 Oracle 6 1.70 1.44 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 7 2.73 2.62 2.62 2.62 2.67 2.64 2.66 2.63 2.65 2.66 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 8 2.65 2.62 2.63 2.62 2.63 2.62 2.61 2.62 2.61 2.62 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 9 2.63 2.62 2.64 2.62 2.64 2.69 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 Oracle 10 1.28 1.04 1.04 1.03 1.03 1.03 1.03 1.03 1.03 1.07 1.06 1.10 1.05 1.08 1.03 1.04 1.03 1.07 1.04 1.08 Oracle 11 1.27 1.03 1.02 1.03 1.03 1.03 1.04 1.05 1.04 1.04 1.04 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 Oracle 12 1.28 1.03 1.03 1.09 1.03 1.03 1.04 1.03 1.03 1.04 1.05 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 Oracle 13 1.73 1.70 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.44 1.45 1.52 1.47 1.46 1.52 1.52 1.52 1.52 1.52 1.52 Oracle 14 1.70 1.71 1.44 1.44 1.44 1.45 1.48 1.48 1.45 1.46 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 Oracle 15 2.67 2.70 2.66 2.66 2.72 2.68 2.70 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 16 1.29 1.33 1.05 1.05 1.05 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 17 2.77 1.28 1.27 1.03 1.03 1.04 1.03 1.07 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 18 2.63 2.62 2.62 2.62 2.62 2.64 2.77 2.65 2.63 2.64 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 Oracle 19 2.62 2.63 2.63 2.61 2.61 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 Oracle 20 2.70 2.66 2.67 2.66 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 21 2.63 2.63 2.63 2.65 2.64 2.63 2.64 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 22 2.70 2.70 2.69 2.69 2.70 2.68 2.66 2.70 2.70 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 23 2.63 2.63 2.62 2.61 2.63 2.61 2.63 2.63 2.62 2.64 2.65 2.73 2.70 2.64 2.73 2.73 2.73 2.73 2.73 2.73 Oracle 24 2.65 2.88 2.66 2.80 2.69 2.67 2.66 2.70 2.69 2.75 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 25 2.69 2.66 2.66 2.66 2.65 2.66 2.68 2.66 2.68 2.68 2.68 2.68 2.68 2.68 2.68 2.68 2.68 2.68 2.68 2.68 Oracle 26 2.66 2.66 2.66 2.66 2.65 2.65 2.79 2.69 2.69 2.67 2.67 2.70 2.71 2.70 2.74 2.74 2.74 2.74 2.74 2.74 Oracle 27 2.69 2.67 2.66 2.66 2.66 2.66 2.65 2.70 2.68 2.73 2.75 2.73 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 28 2.69 2.69 2.66 2.66 2.66 2.70 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 Oracle 29 2.73 2.73 3.20 2.72 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 Oracle 30 2.66 2.67 2.66 2.66 2.66 2.66 2.66 2.82 2.71 2.66 2.67 2.70 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 31 2.74 2.74 2.71 2.71 2.72 2.72 2.70 2.86 2.73 2.76 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 2.78 Oracle 32 2.70 2.72 2.71 2.71 2.70 2.70 2.76 2.96 2.73 2.75 2.74 2.73 2.75 2.74 2.74 2.74 2.74 2.74 2.74 2.74 Oracle 33 2.71 2.70 2.71 2.71 2.73 2.80 2.74 2.71 2.73 2.73 2.73 2.76 2.76 2.76 2.76 2.76 2.76 2.76 2.76 2.76 Oracle 34 2.71 2.70 2.71 2.88 2.72 2.76 2.73 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 2.77 Oracle 35 2.63 2.68 2.66 2.61 2.66 2.66 2.63 2.64 2.64 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 36 1.28 1.03 1.02 1.03 1.02 1.02 1.04 1.03 1.04 1.06 1.05 1.05 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 37 1.27 1.03 1.03 1.03 1.04 1.04 1.09 1.04 1.06 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 38 2.67 2.64 2.60 2.66 2.62 2.63 2.71 2.65 2.63 2.68 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 2.69 Oracle 39 2.67 2.69 2.74 2.73 2.72 2.68 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 40 2.70 2.67 2.67 2.66 2.67 2.66 2.66 2.67 2.67 2.80 2.83 2.69 2.68 2.74 2.74 2.74 2.74 2.74 2.74 2.74 Oracle 41 2.70 2.68 2.66 2.66 2.66 2.94 2.69 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 2.71 Oracle 42 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 2.65 Oracle 43 2.70 2.67 2.68 2.68 2.66 2.70 2.70 2.74 2.69 2.75 2.75 2.75 2.75 2.75 2.75 2.75 2.75 2.75 2.75 2.75 Oracle 44 2.66 2.66 2.66 2.68 2.71 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 45 2.69 2.66 2.68 2.67 2.69 2.68 2.70 2.72 2.74 2.74 2.74 2.74 2.74 2.74 2.74 2.74 2.74 2.74 2.74 2.74 Oracle 46 2.63 2.62 2.66 2.62 2.63 2.62 2.66 2.62 2.71 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 2.63 Oracle 47 2.67 2.64 2.62 2.62 2.61 2.62 2.64 2.77 2.70 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 2.66 Oracle 48 2.64 2.61 2.62 2.62 2.67 2.91 2.63 2.64 2.64 2.67 2.68 2.68 2.70 2.70 2.70 2.70 2.70 2.70 2.70 2.70 Oracle 49 1.28 1.03 1.02 1.07 1.04 1.04 1.05 1.06 1.11 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 50 2.66 2.66 2.67 2.67 2.67 2.69 2.77 2.72 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 Total 118 115 114 114 114 115 115 115 115 115 115 116 115 115 116 116 116 116 116 116 Average 2.37 2.30 2.29 2.28 2.28 2.30 2.30 2.31 2.30 2.30 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 2.31 D.15. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS341
Size Results of configuration Shattered Document with Markov Chains
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 36 4 4 4.00 4 4 0 1.00 1.00 Oracle 2 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 3 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 4 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 5 32 4 4 4.00 4 4 0 1.00 1.00 Oracle 6 45 4 5 4.09 5 4 1 0.80 1.25 Oracle 7 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 8 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 9 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 10 61 3 4 3.05 4 3 1 0.75 1.34 Oracle 11 37 3 4 3.09 4 3 1 0.75 1.34 Oracle 12 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 13 62 4 5 4.13 5 4 1 0.80 1.25 Oracle 14 46 4 5 4.18 5 4 1 0.80 1.25 Oracle 15 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 16 20 3 4 3.34 4 3 1 0.75 1.34 Oracle 17 33 3 7 3.66 7 3 4 0.43 2.34 Oracle 18 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 19 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 20 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 21 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 22 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 23 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 24 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 27 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 28 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 29 35 7 7 7.00 7 7 0 1.00 1.00 Oracle 30 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 31 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 32 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 33 84 7 7 7.00 7 7 0 1.00 1.00 Oracle 34 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 35 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 36 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 37 34 3 4 3.09 4 3 1 0.75 1.34 Oracle 38 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 39 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 40 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 41 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 44 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 45 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 46 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 47 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 48 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 49 31 3 4 3.10 4 3 1 0.75 1.34 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 2981 303 317 304.88 317 303 14 47.08 54.41 Average 59.62 6.06 6.34 6.10 6.34 6.06 0.28 0.95 1.09 Micro avg ····················· 0.95 1.05 342 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Markov Chains
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 2 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 3 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 4 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 6 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 8 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 9 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 10 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 11 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 12 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 13 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 14 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 15 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 16 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 17 7 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 18 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 19 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 20 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 21 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 22 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 23 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 24 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 27 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 28 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 29 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 30 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 31 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 32 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 33 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 34 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 35 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 36 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 37 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 38 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 39 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 40 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 41 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 44 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 45 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 46 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 47 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 48 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 49 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 317 307 304 303 303 303 303 303 303 303 303 303 303 303 303 303 303 303 303 303 Average 6.34 6.14 6.08 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 D.15. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS343
Cost Results of configuration Shattered Document with Markov Chains
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 36.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 2 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 3 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 4 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 5 32.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 6 45.00 4.00 5.00 4.09 5.00 4.00 1.00 0.80 1.25 Oracle 7 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 8 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 9 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 10 61.00 3.00 4.00 3.05 4.00 3.00 1.00 0.75 1.34 Oracle 11 37.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 12 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 13 62.00 4.00 5.00 4.13 5.00 4.00 1.00 0.80 1.25 Oracle 14 46.00 4.00 5.00 4.18 5.00 4.00 1.00 0.80 1.25 Oracle 15 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 16 20.00 3.00 4.00 3.34 4.00 3.00 1.00 0.75 1.34 Oracle 17 33.00 3.00 7.00 3.66 7.00 3.00 4.00 0.43 2.34 Oracle 18 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 19 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 20 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 21 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 22 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 23 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 24 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 27 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 28 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 29 35.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 30 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 31 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 32 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 33 84.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 34 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 35 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 36 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 37 34.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 38 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 39 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 40 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 41 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 44 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 45 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 46 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 47 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 48 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 49 31.00 3.00 4.00 3.10 4.00 3.00 1.00 0.75 1.34 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 2981.00 303.00 317.00 304.88 317.00 303.00 14.00 47.08 54.41 Average 59.62 6.06 6.34 6.10 6.34 6.06 0.28 0.95 1.09 Micro avg ····················· 0.95 1.05 344 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Markov Chains
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 2 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 3 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 4 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 5 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 6 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 7 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 8 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 9 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 10 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 11 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 12 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 13 5.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 14 5.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 15 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 16 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 17 7.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 18 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 19 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 20 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 21 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 22 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 23 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 24 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 27 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 28 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 29 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 30 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 31 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 32 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 33 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 34 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 35 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 36 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 37 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 38 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 39 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 40 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 41 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 44 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 45 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 46 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 47 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 48 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 49 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 317 307 304 303 303 303 303 303 303 303 303 303 303 303 303 303 303 303 303 303 Average 6.34 6.14 6.08 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 6.06 D.16. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS WITH HEURISTICS345 D.16 Complete results of configuration Shattered Document with Markov Chains with heuris- tics
The Shattered Documents format, using Markov Chains (1st order) and the same heuristics concerning seen atoms as in Kasim2.Activation. Using special functions Start, Choose, Recentre from Kasim2.Markov With Heuristics. Using the model of the ARM on Kasim2.Documents.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.84
Size0 = 5.84
Size1 = 4.22 Size gain = 1.38
Cost0 = 5.84
Cost1 = 4.22 Cost gain = 1.38 346 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Markov Chains with heuristics
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 0 0 0 9 0.00 1.00 Oracle 2 30 6 0 0 0 0 6 0.00 1.00 Oracle 3 40 8 0 0 0 0 8 0.00 1.00 Oracle 4 46 7 0 0 0 0 7 0.00 1.00 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 0 0 0 0 7 0.00 1.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 0 0 0 8 0.00 1.00 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 0 0 0 9 0.00 1.00 Oracle 18 269 11 0 0 0 0 11 0.00 1.00 Oracle 19 290 6 0 0 0 0 6 0.00 1.00 Oracle 20 307 6 0 0 0 0 6 0.00 1.00 Oracle 21 366 10 0 1 0 1 9 0.10 0.90 Oracle 22 410 10 0 0 0 0 10 0.00 1.00 Oracle 23 455 15 0 1 0 1 14 0.07 0.93 Oracle 24 508 11 0 0 0 0 11 0.00 1.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 15 0 15 0 1.00 0.00 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 0 0 0 7 0.00 1.00 Oracle 29 542 5 0 1 0 1 4 0.20 0.80 Oracle 30 554 13 0 0 0 0 13 0.00 1.00 Oracle 31 601 11 0 11 0 11 0 1.00 0.00 Oracle 32 606 14 0 14 0 14 0 1.00 0.00 Oracle 33 616 12 0 1 0 1 11 0.09 0.91 Oracle 34 649 8 0 8 0 8 0 1.00 0.00 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 0 0 0 11 0.00 1.00 Oracle 39 786 7 0 7 0 7 0 1.00 0.00 Oracle 40 840 14 0 1 0 1 13 0.07 0.93 Oracle 41 846 8 0 0 0 0 8 0.00 1.00 Oracle 42 883 1 0 0 0 0 1 0.00 1.00 Oracle 43 890 10 0 1 0 1 9 0.10 0.90 Oracle 44 897 6 0 0 0 0 6 0.00 1.00 Oracle 45 899 9 0 0 0 0 9 0.00 1.00 Oracle 46 929 10 0 0 0 0 10 0.00 1.00 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 0 0 0 0 13 0.00 1.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 79 0 79 421 7.63 42.38 Average ··· 10.00 0.00 1.58 0.00 1.58 8.42 0.16 0.84 Micro avg ····················· 0.16 0.84 D.16. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS WITH HEURISTICS347
Duration Results of configuration Shattered Document with Markov Chains with heuristics
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 13.47 1.48 1.53 1.50 1.48 1.53 -0.05 1.03 0.97 Oracle 2 9.70 1.49 2.21 1.62 2.21 1.51 0.70 0.68 1.47 Oracle 3 12.84 1.49 2.19 1.60 2.19 1.53 0.66 0.70 1.43 Oracle 4 11.24 1.49 2.19 1.61 2.19 1.53 0.66 0.70 1.43 Oracle 5 10.69 1.32 1.36 1.34 1.32 1.33 -0.01 1.00 1.00 Oracle 6 16.80 1.49 1.78 1.53 1.78 1.50 0.28 0.84 1.19 Oracle 7 15.30 1.09 2.20 1.39 2.04 1.09 0.95 0.54 1.87 Oracle 8 14.99 1.08 2.62 1.36 1.55 1.09 0.46 0.70 1.42 Oracle 9 10.41 1.09 2.61 1.48 2.61 1.34 1.27 0.52 1.95 Oracle 10 23.20 1.08 1.35 1.16 1.35 1.09 0.27 0.80 1.25 Oracle 11 13.43 1.08 1.38 1.12 1.38 1.14 0.23 0.83 1.20 Oracle 12 14.44 1.08 1.38 1.11 1.38 1.09 0.29 0.79 1.27 Oracle 13 23.20 1.49 1.76 1.55 1.76 1.49 0.27 0.85 1.18 Oracle 14 17.18 1.49 1.82 1.56 1.76 1.50 0.27 0.85 1.17 Oracle 15 9.60 1.09 1.78 1.20 1.78 1.09 0.69 0.62 1.63 Oracle 16 7.50 1.11 1.39 1.25 1.38 1.11 0.27 0.80 1.24 Oracle 17 11.29 1.09 1.91 1.26 1.91 1.12 0.80 0.59 1.71 Oracle 18 13.48 1.08 1.77 1.23 1.77 1.38 0.40 0.77 1.29 Oracle 19 9.08 1.49 1.55 1.52 1.53 1.51 0.02 0.98 1.02 Oracle 20 10.33 1.52 2.63 1.72 2.63 1.52 1.12 0.58 1.73 Oracle 21 17.85 1.52 2.77 1.78 2.77 1.53 1.24 0.55 1.81 Oracle 22 16.52 1.51 2.64 1.66 2.64 1.51 1.13 0.57 1.75 Oracle 23 19.70 1.08 2.79 1.31 2.79 1.13 1.66 0.41 2.48 Oracle 24 16.29 1.34 2.61 1.48 2.61 1.52 1.09 0.58 1.72 Oracle 25 25.23 2.76 2.89 2.80 2.79 2.81 -0.02 1.01 0.99 Oracle 26 41.99 2.76 2.88 2.80 2.80 2.81 -0.02 1.01 0.99 Oracle 27 17.67 1.10 1.84 1.36 1.84 1.34 0.52 0.72 1.38 Oracle 28 13.13 1.34 2.60 1.88 1.98 1.35 0.63 0.68 1.47 Oracle 29 10.18 1.53 2.88 2.04 2.88 1.57 1.30 0.55 1.84 Oracle 30 17.38 1.08 2.19 1.34 2.19 1.36 0.83 0.63 1.61 Oracle 31 30.82 2.77 2.85 2.80 2.85 2.81 0.04 0.98 1.02 Oracle 32 39.25 2.77 2.88 2.80 2.77 2.78 -0.01 1.00 1.00 Oracle 33 20.55 1.52 2.84 1.71 2.84 1.59 1.25 0.56 1.79 Oracle 34 22.86 2.83 2.92 2.86 2.83 2.83 0.00 1.00 1.00 Oracle 35 13.48 1.09 2.64 1.35 2.64 1.09 1.55 0.41 2.43 Oracle 36 19.72 1.49 1.59 1.52 1.59 1.51 0.09 0.95 1.05 Oracle 37 14.28 1.08 2.62 1.30 2.62 1.09 1.53 0.41 2.41 Oracle 38 14.30 1.09 2.27 1.30 2.27 1.12 1.15 0.49 2.03 Oracle 39 19.90 2.81 2.89 2.84 2.81 2.88 -0.07 1.02 0.98 Oracle 40 23.60 1.52 2.91 1.69 2.91 1.56 1.35 0.54 1.87 Oracle 41 13.00 1.52 2.20 1.63 2.20 1.61 0.59 0.73 1.37 Oracle 42 2.59 2.59 2.59 2.59 2.59 2.59 0.00 1.00 1.00 Oracle 43 20.89 1.94 2.95 2.09 2.95 1.98 0.97 0.67 1.48 Oracle 44 10.28 1.52 2.63 1.71 2.63 1.53 1.11 0.58 1.72 Oracle 45 14.02 1.52 1.79 1.55 1.79 1.52 0.27 0.85 1.17 Oracle 46 13.48 1.09 2.72 1.34 2.72 1.13 1.59 0.41 2.41 Oracle 47 13.05 1.08 2.23 1.30 2.23 1.09 1.15 0.48 2.05 Oracle 48 18.09 1.09 2.63 1.39 2.37 1.34 1.02 0.57 1.77 Oracle 49 12.30 1.09 1.40 1.23 1.36 1.34 0.02 0.98 1.02 Oracle 50 25.82 2.81 2.99 2.87 2.90 2.81 0.09 0.97 1.03 Total 826.41 76.82 114.08 84.42 111.22 79.60 31.62 36.49 74.05 Average 16.53 1.54 2.28 1.69 2.23 1.59 0.63 0.73 1.48 Micro avg ····················· 0.72 1.40 348 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Markov Chains with heuristics
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 1.48 1.48 1.49 1.52 1.48 1.48 1.49 1.50 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 Oracle 2 2.21 1.49 1.50 1.49 1.49 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 Oracle 3 2.19 1.49 1.53 1.49 1.55 1.52 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 Oracle 4 2.19 1.51 1.49 1.52 1.50 1.50 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 Oracle 5 1.32 1.36 1.33 1.32 1.33 1.34 1.36 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 Oracle 6 1.78 1.52 1.49 1.49 1.49 1.52 1.49 1.49 1.50 1.51 1.50 1.50 1.50 1.50 1.50 1.50 1.50 1.50 1.50 1.50 Oracle 7 2.04 2.20 1.33 1.35 1.37 1.34 1.09 1.09 1.09 1.34 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 8 1.55 2.62 1.35 1.34 1.09 1.09 1.34 1.09 1.08 1.35 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 9 2.61 1.34 1.09 1.33 1.34 1.36 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 Oracle 10 1.35 1.10 1.10 1.35 1.09 1.14 1.11 1.10 1.35 1.10 1.09 1.08 1.09 1.34 1.10 1.34 1.09 1.11 1.09 1.09 Oracle 11 1.38 1.09 1.09 1.08 1.10 1.10 1.08 1.09 1.09 1.11 1.09 1.14 1.14 1.14 1.14 1.14 1.14 1.14 1.14 1.14 Oracle 12 1.38 1.09 1.10 1.09 1.09 1.09 1.08 1.08 1.09 1.09 1.08 1.10 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 13 1.76 1.76 1.53 1.53 1.52 1.56 1.50 1.50 1.49 1.50 1.55 1.53 1.49 1.50 1.49 1.49 1.49 1.49 1.49 1.49 Oracle 14 1.76 1.82 1.54 1.53 1.51 1.50 1.51 1.50 1.52 1.49 1.50 1.50 1.50 1.50 1.50 1.50 1.50 1.50 1.50 1.50 Oracle 15 1.78 1.15 1.09 1.10 1.18 1.12 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 16 1.38 1.37 1.12 1.13 1.39 1.11 1.11 1.11 1.11 1.11 1.11 1.11 1.11 1.11 1.11 1.11 1.11 1.11 1.11 1.11 Oracle 17 1.91 1.41 1.10 1.15 1.09 1.09 1.09 1.34 1.12 1.12 1.12 1.12 1.12 1.12 1.12 1.12 1.12 1.12 1.12 1.12 Oracle 18 1.77 1.34 1.09 1.09 1.09 1.33 1.09 1.08 1.13 1.10 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 19 1.53 1.55 1.50 1.49 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 Oracle 20 2.63 1.55 1.52 1.52 1.58 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 Oracle 21 2.77 2.61 1.52 1.52 1.52 1.61 1.53 1.52 1.71 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 Oracle 22 2.64 1.55 1.73 1.51 1.51 1.51 1.52 1.55 1.52 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 Oracle 23 2.79 2.63 1.09 1.12 1.08 1.09 1.09 1.10 1.09 1.09 1.09 1.10 1.13 1.12 1.13 1.13 1.13 1.13 1.13 1.13 Oracle 24 2.61 1.34 1.40 1.35 1.34 1.38 1.34 1.34 1.34 1.36 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 Oracle 25 2.79 2.78 2.89 2.77 2.77 2.76 2.83 2.84 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 Oracle 26 2.80 2.79 2.88 2.76 2.77 2.78 2.77 2.77 2.80 2.83 2.85 2.84 2.77 2.77 2.81 2.81 2.81 2.81 2.81 2.81 Oracle 27 1.84 1.18 1.43 1.10 1.34 1.36 1.34 1.34 1.33 1.36 1.35 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 Oracle 28 1.98 1.95 1.97 1.92 2.60 1.34 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 1.35 Oracle 29 2.88 2.66 1.53 1.55 1.57 1.57 1.57 1.57 1.57 1.57 1.57 1.57 1.57 1.57 1.57 1.57 1.57 1.57 1.57 1.57 Oracle 30 2.19 1.37 1.51 1.13 1.37 1.09 1.08 1.35 1.09 1.09 1.37 1.39 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 Oracle 31 2.85 2.80 2.79 2.82 2.78 2.79 2.80 2.83 2.77 2.78 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 Oracle 32 2.77 2.88 2.84 2.77 2.77 2.78 2.77 2.77 2.84 2.77 2.77 2.84 2.88 2.78 2.78 2.78 2.78 2.78 2.78 2.78 Oracle 33 2.84 2.22 1.55 1.52 1.53 1.63 1.52 1.52 1.52 1.53 1.57 1.59 1.59 1.59 1.59 1.59 1.59 1.59 1.59 1.59 Oracle 34 2.83 2.83 2.88 2.92 2.85 2.85 2.88 2.83 2.83 2.83 2.83 2.83 2.83 2.83 2.83 2.83 2.83 2.83 2.83 2.83 Oracle 35 2.64 1.34 1.35 1.34 1.34 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 36 1.59 1.52 1.52 1.51 1.50 1.50 1.49 1.50 1.52 1.54 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 Oracle 37 2.62 1.12 1.37 1.34 1.09 1.34 1.09 1.09 1.09 1.08 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 38 2.27 1.14 1.10 1.09 1.34 1.34 1.36 1.09 1.35 1.11 1.12 1.12 1.12 1.12 1.12 1.12 1.12 1.12 1.12 1.12 Oracle 39 2.81 2.85 2.89 2.81 2.81 2.84 2.88 2.88 2.88 2.88 2.88 2.88 2.88 2.88 2.88 2.88 2.88 2.88 2.88 2.88 Oracle 40 2.91 2.28 1.53 1.54 1.52 1.52 1.56 1.52 1.52 1.55 1.52 1.52 1.52 1.56 1.56 1.56 1.56 1.56 1.56 1.56 Oracle 41 2.20 1.52 1.52 1.52 1.54 1.55 1.55 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 1.61 Oracle 42 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 2.59 Oracle 43 2.95 2.24 1.94 1.94 1.94 1.96 1.95 2.01 1.98 1.98 1.98 1.98 1.98 1.98 1.98 1.98 1.98 1.98 1.98 1.98 Oracle 44 2.63 1.52 1.52 1.54 1.52 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 1.53 Oracle 45 1.79 1.54 1.52 1.52 1.52 1.52 1.55 1.55 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 1.52 Oracle 46 2.72 1.09 1.10 1.34 1.39 1.13 1.10 1.12 1.37 1.13 1.13 1.13 1.13 1.13 1.13 1.13 1.13 1.13 1.13 1.13 Oracle 47 2.23 1.11 1.35 1.09 1.08 1.08 1.34 1.34 1.36 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 48 2.37 2.63 1.34 1.35 1.11 1.12 1.34 1.12 1.09 1.09 1.10 1.10 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 Oracle 49 1.36 1.09 1.34 1.38 1.40 1.09 1.09 1.11 1.13 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 Oracle 50 2.90 2.84 2.99 2.84 2.82 2.90 2.86 2.84 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 2.81 Total 111 90 81 80 81 79 79 79 79 79 79 79 80 80 80 80 80 80 80 80 Average 2.23 1.80 1.63 1.61 1.63 1.59 1.59 1.58 1.59 1.58 1.59 1.59 1.59 1.59 1.59 1.59 1.59 1.59 1.59 1.59 D.16. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS WITH HEURISTICS349
Size Results of configuration Shattered Document with Markov Chains with heuristics
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 36 4 4 4.00 4 4 0 1.00 1.00 Oracle 2 26 4 6 4.34 6 4 2 0.66 1.50 Oracle 3 34 4 6 4.25 6 4 2 0.66 1.50 Oracle 4 30 4 6 4.29 6 4 2 0.66 1.50 Oracle 5 32 4 4 4.00 4 4 0 1.00 1.00 Oracle 6 45 4 5 4.09 5 4 1 0.80 1.25 Oracle 7 43 3 6 3.91 5 3 2 0.60 1.66 Oracle 8 42 3 7 3.82 4 3 1 0.75 1.34 Oracle 9 30 3 7 4.29 7 4 3 0.57 1.75 Oracle 10 65 3 4 3.25 4 3 1 0.75 1.34 Oracle 11 37 3 4 3.09 4 3 1 0.75 1.34 Oracle 12 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 13 62 4 5 4.13 5 4 1 0.80 1.25 Oracle 14 46 4 5 4.18 5 4 1 0.80 1.25 Oracle 15 26 3 5 3.25 5 3 2 0.60 1.66 Oracle 16 21 3 4 3.50 4 3 1 0.75 1.34 Oracle 17 31 3 5 3.45 5 3 2 0.60 1.66 Oracle 18 38 3 5 3.45 5 4 1 0.80 1.25 Oracle 19 24 4 4 4.00 4 4 0 1.00 1.00 Oracle 20 27 4 7 4.50 7 4 3 0.57 1.75 Oracle 21 46 4 7 4.60 7 4 3 0.57 1.75 Oracle 22 43 4 7 4.30 7 4 3 0.57 1.75 Oracle 23 53 3 7 3.53 7 3 4 0.43 2.34 Oracle 24 47 4 7 4.27 7 4 3 0.57 1.75 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 27 51 3 5 3.92 5 4 1 0.80 1.25 Oracle 28 35 4 7 5.00 5 4 1 0.80 1.25 Oracle 29 26 4 7 5.20 7 4 3 0.57 1.75 Oracle 30 49 3 6 3.77 6 4 2 0.66 1.50 Oracle 31 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 32 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 33 53 4 7 4.41 7 4 3 0.57 1.75 Oracle 34 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 35 38 3 7 3.80 7 3 4 0.43 2.34 Oracle 36 52 4 4 4.00 4 4 0 1.00 1.00 Oracle 37 40 3 7 3.63 7 3 4 0.43 2.34 Oracle 38 40 3 6 3.63 6 3 3 0.50 2.00 Oracle 39 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 40 61 4 7 4.36 7 4 3 0.57 1.75 Oracle 41 34 4 6 4.25 6 4 2 0.66 1.50 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 52 5 7 5.20 7 5 2 0.71 1.40 Oracle 44 27 4 7 4.50 7 4 3 0.57 1.75 Oracle 45 37 4 5 4.11 5 4 1 0.80 1.25 Oracle 46 37 3 7 3.70 7 3 4 0.43 2.34 Oracle 47 37 3 6 3.70 6 3 3 0.50 2.00 Oracle 48 50 3 7 3.84 6 4 2 0.66 1.50 Oracle 49 35 3 4 3.50 4 4 0 1.00 1.00 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 2196 205 299 224.10 292 211 81 36.72 72.90 Average 43.92 4.10 5.98 4.48 5.84 4.22 1.62 0.73 1.46 Micro avg ····················· 0.72 1.38 350 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Markov Chains with heuristics
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 2 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 3 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 4 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 6 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 7 5 6 4 4 4 4 3 3 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 8 4 7 4 4 3 3 4 3 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 9 7 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 10 4 3 3 4 3 3 3 3 4 3 3 3 3 4 3 4 3 3 3 3 Oracle 11 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 12 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 13 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 14 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 15 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 16 4 4 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 17 5 4 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 18 5 4 3 3 3 4 3 3 3 3 4 4 4 4 4 4 4 4 4 4 Oracle 19 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 20 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 21 7 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 22 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 23 7 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 24 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 27 5 3 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 28 5 5 5 5 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 29 7 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 30 6 4 4 3 4 3 3 4 3 3 4 4 4 4 4 4 4 4 4 4 Oracle 31 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 32 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 33 7 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 34 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 35 7 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 36 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 37 7 3 4 4 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 38 6 3 3 3 4 4 4 3 4 3 3 3 3 3 3 3 3 3 3 3 Oracle 39 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 40 7 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 41 6 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 7 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 44 7 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 45 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 46 7 3 3 4 4 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 Oracle 47 6 3 4 3 3 3 4 4 4 3 3 3 3 3 3 3 3 3 3 3 Oracle 48 6 7 4 4 3 3 4 3 3 3 3 3 4 4 4 4 4 4 4 4 Oracle 49 4 3 4 4 4 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 292 239 215 215 217 211 211 210 211 210 210 210 211 212 211 212 211 211 211 211 Average 5.84 4.78 4.30 4.30 4.34 4.22 4.22 4.20 4.22 4.20 4.20 4.20 4.22 4.24 4.22 4.24 4.22 4.22 4.22 4.22 D.16. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS WITH HEURISTICS351
Cost Results of configuration Shattered Document with Markov Chains with heuristics
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 36.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 2 26.00 4.00 6.00 4.34 6.00 4.00 2.00 0.66 1.50 Oracle 3 34.00 4.00 6.00 4.25 6.00 4.00 2.00 0.66 1.50 Oracle 4 30.00 4.00 6.00 4.29 6.00 4.00 2.00 0.66 1.50 Oracle 5 32.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 6 45.00 4.00 5.00 4.09 5.00 4.00 1.00 0.80 1.25 Oracle 7 43.00 3.00 6.00 3.91 5.00 3.00 2.00 0.60 1.66 Oracle 8 42.00 3.00 7.00 3.82 4.00 3.00 1.00 0.75 1.34 Oracle 9 30.00 3.00 7.00 4.29 7.00 4.00 3.00 0.57 1.75 Oracle 10 65.00 3.00 4.00 3.25 4.00 3.00 1.00 0.75 1.34 Oracle 11 37.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 12 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 13 62.00 4.00 5.00 4.13 5.00 4.00 1.00 0.80 1.25 Oracle 14 46.00 4.00 5.00 4.18 5.00 4.00 1.00 0.80 1.25 Oracle 15 26.00 3.00 5.00 3.25 5.00 3.00 2.00 0.60 1.66 Oracle 16 21.00 3.00 4.00 3.50 4.00 3.00 1.00 0.75 1.34 Oracle 17 31.00 3.00 5.00 3.45 5.00 3.00 2.00 0.60 1.66 Oracle 18 38.00 3.00 5.00 3.45 5.00 4.00 1.00 0.80 1.25 Oracle 19 24.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 20 27.00 4.00 7.00 4.50 7.00 4.00 3.00 0.57 1.75 Oracle 21 46.00 4.00 7.00 4.60 7.00 4.00 3.00 0.57 1.75 Oracle 22 43.00 4.00 7.00 4.30 7.00 4.00 3.00 0.57 1.75 Oracle 23 53.00 3.00 7.00 3.53 7.00 3.00 4.00 0.43 2.34 Oracle 24 47.00 4.00 7.00 4.27 7.00 4.00 3.00 0.57 1.75 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 27 51.00 3.00 5.00 3.92 5.00 4.00 1.00 0.80 1.25 Oracle 28 35.00 4.00 7.00 5.00 5.00 4.00 1.00 0.80 1.25 Oracle 29 26.00 4.00 7.00 5.20 7.00 4.00 3.00 0.57 1.75 Oracle 30 49.00 3.00 6.00 3.77 6.00 4.00 2.00 0.66 1.50 Oracle 31 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 32 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 33 53.00 4.00 7.00 4.41 7.00 4.00 3.00 0.57 1.75 Oracle 34 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 35 38.00 3.00 7.00 3.80 7.00 3.00 4.00 0.43 2.34 Oracle 36 52.00 4.00 4.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 37 40.00 3.00 7.00 3.63 7.00 3.00 4.00 0.43 2.34 Oracle 38 40.00 3.00 6.00 3.63 6.00 3.00 3.00 0.50 2.00 Oracle 39 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 40 61.00 4.00 7.00 4.36 7.00 4.00 3.00 0.57 1.75 Oracle 41 34.00 4.00 6.00 4.25 6.00 4.00 2.00 0.66 1.50 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 52.00 5.00 7.00 5.20 7.00 5.00 2.00 0.71 1.40 Oracle 44 27.00 4.00 7.00 4.50 7.00 4.00 3.00 0.57 1.75 Oracle 45 37.00 4.00 5.00 4.11 5.00 4.00 1.00 0.80 1.25 Oracle 46 37.00 3.00 7.00 3.70 7.00 3.00 4.00 0.43 2.34 Oracle 47 37.00 3.00 6.00 3.70 6.00 3.00 3.00 0.50 2.00 Oracle 48 50.00 3.00 7.00 3.84 6.00 4.00 2.00 0.66 1.50 Oracle 49 35.00 3.00 4.00 3.50 4.00 4.00 0.00 1.00 1.00 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 2196.00 205.00 299.00 224.10 292.00 211.00 81.00 36.72 72.90 Average 43.92 4.10 5.98 4.48 5.84 4.22 1.62 0.73 1.46 Micro avg ····················· 0.72 1.38 352 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Markov Chains with heuristics
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 2 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 3 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 4 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 5 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 6 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 7 5.00 6.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 8 4.00 7.00 4.00 4.00 3.00 3.00 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 9 7.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 10 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 Oracle 11 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 12 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 13 5.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 14 5.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 15 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 16 4.00 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 17 5.00 4.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 18 5.00 4.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 19 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 20 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 21 7.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 22 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 23 7.00 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 24 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 27 5.00 3.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 28 5.00 5.00 5.00 5.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 29 7.00 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 30 6.00 4.00 4.00 3.00 4.00 3.00 3.00 4.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 31 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 32 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 33 7.00 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 34 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 35 7.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 36 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 37 7.00 3.00 4.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 38 6.00 3.00 3.00 3.00 4.00 4.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 39 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 40 7.00 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 41 6.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 7.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 44 7.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 45 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 46 7.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 47 6.00 3.00 4.00 3.00 3.00 3.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 48 6.00 7.00 4.00 4.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 49 4.00 3.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 292 239 215 215 217 211 211 210 211 210 210 210 211 212 211 212 211 211 211 211 Average 5.84 4.78 4.30 4.30 4.34 4.22 4.22 4.20 4.22 4.20 4.20 4.20 4.22 4.24 4.22 4.24 4.22 4.22 4.22 4.22 D.17. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, TRIMMED TREE353 D.17 Complete results of configuration Shattered Document, Trimmed Tree
Like Shattered Document, but with a trimmed document tree.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.96
Size0 = 5.06
Size1 = 3.18 Size gain = 1.59
Cost0 = 5.04
Cost1 = 3.16 Cost gain = 1.59 354 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document, Trimmed Tree
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 0 0 0 9 0.00 1.00 Oracle 2 30 6 0 0 0 0 6 0.00 1.00 Oracle 3 40 8 0 0 0 0 8 0.00 1.00 Oracle 4 46 7 0 0 0 0 7 0.00 1.00 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 0 0 0 0 7 0.00 1.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 0 0 0 8 0.00 1.00 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 0 0 0 9 0.00 1.00 Oracle 18 269 11 0 0 0 0 11 0.00 1.00 Oracle 19 290 6 0 0 0 0 6 0.00 1.00 Oracle 20 307 6 0 3 0 3 3 0.50 0.50 Oracle 21 366 10 0 0 0 0 10 0.00 1.00 Oracle 22 410 10 0 0 0 0 10 0.00 1.00 Oracle 23 455 15 0 0 0 0 15 0.00 1.00 Oracle 24 508 11 0 1 0 1 10 0.09 0.91 Oracle 25 512 9 0 0 0 0 9 0.00 1.00 Oracle 26 513 15 0 0 0 0 15 0.00 1.00 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 0 0 0 7 0.00 1.00 Oracle 29 542 5 0 0 0 0 5 0.00 1.00 Oracle 30 554 13 0 0 0 0 13 0.00 1.00 Oracle 31 601 11 0 0 0 0 11 0.00 1.00 Oracle 32 606 14 0 0 0 0 14 0.00 1.00 Oracle 33 616 12 0 0 0 0 12 0.00 1.00 Oracle 34 649 8 0 5 0 5 3 0.63 0.38 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 1 0 1 10 0.09 0.91 Oracle 39 786 7 0 0 0 0 7 0.00 1.00 Oracle 40 840 14 0 0 0 0 14 0.00 1.00 Oracle 41 846 8 0 0 0 0 8 0.00 1.00 Oracle 42 883 1 0 1 0 1 0 1.00 0.00 Oracle 43 890 10 0 0 0 0 10 0.00 1.00 Oracle 44 897 6 0 0 0 0 6 0.00 1.00 Oracle 45 899 9 0 0 0 0 9 0.00 1.00 Oracle 46 929 10 0 1 0 1 9 0.10 0.90 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 0 0 0 0 13 0.00 1.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 9 0 0 9 0 1.00 0.00 Total ··· 500 9 12 0 21 479 3.41 46.59 Average ··· 10.00 0.18 0.24 0.00 0.42 9.58 0.07 0.93 Micro avg ····················· 0.04 0.96 D.17. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, TRIMMED TREE355
Duration Results of configuration Shattered Document, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 9.68 1.03 1.38 1.08 1.38 1.04 0.34 0.75 1.34 Oracle 2 8.88 1.03 2.69 1.48 2.69 1.03 1.66 0.38 2.60 Oracle 3 10.16 1.02 1.87 1.27 1.87 1.38 0.48 0.74 1.34 Oracle 4 8.84 1.03 1.77 1.27 1.77 1.03 0.73 0.59 1.72 Oracle 5 13.30 1.34 2.30 1.66 2.30 1.34 0.96 0.59 1.72 Oracle 6 13.34 1.04 1.79 1.21 1.79 1.33 0.47 0.74 1.35 Oracle 7 16.22 1.07 1.91 1.48 1.91 1.47 0.45 0.77 1.30 Oracle 8 13.43 1.02 2.23 1.22 1.91 1.04 0.88 0.55 1.84 Oracle 9 8.38 1.02 2.20 1.20 2.20 1.03 1.17 0.47 2.14 Oracle 10 19.95 0.91 1.76 1.00 1.76 0.91 0.84 0.52 1.92 Oracle 11 14.08 1.02 2.24 1.17 2.24 1.02 1.22 0.46 2.19 Oracle 12 13.72 1.02 1.40 1.05 1.40 1.02 0.38 0.73 1.36 Oracle 13 10.34 0.60 1.03 0.69 1.03 0.61 0.41 0.59 1.68 Oracle 14 11.35 1.02 1.09 1.03 1.05 1.09 -0.04 1.04 0.96 Oracle 15 5.25 0.60 0.96 0.66 0.96 0.61 0.35 0.63 1.58 Oracle 16 7.42 0.95 2.24 1.23 2.24 0.95 1.28 0.43 2.34 Oracle 17 9.99 0.61 3.08 1.11 3.08 0.95 2.14 0.30 3.27 Oracle 18 10.11 0.61 1.41 0.92 1.41 0.94 0.47 0.67 1.50 Oracle 19 7.42 0.93 2.73 1.23 2.73 0.93 1.80 0.34 2.94 Oracle 20 12.12 0.94 2.66 2.02 1.02 0.94 0.09 0.91 1.09 Oracle 21 11.75 1.02 2.45 1.17 2.45 1.03 1.43 0.42 2.39 Oracle 22 10.63 1.02 1.36 1.06 1.36 1.03 0.33 0.76 1.32 Oracle 23 17.13 1.02 2.65 1.14 2.65 1.03 1.62 0.39 2.57 Oracle 24 14.13 1.02 2.73 1.28 2.73 1.02 1.70 0.38 2.66 Oracle 25 5.55 0.61 0.65 0.62 0.61 0.65 -0.04 1.06 0.94 Oracle 26 16.91 1.02 1.38 1.13 1.38 1.04 0.34 0.75 1.34 Oracle 27 15.49 1.02 1.83 1.20 1.83 1.05 0.77 0.58 1.73 Oracle 28 8.33 1.02 1.31 1.19 1.31 1.03 0.28 0.78 1.27 Oracle 29 6.32 1.02 1.75 1.27 1.75 1.48 0.27 0.84 1.18 Oracle 30 19.18 1.44 1.80 1.48 1.80 1.45 0.35 0.80 1.24 Oracle 31 14.63 1.02 1.81 1.33 1.81 1.46 0.35 0.80 1.24 Oracle 32 18.11 0.94 1.79 1.30 1.79 0.96 0.83 0.54 1.85 Oracle 33 10.14 0.45 1.28 0.84 0.87 0.46 0.41 0.53 1.89 Oracle 34 17.43 0.61 2.77 2.18 2.67 0.61 2.07 0.23 4.40 Oracle 35 5.56 0.45 0.65 0.55 0.46 0.62 -0.16 1.33 0.75 Oracle 36 12.31 0.61 1.43 0.95 1.43 1.03 0.41 0.72 1.39 Oracle 37 11.87 0.61 1.83 1.08 1.83 1.03 0.79 0.56 1.77 Oracle 38 13.91 0.61 2.70 1.27 2.70 1.03 1.66 0.38 2.61 Oracle 39 10.65 1.45 1.90 1.52 1.45 1.51 -0.05 1.04 0.96 Oracle 40 19.03 1.02 2.74 1.36 2.74 1.04 1.71 0.38 2.65 Oracle 41 9.03 1.02 1.80 1.13 1.80 1.02 0.77 0.57 1.75 Oracle 42 2.73 2.73 2.73 2.73 2.73 2.73 0.00 1.00 1.00 Oracle 43 8.05 0.61 1.09 0.80 1.09 0.95 0.15 0.87 1.16 Oracle 44 11.86 1.84 2.66 1.98 2.66 1.84 0.83 0.69 1.45 Oracle 45 8.18 0.61 2.28 0.91 2.28 0.94 1.34 0.41 2.43 Oracle 46 10.95 0.61 2.66 1.09 2.66 0.89 1.77 0.34 3.00 Oracle 47 10.07 0.63 2.23 1.01 2.23 0.93 1.30 0.41 2.40 Oracle 48 13.84 0.90 1.33 1.06 1.31 0.90 0.42 0.68 1.47 Oracle 49 11.23 1.03 1.46 1.13 1.46 1.04 0.42 0.71 1.41 Oracle 50 272.43 30.07 30.44 30.27 30.44 30.22 0.22 0.99 1.01 Total 841.39 76.88 124.22 90.00 121.04 82.69 38.35 32.16 89.39 Average 16.83 1.54 2.48 1.80 2.42 1.66 0.77 0.64 1.79 Micro avg ····················· 0.68 1.46 356 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document, Trimmed Tree
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 1.38 1.04 1.03 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 2 2.69 1.38 1.38 1.03 1.38 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 3 1.87 1.40 1.04 1.02 1.02 1.40 1.03 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 4 1.77 1.33 1.03 1.03 1.33 1.33 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 5 2.30 1.87 1.88 1.84 1.38 1.35 1.35 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 1.34 Oracle 6 1.79 1.35 1.04 1.32 1.04 1.04 1.04 1.04 1.05 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 1.33 Oracle 7 1.91 1.49 1.48 1.48 1.07 1.47 1.46 1.46 1.46 1.46 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 Oracle 8 1.91 2.23 1.03 1.02 1.03 1.04 1.02 1.03 1.03 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 9 2.20 1.04 1.02 1.02 1.02 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 10 1.76 1.05 1.04 1.37 0.94 0.92 0.92 0.92 0.91 0.91 0.92 0.91 0.92 0.91 0.91 0.91 0.94 0.93 0.93 0.91 Oracle 11 2.24 1.48 1.05 1.02 1.03 1.03 1.03 1.03 1.03 1.03 1.09 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 12 1.40 1.02 1.03 1.02 1.02 1.03 1.02 1.02 1.03 1.02 1.02 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 13 1.03 0.61 0.61 0.60 0.61 0.62 0.61 0.61 0.86 0.86 0.87 0.61 0.62 0.62 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 14 1.05 1.02 1.02 1.02 1.02 1.03 1.03 1.02 1.03 1.03 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 1.09 Oracle 15 0.96 0.61 0.60 0.62 0.61 0.62 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 16 2.24 1.37 0.95 0.95 0.97 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 17 3.08 0.61 0.94 0.62 0.97 0.95 0.95 0.94 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 18 1.41 0.93 0.93 0.93 0.94 0.61 0.63 0.94 0.94 0.93 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 Oracle 19 2.73 0.94 0.94 0.94 0.94 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 Oracle 20 1.02 2.65 2.66 2.65 2.21 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 Oracle 21 2.45 1.02 1.03 1.03 1.06 1.04 1.03 1.03 1.02 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 22 1.36 1.04 1.02 1.02 1.02 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 23 2.65 1.02 1.03 1.03 1.09 1.03 1.02 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 24 2.73 2.13 1.02 1.04 1.04 1.02 1.04 1.03 1.03 1.03 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 25 0.61 0.61 0.61 0.61 0.62 0.61 0.62 0.61 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 0.65 Oracle 26 1.38 1.38 1.02 1.04 1.38 1.03 1.38 1.02 1.04 1.03 1.03 1.07 1.03 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 27 1.83 1.35 1.02 1.36 1.02 1.02 1.35 1.35 1.02 1.03 1.03 1.02 1.05 1.05 1.05 1.05 1.05 1.05 1.05 1.05 Oracle 28 1.31 1.02 1.03 1.30 1.31 1.31 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 29 1.75 1.04 1.02 1.03 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 1.48 Oracle 30 1.80 1.45 1.45 1.44 1.44 1.47 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 Oracle 31 1.81 1.03 1.02 1.45 1.45 1.45 1.46 1.03 1.02 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 32 1.79 1.45 1.45 1.45 1.45 1.45 1.03 1.03 1.03 1.38 1.35 1.35 0.94 0.96 0.96 0.96 0.96 0.96 0.96 0.96 Oracle 33 0.87 1.27 1.27 1.27 1.28 0.87 0.87 0.61 0.46 0.45 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 0.46 Oracle 34 2.67 2.77 2.68 2.66 2.66 2.73 0.64 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 0.61 Oracle 35 0.46 0.45 0.45 0.45 0.62 0.65 0.62 0.61 0.63 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 0.62 Oracle 36 1.43 0.96 0.61 0.97 0.61 0.62 0.64 0.62 1.39 1.38 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 37 1.83 0.95 1.00 0.95 0.61 1.03 1.02 1.03 1.03 1.38 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 38 2.70 2.25 0.63 0.94 0.61 0.94 1.40 1.03 1.36 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 1.03 Oracle 39 1.45 1.45 1.90 1.45 1.45 1.45 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 1.51 Oracle 40 2.74 2.30 2.25 1.45 1.02 1.03 1.02 1.03 1.02 1.04 1.04 1.03 1.02 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 41 1.80 1.05 1.02 1.02 1.04 1.04 1.04 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 1.02 Oracle 42 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 2.73 Oracle 43 1.09 1.02 0.95 0.61 0.62 0.95 0.62 0.62 0.63 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 Oracle 44 2.66 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.84 Oracle 45 2.28 0.63 0.95 0.61 0.95 0.61 0.61 0.62 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 Oracle 46 2.66 2.54 0.89 0.62 0.89 0.62 0.62 0.61 0.62 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.89 Oracle 47 2.23 0.95 0.93 0.66 0.97 0.63 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 Oracle 48 1.31 1.04 1.04 1.04 0.92 0.90 0.93 1.31 0.91 0.91 1.33 1.32 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 Oracle 49 1.46 1.07 1.04 1.04 1.03 1.03 1.03 1.46 1.03 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 1.04 Oracle 50 30.44 30.09 30.38 30.37 30.34 30.07 30.27 30.26 30.22 30.22 30.22 30.22 30.22 30.22 30.22 30.22 30.22 30.22 30.22 30.22 Total 121 95 88 87 86 84 82 82 82 84 84 83 83 83 83 83 83 83 83 83 Average 2.42 1.91 1.76 1.74 1.72 1.68 1.64 1.64 1.65 1.68 1.67 1.67 1.66 1.66 1.66 1.66 1.66 1.66 1.66 1.66 D.17. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, TRIMMED TREE357
Size Results of configuration Shattered Document, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 28 3 4 3.11 4 3 1 0.75 1.34 Oracle 2 25 3 7 4.16 7 3 4 0.43 2.34 Oracle 3 29 3 5 3.63 5 4 1 0.80 1.25 Oracle 4 26 3 5 3.71 5 3 2 0.60 1.66 Oracle 5 37 4 6 4.63 6 4 2 0.66 1.50 Oracle 6 39 3 5 3.55 5 4 1 0.80 1.25 Oracle 7 44 3 5 4.00 5 4 1 0.80 1.25 Oracle 8 38 3 6 3.45 5 3 2 0.60 1.66 Oracle 9 24 3 6 3.43 6 3 3 0.50 2.00 Oracle 10 63 3 5 3.15 5 3 2 0.60 1.66 Oracle 11 40 3 6 3.34 6 3 3 0.50 2.00 Oracle 12 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 13 34 2 3 2.27 3 2 1 0.66 1.50 Oracle 14 33 3 3 3.00 3 3 0 1.00 1.00 Oracle 15 17 2 3 2.13 3 2 1 0.66 1.50 Oracle 16 22 3 6 3.66 6 3 3 0.50 2.00 Oracle 17 29 2 7 3.22 7 3 4 0.43 2.34 Oracle 18 32 2 4 2.91 4 3 1 0.75 1.34 Oracle 19 22 3 7 3.66 7 3 4 0.43 2.34 Oracle 20 33 3 7 5.50 3 3 0 1.00 1.00 Oracle 21 33 3 6 3.30 6 3 3 0.50 2.00 Oracle 22 31 3 4 3.10 4 3 1 0.75 1.34 Oracle 23 49 3 7 3.27 7 3 4 0.43 2.34 Oracle 24 40 3 7 3.63 7 3 4 0.43 2.34 Oracle 25 18 2 2 2.00 2 2 0 1.00 1.00 Oracle 26 49 3 4 3.27 4 3 1 0.75 1.34 Oracle 27 45 3 5 3.46 5 3 2 0.60 1.66 Oracle 28 25 3 4 3.57 4 3 1 0.75 1.34 Oracle 29 18 3 5 3.60 5 4 1 0.80 1.25 Oracle 30 53 4 5 4.08 5 4 1 0.80 1.25 Oracle 31 41 3 5 3.73 5 4 1 0.80 1.25 Oracle 32 52 3 5 3.71 5 3 2 0.60 1.66 Oracle 33 35 2 4 2.91 3 2 1 0.66 1.50 Oracle 34 46 2 7 5.75 7 2 5 0.29 3.50 Oracle 35 20 2 2 2.00 2 2 0 1.00 1.00 Oracle 36 37 2 4 2.84 4 3 1 0.75 1.34 Oracle 37 35 2 5 3.18 5 3 2 0.60 1.66 Oracle 38 40 2 7 3.63 7 3 4 0.43 2.34 Oracle 39 29 4 5 4.14 4 4 0 1.00 1.00 Oracle 40 53 3 7 3.79 7 3 4 0.43 2.34 Oracle 41 26 3 5 3.25 5 3 2 0.60 1.66 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 25 2 3 2.50 3 3 0 1.00 1.00 Oracle 44 32 5 7 5.34 7 5 2 0.71 1.40 Oracle 45 25 2 6 2.78 6 3 3 0.50 2.00 Oracle 46 33 2 7 3.30 7 3 4 0.43 2.34 Oracle 47 31 2 6 3.10 6 3 3 0.50 2.00 Oracle 48 43 3 4 3.30 4 3 1 0.75 1.34 Oracle 49 32 3 4 3.20 4 3 1 0.75 1.34 Oracle 50 40 4 7 4.45 7 4 3 0.57 1.75 Total 1698 145 260 176.78 253 159 94 33.41 81.48 Average 33.96 2.90 5.20 3.54 5.06 3.18 1.88 0.67 1.63 Micro avg ····················· 0.63 1.59 358 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document, Trimmed Tree
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 2 7 4 4 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 3 5 4 3 3 3 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 4 5 4 3 3 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 5 6 5 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 6 5 4 3 4 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 Oracle 7 5 4 4 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 8 5 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 9 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 10 5 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 11 6 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 12 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 13 3 2 2 2 2 2 2 2 3 3 3 2 2 2 2 2 2 2 2 2 Oracle 14 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 15 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 16 6 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 17 7 2 3 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 18 4 3 3 3 3 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 19 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 20 3 7 7 7 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 21 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 22 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 23 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 24 7 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 25 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 26 4 4 3 3 4 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 27 5 4 3 4 3 3 4 4 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 28 4 3 3 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 29 5 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 30 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 31 5 3 3 4 4 4 4 3 3 4 4 4 4 4 4 4 4 4 4 4 Oracle 32 5 4 4 4 4 4 3 3 3 4 4 4 3 3 3 3 3 3 3 3 Oracle 33 3 4 4 4 4 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 34 7 7 7 7 7 7 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 35 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 36 4 3 2 3 2 2 2 2 4 4 3 3 3 3 3 3 3 3 3 3 Oracle 37 5 3 3 3 2 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 38 7 6 2 3 2 3 4 3 4 3 3 3 3 3 3 3 3 3 3 3 Oracle 39 4 4 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 40 7 6 6 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 41 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 3 3 3 2 2 3 2 2 2 3 3 3 3 3 3 3 3 3 3 3 Oracle 44 7 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 45 6 2 3 2 3 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 46 7 7 3 2 3 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 Oracle 47 6 3 3 2 3 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 48 4 3 3 3 3 3 3 4 3 3 4 4 3 3 3 3 3 3 3 3 Oracle 49 4 3 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 50 7 4 4 4 4 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Total 253 190 171 169 167 163 156 156 158 163 162 161 159 159 159 159 159 159 159 159 Average 5.06 3.80 3.42 3.38 3.34 3.26 3.12 3.12 3.16 3.26 3.24 3.22 3.18 3.18 3.18 3.18 3.18 3.18 3.18 3.18 D.17. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT, TRIMMED TREE359
Cost Results of configuration Shattered Document, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 28.00 3.00 4.00 3.11 4.00 3.00 1.00 0.75 1.34 Oracle 2 25.00 3.00 7.00 4.16 7.00 3.00 4.00 0.43 2.34 Oracle 3 29.00 3.00 5.00 3.63 5.00 4.00 1.00 0.80 1.25 Oracle 4 26.00 3.00 5.00 3.71 5.00 3.00 2.00 0.60 1.66 Oracle 5 37.00 4.00 6.00 4.63 6.00 4.00 2.00 0.66 1.50 Oracle 6 39.00 3.00 5.00 3.55 5.00 4.00 1.00 0.80 1.25 Oracle 7 44.00 3.00 5.00 4.00 5.00 4.00 1.00 0.80 1.25 Oracle 8 38.00 3.00 6.00 3.45 5.00 3.00 2.00 0.60 1.66 Oracle 9 24.00 3.00 6.00 3.43 6.00 3.00 3.00 0.50 2.00 Oracle 10 63.00 3.00 5.00 3.15 5.00 3.00 2.00 0.60 1.66 Oracle 11 40.00 3.00 6.00 3.34 6.00 3.00 3.00 0.50 2.00 Oracle 12 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 13 34.00 2.00 3.00 2.27 3.00 2.00 1.00 0.66 1.50 Oracle 14 33.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 15 17.00 2.00 3.00 2.13 3.00 2.00 1.00 0.66 1.50 Oracle 16 22.00 3.00 6.00 3.66 6.00 3.00 3.00 0.50 2.00 Oracle 17 29.00 2.00 7.00 3.22 7.00 3.00 4.00 0.43 2.34 Oracle 18 32.00 2.00 4.00 2.91 4.00 3.00 1.00 0.75 1.34 Oracle 19 22.00 3.00 7.00 3.66 7.00 3.00 4.00 0.43 2.34 Oracle 20 33.00 3.00 7.00 5.50 3.00 3.00 0.00 1.00 1.00 Oracle 21 33.00 3.00 6.00 3.30 6.00 3.00 3.00 0.50 2.00 Oracle 22 31.00 3.00 4.00 3.10 4.00 3.00 1.00 0.75 1.34 Oracle 23 49.00 3.00 7.00 3.27 7.00 3.00 4.00 0.43 2.34 Oracle 24 40.00 3.00 7.00 3.63 7.00 3.00 4.00 0.43 2.34 Oracle 25 18.00 2.00 2.00 2.00 2.00 2.00 0.00 1.00 1.00 Oracle 26 49.00 3.00 4.00 3.27 4.00 3.00 1.00 0.75 1.34 Oracle 27 45.00 3.00 5.00 3.46 5.00 3.00 2.00 0.60 1.66 Oracle 28 25.00 3.00 4.00 3.57 4.00 3.00 1.00 0.75 1.34 Oracle 29 18.00 3.00 5.00 3.60 5.00 4.00 1.00 0.80 1.25 Oracle 30 53.00 4.00 5.00 4.08 5.00 4.00 1.00 0.80 1.25 Oracle 31 41.00 3.00 5.00 3.73 5.00 4.00 1.00 0.80 1.25 Oracle 32 52.00 3.00 5.00 3.71 5.00 3.00 2.00 0.60 1.66 Oracle 33 35.00 2.00 4.00 2.91 3.00 2.00 1.00 0.66 1.50 Oracle 34 46.00 2.00 7.00 5.75 7.00 2.00 5.00 0.29 3.50 Oracle 35 20.00 2.00 2.00 2.00 2.00 2.00 0.00 1.00 1.00 Oracle 36 37.00 2.00 4.00 2.84 4.00 3.00 1.00 0.75 1.34 Oracle 37 35.00 2.00 5.00 3.18 5.00 3.00 2.00 0.60 1.66 Oracle 38 40.00 2.00 7.00 3.63 7.00 3.00 4.00 0.43 2.34 Oracle 39 29.00 4.00 5.00 4.14 4.00 4.00 0.00 1.00 1.00 Oracle 40 53.00 3.00 7.00 3.79 7.00 3.00 4.00 0.43 2.34 Oracle 41 26.00 3.00 5.00 3.25 5.00 3.00 2.00 0.60 1.66 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 25.00 2.00 3.00 2.50 3.00 3.00 0.00 1.00 1.00 Oracle 44 32.00 5.00 7.00 5.34 7.00 5.00 2.00 0.71 1.40 Oracle 45 25.00 2.00 6.00 2.78 6.00 3.00 3.00 0.50 2.00 Oracle 46 33.00 2.00 7.00 3.30 7.00 3.00 4.00 0.43 2.34 Oracle 47 31.00 2.00 6.00 3.10 6.00 3.00 3.00 0.50 2.00 Oracle 48 43.00 3.00 4.00 3.30 4.00 3.00 1.00 0.75 1.34 Oracle 49 32.00 3.00 4.00 3.20 4.00 3.00 1.00 0.75 1.34 Oracle 50 31.00 3.00 6.00 3.45 6.00 3.00 3.00 0.50 2.00 Total 1689.00 144.00 259.00 175.78 252.00 158.00 94.00 33.34 81.73 Average 33.78 2.88 5.18 3.52 5.04 3.16 1.88 0.66 1.63 Micro avg ····················· 0.63 1.59 360 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document, Trimmed Tree
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 2 7.00 4.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 3 5.00 4.00 3.00 3.00 3.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 4 5.00 4.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 5 6.00 5.00 5.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 6 5.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 7 5.00 4.00 4.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 8 5.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 9 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 10 5.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 11 6.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 12 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 13 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 14 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 15 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 16 6.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 17 7.00 2.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 18 4.00 3.00 3.00 3.00 3.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 19 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 20 3.00 7.00 7.00 7.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 21 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 22 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 23 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 24 7.00 6.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 25 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 26 4.00 4.00 3.00 3.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 27 5.00 4.00 3.00 4.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 28 4.00 3.00 3.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 29 5.00 3.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 30 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 31 5.00 3.00 3.00 4.00 4.00 4.00 4.00 3.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 32 5.00 4.00 4.00 4.00 4.00 4.00 3.00 3.00 3.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 33 3.00 4.00 4.00 4.00 4.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 34 7.00 7.00 7.00 7.00 7.00 7.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 35 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 36 4.00 3.00 2.00 3.00 2.00 2.00 2.00 2.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 37 5.00 3.00 3.00 3.00 2.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 38 7.00 6.00 2.00 3.00 2.00 3.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 39 4.00 4.00 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 40 7.00 6.00 6.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 41 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 3.00 3.00 3.00 2.00 2.00 3.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 44 7.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 45 6.00 2.00 3.00 2.00 3.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 46 7.00 7.00 3.00 2.00 3.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 47 6.00 3.00 3.00 2.00 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 48 4.00 3.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 49 4.00 3.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 50 6.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Total 252 189 170 168 166 162 155 155 157 162 161 160 158 158 158 158 158 158 158 158 Average 5.04 3.78 3.40 3.36 3.32 3.24 3.10 3.10 3.14 3.24 3.22 3.20 3.16 3.16 3.16 3.16 3.16 3.16 3.16 3.16 D.18. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL, TRIMMED TREE361 D.18 Complete results of configuration Structural, Trimmed Tree
Like Structural, but with a trimmed document tree.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 100 Page size (number of items) = 10 Quantity of expanded items = 0 Update travel matrix strategy = False Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 6.00 Weight of Next links = 0.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 1.00
Size0 = 3.00
Size1 = 3.00 Size gain = 1.00
Cost0 = 11.86
Cost1 = 11.86 Cost gain = 1.00 362 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Structural, Trimmed Tree
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 2 0 0 0 0 2 0.00 1.00 Oracle 2 30 2 0 0 0 0 2 0.00 1.00 Oracle 3 40 2 0 0 0 0 2 0.00 1.00 Oracle 4 46 2 0 0 0 0 2 0.00 1.00 Oracle 5 56 2 0 0 0 0 2 0.00 1.00 Oracle 6 85 2 0 0 0 0 2 0.00 1.00 Oracle 7 108 2 0 0 0 0 2 0.00 1.00 Oracle 8 125 2 0 0 0 0 2 0.00 1.00 Oracle 9 128 2 0 0 0 0 2 0.00 1.00 Oracle 10 130 2 0 0 0 0 2 0.00 1.00 Oracle 11 138 2 0 0 0 0 2 0.00 1.00 Oracle 12 157 2 0 0 0 0 2 0.00 1.00 Oracle 13 171 2 0 0 0 0 2 0.00 1.00 Oracle 14 173 2 0 0 0 0 2 0.00 1.00 Oracle 15 241 2 0 0 0 0 2 0.00 1.00 Oracle 16 265 2 0 0 0 0 2 0.00 1.00 Oracle 17 266 2 0 0 0 0 2 0.00 1.00 Oracle 18 269 2 0 0 0 0 2 0.00 1.00 Oracle 19 290 2 0 0 0 0 2 0.00 1.00 Oracle 20 307 2 0 0 0 0 2 0.00 1.00 Oracle 21 366 2 0 0 0 0 2 0.00 1.00 Oracle 22 410 2 0 0 0 0 2 0.00 1.00 Oracle 23 455 2 0 0 0 0 2 0.00 1.00 Oracle 24 508 2 0 0 0 0 2 0.00 1.00 Oracle 25 512 2 0 0 0 0 2 0.00 1.00 Oracle 26 513 2 0 0 0 0 2 0.00 1.00 Oracle 27 516 2 0 0 0 0 2 0.00 1.00 Oracle 28 521 2 0 0 0 0 2 0.00 1.00 Oracle 29 542 2 0 0 0 0 2 0.00 1.00 Oracle 30 554 2 0 0 0 0 2 0.00 1.00 Oracle 31 601 2 0 0 0 0 2 0.00 1.00 Oracle 32 606 2 0 0 0 0 2 0.00 1.00 Oracle 33 616 2 0 0 0 0 2 0.00 1.00 Oracle 34 649 2 0 0 0 0 2 0.00 1.00 Oracle 35 723 2 0 0 0 0 2 0.00 1.00 Oracle 36 725 2 0 0 0 0 2 0.00 1.00 Oracle 37 738 2 0 0 0 0 2 0.00 1.00 Oracle 38 741 2 0 0 0 0 2 0.00 1.00 Oracle 39 786 2 0 0 0 0 2 0.00 1.00 Oracle 40 840 2 0 0 0 0 2 0.00 1.00 Oracle 41 846 2 0 0 0 0 2 0.00 1.00 Oracle 42 883 2 0 0 0 0 2 0.00 1.00 Oracle 43 890 2 0 0 0 0 2 0.00 1.00 Oracle 44 897 2 0 0 0 0 2 0.00 1.00 Oracle 45 899 2 0 0 0 0 2 0.00 1.00 Oracle 46 929 2 0 0 0 0 2 0.00 1.00 Oracle 47 936 2 0 0 0 0 2 0.00 1.00 Oracle 48 939 2 0 0 0 0 2 0.00 1.00 Oracle 49 944 2 0 0 0 0 2 0.00 1.00 Oracle 50 999 2 0 0 0 0 2 0.00 1.00 Total ··· 100 0 0 0 0 100 0.00 50.00 Average ··· 2.00 0.00 0.00 0.00 0.00 2.00 0.00 1.00 Micro avg ····················· 0.00 1.00 D.18. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL, TRIMMED TREE363
Duration Results of configuration Structural, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 0.70 0.34 0.35 0.35 0.35 0.34 0.01 0.98 1.02 Oracle 2 0.70 0.34 0.35 0.35 0.34 0.35 0.00 1.01 0.99 Oracle 3 0.72 0.35 0.37 0.36 0.35 0.37 -0.02 1.04 0.96 Oracle 4 0.73 0.35 0.38 0.37 0.38 0.35 0.03 0.91 1.09 Oracle 5 0.70 0.35 0.35 0.35 0.35 0.35 0.00 1.01 0.99 Oracle 6 0.70 0.34 0.35 0.35 0.35 0.34 0.01 0.98 1.02 Oracle 7 0.77 0.37 0.41 0.39 0.41 0.37 0.04 0.90 1.11 Oracle 8 0.69 0.34 0.34 0.34 0.34 0.34 0.01 0.99 1.01 Oracle 9 0.70 0.35 0.35 0.35 0.35 0.35 0.00 0.99 1.01 Oracle 10 0.70 0.35 0.35 0.35 0.35 0.35 -0.01 1.02 0.98 Oracle 11 0.70 0.34 0.35 0.34 0.35 0.34 0.00 0.99 1.01 Oracle 12 0.70 0.35 0.35 0.35 0.35 0.35 0.00 1.00 1.00 Oracle 13 0.39 0.19 0.20 0.20 0.19 0.20 -0.02 1.10 0.91 Oracle 14 0.70 0.35 0.35 0.35 0.35 0.35 0.01 0.98 1.02 Oracle 15 0.70 0.35 0.35 0.35 0.35 0.35 0.00 0.99 1.01 Oracle 16 0.70 0.34 0.35 0.35 0.34 0.35 -0.01 1.02 0.98 Oracle 17 0.70 0.35 0.35 0.35 0.35 0.35 0.00 1.00 1.00 Oracle 18 0.70 0.34 0.35 0.35 0.35 0.34 0.00 0.99 1.01 Oracle 19 0.70 0.35 0.35 0.35 0.35 0.35 0.00 0.99 1.01 Oracle 20 0.70 0.35 0.36 0.35 0.36 0.35 0.01 0.98 1.02 Oracle 21 0.70 0.35 0.35 0.35 0.35 0.35 0.00 1.00 1.00 Oracle 22 0.71 0.35 0.35 0.35 0.35 0.35 0.00 0.99 1.01 Oracle 23 0.70 0.35 0.36 0.35 0.35 0.36 -0.01 1.02 0.98 Oracle 24 0.72 0.35 0.37 0.36 0.37 0.35 0.02 0.95 1.05 Oracle 25 0.37 0.18 0.19 0.18 0.19 0.18 0.00 0.98 1.02 Oracle 26 0.70 0.34 0.35 0.35 0.35 0.34 0.01 0.98 1.02 Oracle 27 0.70 0.34 0.35 0.35 0.35 0.34 0.01 0.98 1.02 Oracle 28 0.73 0.35 0.38 0.36 0.38 0.35 0.02 0.95 1.06 Oracle 29 0.73 0.36 0.37 0.36 0.37 0.36 0.01 0.97 1.03 Oracle 30 0.71 0.35 0.36 0.35 0.35 0.36 -0.01 1.03 0.97 Oracle 31 0.73 0.35 0.38 0.37 0.35 0.38 -0.03 1.09 0.91 Oracle 32 0.70 0.35 0.35 0.35 0.35 0.35 0.00 1.01 0.99 Oracle 33 0.37 0.19 0.19 0.19 0.19 0.19 0.00 0.99 1.01 Oracle 34 0.73 0.35 0.38 0.37 0.35 0.38 -0.03 1.09 0.92 Oracle 35 0.37 0.18 0.19 0.18 0.19 0.18 0.00 0.99 1.01 Oracle 36 0.70 0.35 0.35 0.35 0.35 0.35 0.00 1.00 1.00 Oracle 37 0.70 0.35 0.35 0.35 0.35 0.35 0.00 0.99 1.01 Oracle 38 0.70 0.35 0.35 0.35 0.35 0.35 0.00 1.00 1.00 Oracle 39 0.70 0.35 0.36 0.35 0.36 0.35 0.01 0.98 1.02 Oracle 40 0.70 0.35 0.35 0.35 0.35 0.35 0.00 0.99 1.01 Oracle 41 0.73 0.35 0.38 0.36 0.38 0.35 0.02 0.94 1.06 Oracle 42 0.70 0.35 0.35 0.35 0.35 0.35 0.00 0.99 1.01 Oracle 43 0.70 0.35 0.35 0.35 0.35 0.35 0.00 0.99 1.01 Oracle 44 0.71 0.35 0.36 0.35 0.36 0.35 0.01 0.98 1.02 Oracle 45 0.70 0.35 0.35 0.35 0.35 0.35 0.00 1.00 1.00 Oracle 46 0.70 0.35 0.35 0.35 0.35 0.35 0.00 1.00 1.00 Oracle 47 0.72 0.34 0.37 0.36 0.37 0.34 0.02 0.94 1.07 Oracle 48 0.70 0.34 0.35 0.35 0.35 0.34 0.01 0.98 1.02 Oracle 49 0.70 0.35 0.35 0.35 0.35 0.35 0.00 1.00 1.00 Oracle 50 0.70 0.35 0.35 0.35 0.35 0.35 0.01 0.98 1.02 Total 34.02 16.81 17.21 17.02 17.08 16.95 0.13 49.70 50.38 Average 0.68 0.34 0.34 0.34 0.34 0.34 0.00 0.99 1.01 Micro avg ····················· 0.99 1.01 364 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Structural, Trimmed Tree
1 2 Oracle 1 0.35 0.34 Oracle 2 0.34 0.35 Oracle 3 0.35 0.37 Oracle 4 0.38 0.35 Oracle 5 0.35 0.35 Oracle 6 0.35 0.34 Oracle 7 0.41 0.37 Oracle 8 0.34 0.34 Oracle 9 0.35 0.35 Oracle 10 0.35 0.35 Oracle 11 0.35 0.34 Oracle 12 0.35 0.35 Oracle 13 0.19 0.20 Oracle 14 0.35 0.35 Oracle 15 0.35 0.35 Oracle 16 0.34 0.35 Oracle 17 0.35 0.35 Oracle 18 0.35 0.34 Oracle 19 0.35 0.35 Oracle 20 0.36 0.35 Oracle 21 0.35 0.35 Oracle 22 0.35 0.35 Oracle 23 0.35 0.36 Oracle 24 0.37 0.35 Oracle 25 0.19 0.18 Oracle 26 0.35 0.34 Oracle 27 0.35 0.34 Oracle 28 0.38 0.35 Oracle 29 0.37 0.36 Oracle 30 0.35 0.36 Oracle 31 0.35 0.38 Oracle 32 0.35 0.35 Oracle 33 0.19 0.19 Oracle 34 0.35 0.38 Oracle 35 0.19 0.18 Oracle 36 0.35 0.35 Oracle 37 0.35 0.35 Oracle 38 0.35 0.35 Oracle 39 0.36 0.35 Oracle 40 0.35 0.35 Oracle 41 0.38 0.35 Oracle 42 0.35 0.35 Oracle 43 0.35 0.35 Oracle 44 0.36 0.35 Oracle 45 0.35 0.35 Oracle 46 0.35 0.35 Oracle 47 0.37 0.34 Oracle 48 0.35 0.34 Oracle 49 0.35 0.35 Oracle 50 0.35 0.35 Total 17 17 Average 0.34 0.34 D.18. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL, TRIMMED TREE365
Size Results of configuration Structural, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 2 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 3 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 4 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 5 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 6 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 7 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 8 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 9 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 10 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 11 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 12 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 13 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 14 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 15 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 16 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 17 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 18 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 19 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 20 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 21 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 22 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 23 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 24 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 25 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 26 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 27 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 28 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 29 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 30 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 31 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 32 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 33 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 34 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 35 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 36 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 37 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 38 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 39 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 40 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 41 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 42 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 43 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 44 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 45 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 46 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 47 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 48 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 49 6 3 3 3.00 3 3 0 1.00 1.00 Oracle 50 6 3 3 3.00 3 3 0 1.00 1.00 Total 300 150 150 150.00 150 150 0 50.00 50.00 Average 6.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Micro avg ····················· 1.00 1.00 366 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Structural, Trimmed Tree
1 2 Oracle 1 3 3 Oracle 2 3 3 Oracle 3 3 3 Oracle 4 3 3 Oracle 5 3 3 Oracle 6 3 3 Oracle 7 3 3 Oracle 8 3 3 Oracle 9 3 3 Oracle 10 3 3 Oracle 11 3 3 Oracle 12 3 3 Oracle 13 3 3 Oracle 14 3 3 Oracle 15 3 3 Oracle 16 3 3 Oracle 17 3 3 Oracle 18 3 3 Oracle 19 3 3 Oracle 20 3 3 Oracle 21 3 3 Oracle 22 3 3 Oracle 23 3 3 Oracle 24 3 3 Oracle 25 3 3 Oracle 26 3 3 Oracle 27 3 3 Oracle 28 3 3 Oracle 29 3 3 Oracle 30 3 3 Oracle 31 3 3 Oracle 32 3 3 Oracle 33 3 3 Oracle 34 3 3 Oracle 35 3 3 Oracle 36 3 3 Oracle 37 3 3 Oracle 38 3 3 Oracle 39 3 3 Oracle 40 3 3 Oracle 41 3 3 Oracle 42 3 3 Oracle 43 3 3 Oracle 44 3 3 Oracle 45 3 3 Oracle 46 3 3 Oracle 47 3 3 Oracle 48 3 3 Oracle 49 3 3 Oracle 50 3 3 Total 150 150 Average 3.00 3.00
(Due to an yet unsolved bug in the chart generation software, the compact and moving average charts for this table are not included. Note that the values are trivial, making the charts not stricly required for interpretation. Probably it is because the values are trivial that the chart generation bug manifests itself.) D.18. COMPLETE RESULTS OF CONFIGURATION STRUCTURAL, TRIMMED TREE367
Cost Results of configuration Structural, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 12.00 6.00 6.00 6.00 6.00 6.00 0.00 1.00 1.00 Oracle 2 26.00 13.00 13.00 13.00 13.00 13.00 0.00 1.00 1.00 Oracle 3 26.00 13.00 13.00 13.00 13.00 13.00 0.00 1.00 1.00 Oracle 4 26.00 13.00 13.00 13.00 13.00 13.00 0.00 1.00 1.00 Oracle 5 18.00 9.00 9.00 9.00 9.00 9.00 0.00 1.00 1.00 Oracle 6 12.00 6.00 6.00 6.00 6.00 6.00 0.00 1.00 1.00 Oracle 7 14.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 8 22.00 11.00 11.00 11.00 11.00 11.00 0.00 1.00 1.00 Oracle 9 22.00 11.00 11.00 11.00 11.00 11.00 0.00 1.00 1.00 Oracle 10 22.00 11.00 11.00 11.00 11.00 11.00 0.00 1.00 1.00 Oracle 11 16.00 8.00 8.00 8.00 8.00 8.00 0.00 1.00 1.00 Oracle 12 20.00 10.00 10.00 10.00 10.00 10.00 0.00 1.00 1.00 Oracle 13 22.00 11.00 11.00 11.00 11.00 11.00 0.00 1.00 1.00 Oracle 14 22.00 11.00 11.00 11.00 11.00 11.00 0.00 1.00 1.00 Oracle 15 30.00 15.00 15.00 15.00 15.00 15.00 0.00 1.00 1.00 Oracle 16 38.00 19.00 19.00 19.00 19.00 19.00 0.00 1.00 1.00 Oracle 17 38.00 19.00 19.00 19.00 19.00 19.00 0.00 1.00 1.00 Oracle 18 38.00 19.00 19.00 19.00 19.00 19.00 0.00 1.00 1.00 Oracle 19 38.00 19.00 19.00 19.00 19.00 19.00 0.00 1.00 1.00 Oracle 20 38.00 19.00 19.00 19.00 19.00 19.00 0.00 1.00 1.00 Oracle 21 26.00 13.00 13.00 13.00 13.00 13.00 0.00 1.00 1.00 Oracle 22 26.00 13.00 13.00 13.00 13.00 13.00 0.00 1.00 1.00 Oracle 23 30.00 15.00 15.00 15.00 15.00 15.00 0.00 1.00 1.00 Oracle 24 28.00 14.00 14.00 14.00 14.00 14.00 0.00 1.00 1.00 Oracle 25 16.00 8.00 8.00 8.00 8.00 8.00 0.00 1.00 1.00 Oracle 26 16.00 8.00 8.00 8.00 8.00 8.00 0.00 1.00 1.00 Oracle 27 16.00 8.00 8.00 8.00 8.00 8.00 0.00 1.00 1.00 Oracle 28 16.00 8.00 8.00 8.00 8.00 8.00 0.00 1.00 1.00 Oracle 29 22.00 11.00 11.00 11.00 11.00 11.00 0.00 1.00 1.00 Oracle 30 18.00 9.00 9.00 9.00 9.00 9.00 0.00 1.00 1.00 Oracle 31 18.00 9.00 9.00 9.00 9.00 9.00 0.00 1.00 1.00 Oracle 32 18.00 9.00 9.00 9.00 9.00 9.00 0.00 1.00 1.00 Oracle 33 38.00 19.00 19.00 19.00 19.00 19.00 0.00 1.00 1.00 Oracle 34 38.00 19.00 19.00 19.00 19.00 19.00 0.00 1.00 1.00 Oracle 35 26.00 13.00 13.00 13.00 13.00 13.00 0.00 1.00 1.00 Oracle 36 26.00 13.00 13.00 13.00 13.00 13.00 0.00 1.00 1.00 Oracle 37 26.00 13.00 13.00 13.00 13.00 13.00 0.00 1.00 1.00 Oracle 38 26.00 13.00 13.00 13.00 13.00 13.00 0.00 1.00 1.00 Oracle 39 22.00 11.00 11.00 11.00 11.00 11.00 0.00 1.00 1.00 Oracle 40 22.00 11.00 11.00 11.00 11.00 11.00 0.00 1.00 1.00 Oracle 41 22.00 11.00 11.00 11.00 11.00 11.00 0.00 1.00 1.00 Oracle 42 16.00 8.00 8.00 8.00 8.00 8.00 0.00 1.00 1.00 Oracle 43 28.00 14.00 14.00 14.00 14.00 14.00 0.00 1.00 1.00 Oracle 44 28.00 14.00 14.00 14.00 14.00 14.00 0.00 1.00 1.00 Oracle 45 28.00 14.00 14.00 14.00 14.00 14.00 0.00 1.00 1.00 Oracle 46 28.00 14.00 14.00 14.00 14.00 14.00 0.00 1.00 1.00 Oracle 47 14.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 48 14.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 49 14.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 50 20.00 10.00 10.00 10.00 10.00 10.00 0.00 1.00 1.00 Total 1186.00 593.00 593.00 593.00 593.00 593.00 0.00 50.00 50.00 Average 23.72 11.86 11.86 11.86 11.86 11.86 0.00 1.00 1.00 Micro avg ····················· 1.00 1.00 368 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Structural, Trimmed Tree
1 2 Oracle 1 6.00 6.00 Oracle 2 13.00 13.00 Oracle 3 13.00 13.00 Oracle 4 13.00 13.00 Oracle 5 9.00 9.00 Oracle 6 6.00 6.00 Oracle 7 7.00 7.00 Oracle 8 11.00 11.00 Oracle 9 11.00 11.00 Oracle 10 11.00 11.00 Oracle 11 8.00 8.00 Oracle 12 10.00 10.00 Oracle 13 11.00 11.00 Oracle 14 11.00 11.00 Oracle 15 15.00 15.00 Oracle 16 19.00 19.00 Oracle 17 19.00 19.00 Oracle 18 19.00 19.00 Oracle 19 19.00 19.00 Oracle 20 19.00 19.00 Oracle 21 13.00 13.00 Oracle 22 13.00 13.00 Oracle 23 15.00 15.00 Oracle 24 14.00 14.00 Oracle 25 8.00 8.00 Oracle 26 8.00 8.00 Oracle 27 8.00 8.00 Oracle 28 8.00 8.00 Oracle 29 11.00 11.00 Oracle 30 9.00 9.00 Oracle 31 9.00 9.00 Oracle 32 9.00 9.00 Oracle 33 19.00 19.00 Oracle 34 19.00 19.00 Oracle 35 13.00 13.00 Oracle 36 13.00 13.00 Oracle 37 13.00 13.00 Oracle 38 13.00 13.00 Oracle 39 11.00 11.00 Oracle 40 11.00 11.00 Oracle 41 11.00 11.00 Oracle 42 8.00 8.00 Oracle 43 14.00 14.00 Oracle 44 14.00 14.00 Oracle 45 14.00 14.00 Oracle 46 14.00 14.00 Oracle 47 7.00 7.00 Oracle 48 7.00 7.00 Oracle 49 7.00 7.00 Oracle 50 10.00 10.00 Total 593 593 Average 11.86 11.86 D.19. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS, TRIMMED TREE369 D.19 Complete results of configuration Shattered Document with Markov Chains, Trimmed Tree
Like Shattered Document with Markov Chains, but with a trimmed document tree.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.41
Size0 = 5.68
Size1 = 5.44 Size gain = 1.05
Cost0 = 5.68
Cost1 = 5.44 Cost gain = 1.05 370 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Markov Chains, Trimmed Tree
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 0 0 0 9 0.00 1.00 Oracle 2 30 6 0 6 0 6 0 1.00 0.00 Oracle 3 40 8 0 0 0 0 8 0.00 1.00 Oracle 4 46 7 0 0 0 0 7 0.00 1.00 Oracle 5 56 8 0 8 0 8 0 1.00 0.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 11 0 11 0 1.00 0.00 Oracle 9 128 7 0 7 0 7 0 1.00 0.00 Oracle 10 130 20 0 20 0 20 0 1.00 0.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 8 0 8 0 1.00 0.00 Oracle 16 265 6 0 6 0 6 0 1.00 0.00 Oracle 17 266 9 0 9 0 9 0 1.00 0.00 Oracle 18 269 11 0 11 0 11 0 1.00 0.00 Oracle 19 290 6 0 6 0 6 0 1.00 0.00 Oracle 20 307 6 0 6 0 6 0 1.00 0.00 Oracle 21 366 10 0 10 0 10 0 1.00 0.00 Oracle 22 410 10 0 0 0 0 10 0.00 1.00 Oracle 23 455 15 0 15 0 15 0 1.00 0.00 Oracle 24 508 11 0 11 0 11 0 1.00 0.00 Oracle 25 512 9 0 9 0 9 0 1.00 0.00 Oracle 26 513 15 0 15 0 15 0 1.00 0.00 Oracle 27 516 13 0 13 0 13 0 1.00 0.00 Oracle 28 521 7 0 7 0 7 0 1.00 0.00 Oracle 29 542 5 0 5 0 5 0 1.00 0.00 Oracle 30 554 13 0 0 0 0 13 0.00 1.00 Oracle 31 601 11 0 11 0 11 0 1.00 0.00 Oracle 32 606 14 0 0 0 0 14 0.00 1.00 Oracle 33 616 12 0 0 0 0 12 0.00 1.00 Oracle 34 649 8 0 8 0 8 0 1.00 0.00 Oracle 35 723 10 0 10 0 10 0 1.00 0.00 Oracle 36 725 13 0 13 0 13 0 1.00 0.00 Oracle 37 738 11 0 11 0 11 0 1.00 0.00 Oracle 38 741 11 0 11 0 11 0 1.00 0.00 Oracle 39 786 7 0 0 0 0 7 0.00 1.00 Oracle 40 840 14 0 14 0 14 0 1.00 0.00 Oracle 41 846 8 0 0 0 0 8 0.00 1.00 Oracle 42 883 1 0 1 0 1 0 1.00 0.00 Oracle 43 890 10 0 0 0 0 10 0.00 1.00 Oracle 44 897 6 0 6 0 6 0 1.00 0.00 Oracle 45 899 9 0 9 0 9 0 1.00 0.00 Oracle 46 929 10 0 10 0 10 0 1.00 0.00 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 0 0 0 0 13 0.00 1.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 9 0 9 0 1.00 0.00 Total ··· 500 0 296 0 296 204 31.00 19.00 Average ··· 10.00 0.00 5.92 0.00 5.92 4.08 0.62 0.38 Micro avg ····················· 0.59 0.41 D.19. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS, TRIMMED TREE371
Duration Results of configuration Shattered Document with Markov Chains, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 4.93 0.54 0.57 0.55 0.55 0.54 0.01 0.98 1.02 Oracle 2 8.22 1.36 1.38 1.37 1.36 1.38 -0.01 1.01 0.99 Oracle 3 4.62 0.54 0.70 0.58 0.68 0.54 0.14 0.80 1.26 Oracle 4 4.04 0.54 0.67 0.58 0.67 0.54 0.13 0.80 1.25 Oracle 5 10.97 1.37 1.38 1.38 1.37 1.38 -0.02 1.01 0.99 Oracle 6 5.98 0.54 0.56 0.55 0.55 0.54 0.01 0.98 1.02 Oracle 7 6.11 0.54 0.67 0.55 0.67 0.55 0.13 0.81 1.23 Oracle 8 15.66 1.41 1.44 1.42 1.44 1.42 0.02 0.99 1.01 Oracle 9 10.00 1.41 1.48 1.43 1.44 1.41 0.02 0.98 1.02 Oracle 10 28.53 1.41 1.51 1.43 1.43 1.43 0.01 1.00 1.00 Oracle 11 6.66 0.54 0.67 0.55 0.67 0.55 0.13 0.81 1.23 Oracle 12 7.18 0.54 0.67 0.55 0.67 0.55 0.13 0.81 1.23 Oracle 13 5.55 0.32 0.47 0.37 0.47 0.33 0.14 0.70 1.44 Oracle 14 6.02 0.54 0.57 0.55 0.55 0.57 -0.02 1.04 0.97 Oracle 15 11.34 1.41 1.42 1.41 1.42 1.42 0.00 1.00 1.00 Oracle 16 8.26 1.36 1.41 1.38 1.41 1.36 0.05 0.97 1.03 Oracle 17 12.62 1.36 1.64 1.40 1.64 1.38 0.27 0.84 1.20 Oracle 18 15.13 1.37 1.41 1.38 1.38 1.38 0.01 0.99 1.01 Oracle 19 8.24 1.37 1.38 1.38 1.38 1.38 0.01 0.99 1.01 Oracle 20 8.29 1.36 1.42 1.38 1.39 1.38 0.02 0.98 1.02 Oracle 21 14.23 1.41 1.45 1.42 1.42 1.45 -0.04 1.02 0.98 Oracle 22 5.56 0.54 0.68 0.55 0.68 0.54 0.13 0.80 1.25 Oracle 23 20.56 1.36 1.38 1.37 1.38 1.37 0.02 0.99 1.01 Oracle 24 15.09 1.37 1.38 1.38 1.38 1.38 0.00 1.00 1.00 Oracle 25 12.28 1.36 1.38 1.37 1.37 1.38 -0.01 1.01 0.99 Oracle 26 21.42 1.41 1.48 1.43 1.43 1.42 0.01 0.99 1.01 Oracle 27 18.50 1.41 1.45 1.42 1.45 1.42 0.02 0.98 1.02 Oracle 28 9.98 1.41 1.43 1.42 1.43 1.42 0.02 0.99 1.01 Oracle 29 6.87 1.37 1.39 1.38 1.39 1.37 0.02 0.98 1.02 Oracle 30 7.16 0.54 0.66 0.55 0.66 0.54 0.13 0.81 1.23 Oracle 31 15.13 1.36 1.44 1.38 1.44 1.37 0.07 0.95 1.05 Oracle 32 7.63 0.54 0.56 0.55 0.55 0.55 0.00 1.00 1.00 Oracle 33 4.88 0.32 0.49 0.41 0.45 0.33 0.12 0.74 1.35 Oracle 34 10.96 1.37 1.38 1.37 1.37 1.38 -0.01 1.01 0.99 Oracle 35 13.77 1.36 1.48 1.38 1.37 1.36 0.02 0.99 1.01 Oracle 36 18.57 1.41 1.46 1.43 1.44 1.46 -0.02 1.02 0.98 Oracle 37 15.65 1.41 1.43 1.42 1.43 1.41 0.02 0.99 1.02 Oracle 38 15.66 1.41 1.44 1.42 1.43 1.42 0.02 0.99 1.01 Oracle 39 3.79 0.54 0.55 0.54 0.55 0.54 0.01 0.98 1.02 Oracle 40 19.21 1.36 1.40 1.38 1.38 1.38 0.00 1.00 1.00 Oracle 41 4.46 0.54 0.67 0.55 0.67 0.54 0.13 0.80 1.24 Oracle 42 1.42 1.42 1.42 1.42 1.42 1.42 0.00 1.00 1.00 Oracle 43 5.61 0.54 0.73 0.56 0.73 0.54 0.19 0.74 1.34 Oracle 44 8.24 1.37 1.38 1.38 1.38 1.38 0.02 0.99 1.01 Oracle 45 12.36 1.36 1.39 1.38 1.39 1.38 0.01 0.99 1.01 Oracle 46 13.70 1.36 1.38 1.37 1.38 1.37 0.01 0.99 1.01 Oracle 47 5.56 0.54 0.67 0.55 0.67 0.54 0.13 0.80 1.25 Oracle 48 7.20 0.54 0.67 0.55 0.67 0.54 0.13 0.80 1.25 Oracle 49 5.45 0.54 0.55 0.55 0.54 0.55 -0.02 1.02 0.98 Oracle 50 12.33 1.36 1.38 1.37 1.38 1.37 0.01 1.00 1.00 Total 521.56 52.63 55.98 53.41 55.35 53.07 2.28 46.89 53.94 Average 10.43 1.05 1.12 1.07 1.11 1.06 0.05 0.94 1.08 Micro avg ····················· 0.96 1.04 372 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Markov Chains, Trimmed Tree
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 0.55 0.54 0.55 0.54 0.57 0.55 0.55 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 2 1.36 1.38 1.37 1.38 1.37 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 3 0.68 0.70 0.54 0.54 0.55 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 4 0.67 0.66 0.54 0.54 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 5 1.37 1.37 1.37 1.37 1.38 1.37 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 6 0.55 0.54 0.54 0.55 0.54 0.55 0.54 0.54 0.54 0.56 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 7 0.67 0.55 0.55 0.55 0.54 0.54 0.55 0.54 0.54 0.54 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 8 1.44 1.41 1.42 1.43 1.43 1.42 1.42 1.42 1.41 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 Oracle 9 1.44 1.43 1.41 1.48 1.42 1.42 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 Oracle 10 1.43 1.51 1.44 1.43 1.41 1.42 1.42 1.41 1.43 1.42 1.42 1.41 1.42 1.41 1.42 1.42 1.43 1.41 1.42 1.43 Oracle 11 0.67 0.55 0.55 0.55 0.55 0.54 0.54 0.55 0.55 0.54 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 12 0.67 0.55 0.55 0.54 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 13 0.47 0.34 0.33 0.33 0.32 0.33 0.32 0.32 0.45 0.45 0.45 0.47 0.33 0.32 0.33 0.33 0.33 0.33 0.33 0.33 Oracle 14 0.55 0.55 0.55 0.54 0.55 0.54 0.54 0.54 0.55 0.55 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 15 1.42 1.41 1.41 1.41 1.41 1.42 1.41 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 Oracle 16 1.41 1.36 1.38 1.38 1.38 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 Oracle 17 1.64 1.38 1.38 1.38 1.36 1.36 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 18 1.38 1.38 1.38 1.37 1.37 1.38 1.37 1.37 1.38 1.41 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 19 1.38 1.38 1.38 1.37 1.37 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 20 1.39 1.42 1.36 1.37 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 21 1.42 1.42 1.41 1.42 1.42 1.41 1.42 1.43 1.41 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 Oracle 22 0.68 0.55 0.55 0.55 0.55 0.54 0.54 0.54 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 23 1.38 1.38 1.37 1.38 1.37 1.37 1.36 1.37 1.38 1.37 1.37 1.38 1.37 1.38 1.37 1.37 1.37 1.37 1.37 1.37 Oracle 24 1.38 1.37 1.37 1.37 1.37 1.37 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 25 1.37 1.38 1.36 1.37 1.36 1.36 1.36 1.36 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 26 1.43 1.42 1.43 1.42 1.42 1.48 1.41 1.41 1.43 1.43 1.45 1.41 1.43 1.41 1.42 1.42 1.42 1.42 1.42 1.42 Oracle 27 1.45 1.43 1.42 1.42 1.42 1.42 1.42 1.42 1.41 1.41 1.41 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 Oracle 28 1.43 1.43 1.43 1.43 1.41 1.41 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 Oracle 29 1.39 1.37 1.37 1.38 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 Oracle 30 0.66 0.54 0.55 0.54 0.55 0.54 0.54 0.54 0.54 0.55 0.54 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 31 1.44 1.38 1.38 1.37 1.37 1.37 1.37 1.38 1.37 1.36 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 Oracle 32 0.55 0.56 0.55 0.54 0.55 0.55 0.54 0.55 0.54 0.55 0.54 0.55 0.54 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 33 0.45 0.33 0.46 0.45 0.49 0.48 0.46 0.46 0.33 0.32 0.33 0.33 0.33 0.33 0.33 0.33 0.33 0.33 0.33 0.33 Oracle 34 1.37 1.37 1.37 1.38 1.37 1.37 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 35 1.37 1.38 1.37 1.37 1.37 1.36 1.36 1.36 1.48 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 1.36 Oracle 36 1.44 1.41 1.42 1.42 1.42 1.42 1.42 1.43 1.43 1.42 1.42 1.45 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 37 1.43 1.43 1.42 1.42 1.41 1.43 1.42 1.43 1.42 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41 Oracle 38 1.43 1.44 1.44 1.43 1.42 1.42 1.42 1.41 1.41 1.41 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 Oracle 39 0.55 0.54 0.54 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 40 1.38 1.38 1.37 1.36 1.36 1.37 1.38 1.37 1.37 1.40 1.38 1.36 1.37 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 41 0.67 0.54 0.55 0.54 0.54 0.54 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 1.42 Oracle 43 0.73 0.55 0.54 0.54 0.54 0.55 0.55 0.54 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 44 1.38 1.38 1.37 1.37 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 45 1.39 1.37 1.36 1.37 1.38 1.38 1.38 1.37 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 1.38 Oracle 46 1.38 1.37 1.37 1.37 1.36 1.38 1.38 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 Oracle 47 0.67 0.55 0.55 0.54 0.55 0.54 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 48 0.67 0.55 0.54 0.55 0.55 0.55 0.54 0.55 0.55 0.55 0.55 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 0.54 Oracle 49 0.54 0.55 0.55 0.55 0.54 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 0.55 Oracle 50 1.38 1.37 1.38 1.37 1.38 1.37 1.36 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 1.37 Total 55 54 53 53 53 53 53 53 53 53 53 53 53 53 53 53 53 53 53 53 Average 1.11 1.07 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 1.06 D.19. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS, TRIMMED TREE373
Size Results of configuration Shattered Document with Markov Chains, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 27 3 3 3.00 3 3 0 1.00 1.00 Oracle 2 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 3 26 3 4 3.25 4 3 1 0.75 1.34 Oracle 4 23 3 4 3.29 4 3 1 0.75 1.34 Oracle 5 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 6 33 3 3 3.00 3 3 0 1.00 1.00 Oracle 7 34 3 4 3.09 4 3 1 0.75 1.34 Oracle 8 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 9 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 10 140 7 7 7.00 7 7 0 1.00 1.00 Oracle 11 37 3 4 3.09 4 3 1 0.75 1.34 Oracle 12 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 13 35 2 3 2.34 3 2 1 0.66 1.50 Oracle 14 33 3 3 3.00 3 3 0 1.00 1.00 Oracle 15 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 16 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 17 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 18 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 19 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 20 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 21 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 22 31 3 4 3.10 4 3 1 0.75 1.34 Oracle 23 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 24 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 25 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 26 105 7 7 7.00 7 7 0 1.00 1.00 Oracle 27 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 28 49 7 7 7.00 7 7 0 1.00 1.00 Oracle 29 35 7 7 7.00 7 7 0 1.00 1.00 Oracle 30 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 31 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 32 42 3 3 3.00 3 3 0 1.00 1.00 Oracle 33 31 2 3 2.59 3 2 1 0.66 1.50 Oracle 34 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 35 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 36 91 7 7 7.00 7 7 0 1.00 1.00 Oracle 37 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 38 77 7 7 7.00 7 7 0 1.00 1.00 Oracle 39 21 3 3 3.00 3 3 0 1.00 1.00 Oracle 40 98 7 7 7.00 7 7 0 1.00 1.00 Oracle 41 25 3 4 3.13 4 3 1 0.75 1.34 Oracle 42 7 7 7 7.00 7 7 0 1.00 1.00 Oracle 43 30 3 3 3.00 3 3 0 1.00 1.00 Oracle 44 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 45 63 7 7 7.00 7 7 0 1.00 1.00 Oracle 46 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 47 31 3 4 3.10 4 3 1 0.75 1.34 Oracle 48 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 49 30 3 3 3.00 3 3 0 1.00 1.00 Oracle 50 63 7 7 7.00 7 7 0 1.00 1.00 Total 2681 272 284 274.18 284 272 12 46.84 54.34 Average 53.62 5.44 5.68 5.48 5.68 5.44 0.24 0.94 1.09 Micro avg ····················· 0.96 1.05 374 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Markov Chains, Trimmed Tree
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 2 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 3 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 5 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 8 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 9 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 10 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 11 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 12 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 13 3 2 2 2 2 2 2 2 3 3 3 3 2 2 2 2 2 2 2 2 Oracle 14 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 15 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 16 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 17 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 18 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 19 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 20 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 21 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 22 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 23 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 24 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 25 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 26 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 27 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 28 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 29 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 30 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 31 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 32 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 33 3 2 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 34 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 35 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 36 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 37 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 38 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 39 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 40 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 41 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 42 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 43 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 44 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 45 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 46 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 47 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 48 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 49 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 50 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Total 284 274 273 273 273 273 273 273 273 273 273 273 272 272 272 272 272 272 272 272 Average 5.68 5.48 5.46 5.46 5.46 5.46 5.46 5.46 5.46 5.46 5.46 5.46 5.44 5.44 5.44 5.44 5.44 5.44 5.44 5.44 D.19. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS, TRIMMED TREE375
Cost Results of configuration Shattered Document with Markov Chains, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 27.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 2 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 3 26.00 3.00 4.00 3.25 4.00 3.00 1.00 0.75 1.34 Oracle 4 23.00 3.00 4.00 3.29 4.00 3.00 1.00 0.75 1.34 Oracle 5 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 6 33.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 7 34.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 8 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 9 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 10 140.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 11 37.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 12 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 13 35.00 2.00 3.00 2.34 3.00 2.00 1.00 0.66 1.50 Oracle 14 33.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 15 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 16 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 17 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 18 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 19 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 20 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 21 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 22 31.00 3.00 4.00 3.10 4.00 3.00 1.00 0.75 1.34 Oracle 23 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 24 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 25 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 26 105.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 27 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 28 49.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 29 35.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 30 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 31 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 32 42.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 33 31.00 2.00 3.00 2.59 3.00 2.00 1.00 0.66 1.50 Oracle 34 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 35 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 36 91.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 37 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 38 77.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 39 21.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 40 98.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 41 25.00 3.00 4.00 3.13 4.00 3.00 1.00 0.75 1.34 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 43 30.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 44 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 45 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 46 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 47 31.00 3.00 4.00 3.10 4.00 3.00 1.00 0.75 1.34 Oracle 48 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 49 30.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 50 63.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Total 2681.00 272.00 284.00 274.18 284.00 272.00 12.00 46.84 54.34 Average 53.62 5.44 5.68 5.48 5.68 5.44 0.24 0.94 1.09 Micro avg ····················· 0.96 1.05 376 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Markov Chains, Trimmed Tree
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 2 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 3 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 4 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 5 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 6 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 7 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 8 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 9 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 10 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 11 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 12 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 13 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 14 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 15 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 16 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 17 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 18 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 19 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 20 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 21 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 22 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 23 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 24 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 25 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 26 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 27 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 28 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 29 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 30 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 31 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 32 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 33 3.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 34 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 35 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 36 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 37 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 38 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 39 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 40 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 41 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 42 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 43 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 44 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 45 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 46 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 47 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 48 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 49 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 50 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Total 284 274 273 273 273 273 273 273 273 273 273 273 272 272 272 272 272 272 272 272 Average 5.68 5.48 5.46 5.46 5.46 5.46 5.46 5.46 5.46 5.46 5.46 5.46 5.44 5.44 5.44 5.44 5.44 5.44 5.44 5.44 D.20. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS WITH HEURISTICS, TRIMMED TREE377 D.20 Complete results of configuration Shattered Document with Markov Chains with heuris- tics, Trimmed Tree
Like Shattered Document with Markov Chains with heuristics, but with a trimmed document tree.
Configuration parameters:
Quantity of atoms = 1000 Quantity of oracles = 50 Quantity of sessions = 500 Page size (number of items) = 10 Quantity of expanded items = 3 Update travel matrix strategy = Central to central Maximum duration of session = 30.00 Maximum session size = 6 Maximum session cost = 60.00 Weight of Next links = 1.00 Weight of heading-to-paragraph Child links = 1.00 Weight of heading-to-heading Child links = 1.00 Residual weight (all links) = 0.00
Principal results: Cost evolution:
Success Rate = 0.95
Size0 = 4.72
Size1 = 3.44 Size gain = 1.38
Cost0 = 4.72
Cost1 = 3.44 Cost gain = 1.38 378 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Outcomes of configuration Shattered Document with Markov Chains with heuristics, Trimmed Tree
Node id Total Timeout Sizeout Costout Failed Successful Fail rate Success rate Oracle 1 9 9 0 0 0 0 9 0.00 1.00 Oracle 2 30 6 0 0 0 0 6 0.00 1.00 Oracle 3 40 8 0 0 0 0 8 0.00 1.00 Oracle 4 46 7 0 0 0 0 7 0.00 1.00 Oracle 5 56 8 0 0 0 0 8 0.00 1.00 Oracle 6 85 11 0 0 0 0 11 0.00 1.00 Oracle 7 108 11 0 0 0 0 11 0.00 1.00 Oracle 8 125 11 0 0 0 0 11 0.00 1.00 Oracle 9 128 7 0 0 0 0 7 0.00 1.00 Oracle 10 130 20 0 0 0 0 20 0.00 1.00 Oracle 11 138 12 0 0 0 0 12 0.00 1.00 Oracle 12 157 13 0 0 0 0 13 0.00 1.00 Oracle 13 171 15 0 0 0 0 15 0.00 1.00 Oracle 14 173 11 0 0 0 0 11 0.00 1.00 Oracle 15 241 8 0 0 0 0 8 0.00 1.00 Oracle 16 265 6 0 0 0 0 6 0.00 1.00 Oracle 17 266 9 0 0 0 0 9 0.00 1.00 Oracle 18 269 11 0 0 0 0 11 0.00 1.00 Oracle 19 290 6 0 4 0 4 2 0.66 0.34 Oracle 20 307 6 0 0 0 0 6 0.00 1.00 Oracle 21 366 10 0 0 0 0 10 0.00 1.00 Oracle 22 410 10 0 0 0 0 10 0.00 1.00 Oracle 23 455 15 0 0 0 0 15 0.00 1.00 Oracle 24 508 11 0 0 0 0 11 0.00 1.00 Oracle 25 512 9 0 0 0 0 9 0.00 1.00 Oracle 26 513 15 0 0 0 0 15 0.00 1.00 Oracle 27 516 13 0 0 0 0 13 0.00 1.00 Oracle 28 521 7 0 0 0 0 7 0.00 1.00 Oracle 29 542 5 0 0 0 0 5 0.00 1.00 Oracle 30 554 13 0 0 0 0 13 0.00 1.00 Oracle 31 601 11 0 0 0 0 11 0.00 1.00 Oracle 32 606 14 0 0 0 0 14 0.00 1.00 Oracle 33 616 12 0 0 0 0 12 0.00 1.00 Oracle 34 649 8 0 8 0 8 0 1.00 0.00 Oracle 35 723 10 0 0 0 0 10 0.00 1.00 Oracle 36 725 13 0 0 0 0 13 0.00 1.00 Oracle 37 738 11 0 0 0 0 11 0.00 1.00 Oracle 38 741 11 0 1 0 1 10 0.09 0.91 Oracle 39 786 7 0 0 0 0 7 0.00 1.00 Oracle 40 840 14 0 0 0 0 14 0.00 1.00 Oracle 41 846 8 0 0 0 0 8 0.00 1.00 Oracle 42 883 1 0 0 0 0 1 0.00 1.00 Oracle 43 890 10 0 0 0 0 10 0.00 1.00 Oracle 44 897 6 0 0 0 0 6 0.00 1.00 Oracle 45 899 9 0 0 0 0 9 0.00 1.00 Oracle 46 929 10 0 10 0 10 0 1.00 0.00 Oracle 47 936 10 0 0 0 0 10 0.00 1.00 Oracle 48 939 13 0 0 0 0 13 0.00 1.00 Oracle 49 944 10 0 0 0 0 10 0.00 1.00 Oracle 50 999 9 0 0 0 0 9 0.00 1.00 Total ··· 500 0 23 0 23 477 2.76 47.24 Average ··· 10.00 0.00 0.46 0.00 0.46 9.54 0.05 0.95 Micro avg ····················· 0.05 0.95 D.20. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS WITH HEURISTICS, TRIMMED TREE379
Duration Results of configuration Shattered Document with Markov Chains with heuristics, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 5.07 0.55 0.57 0.56 0.56 0.57 -0.01 1.02 0.98 Oracle 2 4.63 0.57 1.35 0.77 1.35 0.58 0.77 0.43 2.34 Oracle 3 5.10 0.56 0.71 0.64 0.70 0.71 -0.02 1.02 0.98 Oracle 4 4.55 0.57 0.72 0.65 0.70 0.57 0.13 0.82 1.22 Oracle 5 6.55 0.78 1.02 0.82 1.02 0.80 0.22 0.79 1.27 Oracle 6 6.20 0.56 0.57 0.56 0.56 0.57 -0.01 1.01 0.99 Oracle 7 6.42 0.56 0.69 0.59 0.69 0.57 0.13 0.82 1.22 Oracle 8 7.56 0.57 1.13 0.69 0.79 0.58 0.21 0.73 1.37 Oracle 9 5.02 0.56 1.14 0.72 1.14 0.59 0.55 0.52 1.95 Oracle 10 12.19 0.56 0.91 0.61 0.91 0.57 0.34 0.63 1.60 Oracle 11 6.99 0.56 0.70 0.59 0.70 0.59 0.12 0.84 1.20 Oracle 12 7.56 0.56 0.70 0.58 0.70 0.57 0.13 0.82 1.22 Oracle 13 6.90 0.34 0.48 0.46 0.48 0.48 0.00 0.99 1.01 Oracle 14 6.29 0.56 0.58 0.57 0.57 0.58 -0.02 1.02 0.98 Oracle 15 4.82 0.57 0.79 0.60 0.79 0.57 0.22 0.73 1.38 Oracle 16 8.71 1.44 1.47 1.45 1.45 1.46 -0.02 1.01 0.99 Oracle 17 6.38 0.57 1.41 0.71 1.41 0.59 0.83 0.41 2.42 Oracle 18 7.05 0.57 0.93 0.64 0.93 0.58 0.35 0.63 1.60 Oracle 19 7.43 0.79 1.48 1.24 1.48 1.47 0.01 0.99 1.01 Oracle 20 4.43 0.58 0.93 0.73 0.78 0.71 0.07 0.91 1.09 Oracle 21 6.74 0.57 1.34 0.67 1.34 0.58 0.77 0.43 2.33 Oracle 22 5.85 0.56 0.70 0.59 0.70 0.58 0.12 0.84 1.20 Oracle 23 9.41 0.56 1.35 0.63 1.35 0.57 0.78 0.42 2.36 Oracle 24 7.34 0.57 1.34 0.66 1.34 0.58 0.77 0.43 2.33 Oracle 25 3.09 0.34 0.35 0.34 0.34 0.34 0.00 1.01 0.99 Oracle 26 9.13 0.55 0.71 0.61 0.70 0.57 0.13 0.82 1.23 Oracle 27 8.32 0.56 0.92 0.64 0.92 0.58 0.34 0.63 1.59 Oracle 28 4.52 0.56 0.71 0.65 0.70 0.58 0.12 0.83 1.20 Oracle 29 3.20 0.57 0.91 0.64 0.91 0.57 0.34 0.63 1.61 Oracle 30 7.59 0.57 0.70 0.59 0.70 0.58 0.13 0.82 1.22 Oracle 31 6.66 0.56 0.92 0.60 0.92 0.58 0.35 0.63 1.61 Oracle 32 8.30 0.57 0.71 0.59 0.71 0.57 0.14 0.80 1.24 Oracle 33 5.79 0.48 0.49 0.48 0.48 0.48 -0.01 1.02 0.98 Oracle 34 11.63 1.44 1.48 1.45 1.45 1.45 0.00 1.00 1.00 Oracle 35 3.45 0.34 0.35 0.34 0.34 0.34 0.00 1.01 0.99 Oracle 36 8.36 0.55 0.91 0.64 0.91 0.58 0.34 0.63 1.57 Oracle 37 7.42 0.56 1.13 0.67 1.13 0.58 0.55 0.51 1.97 Oracle 38 8.59 0.56 1.45 0.78 1.45 0.57 0.88 0.39 2.55 Oracle 39 4.05 0.57 0.60 0.58 0.57 0.57 0.00 1.01 0.99 Oracle 40 8.98 0.57 0.91 0.64 0.78 0.59 0.20 0.74 1.34 Oracle 41 4.76 0.57 0.70 0.59 0.70 0.58 0.12 0.83 1.20 Oracle 42 0.91 0.91 0.91 0.91 0.91 0.91 0.00 1.00 1.00 Oracle 43 5.74 0.56 0.63 0.57 0.63 0.58 0.05 0.93 1.08 Oracle 44 7.40 1.21 1.26 1.23 1.23 1.23 -0.01 1.01 0.99 Oracle 45 5.93 0.56 1.35 0.66 1.35 0.57 0.78 0.42 2.37 Oracle 46 14.55 1.43 1.50 1.45 1.50 1.46 0.05 0.97 1.03 Oracle 47 5.84 0.56 0.70 0.59 0.70 0.56 0.13 0.81 1.23 Oracle 48 7.52 0.55 0.70 0.58 0.70 0.57 0.12 0.83 1.21 Oracle 49 5.71 0.56 0.58 0.57 0.57 0.56 0.01 0.98 1.02 Oracle 50 5.45 0.56 0.90 0.61 0.90 0.56 0.34 0.63 1.59 Total 332.09 31.53 45.48 34.77 44.62 33.11 11.51 39.12 69.83 Average 6.64 0.63 0.91 0.70 0.89 0.66 0.23 0.78 1.40 Micro avg ····················· 0.74 1.34 380 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Duration Evolution of configuration Shattered Document with Markov Chains with heuristics, Trimmed Tree
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 0.56 0.55 0.56 0.57 0.55 0.55 0.57 0.56 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 2 1.35 0.70 0.70 0.57 0.72 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 3 0.70 0.70 0.56 0.57 0.58 0.71 0.58 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 Oracle 4 0.70 0.70 0.58 0.58 0.72 0.71 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 5 1.02 0.80 0.80 0.79 0.80 0.78 0.78 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 0.80 Oracle 6 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.57 0.56 0.56 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 7 0.69 0.58 0.57 0.57 0.57 0.58 0.59 0.57 0.56 0.58 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 8 0.79 1.13 0.77 0.71 0.58 0.58 0.58 0.58 0.57 0.71 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 9 1.14 0.70 0.62 0.56 0.70 0.72 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 Oracle 10 0.91 0.57 0.57 0.73 0.57 0.56 0.56 0.56 0.58 0.58 0.57 0.57 0.57 0.72 0.58 0.57 0.57 0.70 0.58 0.57 Oracle 11 0.70 0.56 0.56 0.58 0.57 0.57 0.56 0.59 0.58 0.57 0.57 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 Oracle 12 0.70 0.57 0.56 0.59 0.57 0.57 0.57 0.57 0.57 0.56 0.57 0.58 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 13 0.48 0.34 0.34 0.48 0.48 0.47 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 Oracle 14 0.57 0.56 0.56 0.57 0.57 0.58 0.58 0.57 0.57 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 15 0.79 0.58 0.58 0.58 0.58 0.58 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 16 1.45 1.45 1.46 1.44 1.47 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 17 1.41 0.57 0.70 0.57 0.58 0.70 0.57 0.70 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 0.59 Oracle 18 0.93 0.57 0.71 0.70 0.57 0.57 0.58 0.70 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 19 1.48 0.80 0.79 1.47 1.44 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 1.47 Oracle 20 0.78 0.93 0.71 0.58 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 0.71 Oracle 21 1.34 0.80 0.59 0.57 0.58 0.58 0.57 0.57 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 22 0.70 0.56 0.56 0.57 0.59 0.58 0.57 0.57 0.57 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 23 1.35 0.56 0.56 0.58 0.58 0.58 0.57 0.57 0.58 0.58 0.58 0.60 0.59 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 24 1.34 0.80 0.59 0.57 0.58 0.58 0.58 0.58 0.59 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 25 0.34 0.34 0.34 0.34 0.35 0.34 0.35 0.35 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 Oracle 26 0.70 0.70 0.55 0.56 0.70 0.58 0.71 0.59 0.59 0.59 0.57 0.58 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 27 0.92 0.70 0.56 0.70 0.57 0.57 0.71 0.71 0.57 0.59 0.57 0.57 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 28 0.70 0.56 0.57 0.70 0.71 0.71 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 29 0.91 0.58 0.58 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 30 0.70 0.58 0.58 0.57 0.57 0.58 0.58 0.58 0.57 0.57 0.58 0.57 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 31 0.92 0.56 0.56 0.57 0.59 0.58 0.57 0.58 0.57 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 32 0.71 0.57 0.57 0.57 0.64 0.58 0.61 0.58 0.57 0.57 0.57 0.60 0.59 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 33 0.48 0.48 0.48 0.49 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 0.48 Oracle 34 1.45 1.48 1.46 1.44 1.47 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 1.45 Oracle 35 0.34 0.34 0.34 0.35 0.35 0.34 0.35 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 Oracle 36 0.91 0.70 0.55 0.73 0.59 0.56 0.58 0.59 0.70 0.71 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 37 1.13 0.70 0.71 0.71 0.58 0.56 0.58 0.58 0.59 0.70 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 38 1.45 1.35 0.56 0.70 0.58 0.74 0.70 0.57 0.72 0.65 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 39 0.57 0.57 0.58 0.58 0.59 0.60 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 40 0.78 0.79 0.91 0.71 0.59 0.59 0.58 0.57 0.58 0.57 0.58 0.57 0.58 0.59 0.59 0.59 0.59 0.59 0.59 0.59 Oracle 41 0.70 0.61 0.58 0.57 0.57 0.58 0.57 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 42 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 Oracle 43 0.63 0.56 0.56 0.57 0.57 0.56 0.56 0.56 0.57 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 0.58 Oracle 44 1.23 1.21 1.26 1.24 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 1.23 Oracle 45 1.35 0.57 0.57 0.58 0.56 0.58 0.58 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 46 1.50 1.49 1.43 1.44 1.44 1.45 1.44 1.44 1.45 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 1.46 Oracle 47 0.70 0.56 0.60 0.56 0.59 0.56 0.57 0.57 0.57 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 48 0.70 0.55 0.56 0.57 0.56 0.57 0.56 0.57 0.58 0.57 0.58 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 0.57 Oracle 49 0.57 0.58 0.57 0.57 0.57 0.56 0.58 0.57 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Oracle 50 0.90 0.57 0.57 0.56 0.57 0.57 0.57 0.58 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 0.56 Total 45 35 33 34 34 34 33 33 33 34 33 33 33 33 33 33 33 33 33 33 Average 0.89 0.70 0.66 0.68 0.67 0.67 0.66 0.67 0.66 0.67 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 D.20. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS WITH HEURISTICS, TRIMMED TREE381
Size Results of configuration Shattered Document with Markov Chains with heuristics, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 27 3 3 3.00 3 3 0 1.00 1.00 Oracle 2 25 3 7 4.16 7 3 4 0.43 2.34 Oracle 3 28 3 4 3.50 4 4 0 1.00 1.00 Oracle 4 25 3 4 3.57 4 3 1 0.75 1.34 Oracle 5 33 4 5 4.13 5 4 1 0.80 1.25 Oracle 6 33 3 3 3.00 3 3 0 1.00 1.00 Oracle 7 34 3 4 3.09 4 3 1 0.75 1.34 Oracle 8 40 3 6 3.63 4 3 1 0.75 1.34 Oracle 9 27 3 6 3.86 6 3 3 0.50 2.00 Oracle 10 65 3 5 3.25 5 3 2 0.60 1.66 Oracle 11 37 3 4 3.09 4 3 1 0.75 1.34 Oracle 12 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 13 43 2 3 2.87 3 3 0 1.00 1.00 Oracle 14 33 3 3 3.00 3 3 0 1.00 1.00 Oracle 15 25 3 4 3.13 4 3 1 0.75 1.34 Oracle 16 42 7 7 7.00 7 7 0 1.00 1.00 Oracle 17 34 3 7 3.78 7 3 4 0.43 2.34 Oracle 18 38 3 5 3.45 5 3 2 0.60 1.66 Oracle 19 36 4 7 6.00 7 7 0 1.00 1.00 Oracle 20 24 3 5 4.00 4 4 0 1.00 1.00 Oracle 21 35 3 7 3.50 7 3 4 0.43 2.34 Oracle 22 31 3 4 3.10 4 3 1 0.75 1.34 Oracle 23 49 3 7 3.27 7 3 4 0.43 2.34 Oracle 24 38 3 7 3.45 7 3 4 0.43 2.34 Oracle 25 18 2 2 2.00 2 2 0 1.00 1.00 Oracle 26 49 3 4 3.27 4 3 1 0.75 1.34 Oracle 27 45 3 5 3.46 5 3 2 0.60 1.66 Oracle 28 25 3 4 3.57 4 3 1 0.75 1.34 Oracle 29 17 3 5 3.40 5 3 2 0.60 1.66 Oracle 30 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 31 35 3 5 3.18 5 3 2 0.60 1.66 Oracle 32 43 3 4 3.07 4 3 1 0.75 1.34 Oracle 33 36 3 3 3.00 3 3 0 1.00 1.00 Oracle 34 56 7 7 7.00 7 7 0 1.00 1.00 Oracle 35 20 2 2 2.00 2 2 0 1.00 1.00 Oracle 36 45 3 5 3.46 5 3 2 0.60 1.66 Oracle 37 40 3 6 3.63 6 3 3 0.50 2.00 Oracle 38 45 3 7 4.09 7 3 4 0.43 2.34 Oracle 39 21 3 3 3.00 3 3 0 1.00 1.00 Oracle 40 47 3 5 3.36 4 3 1 0.75 1.34 Oracle 41 25 3 4 3.13 4 3 1 0.75 1.34 Oracle 42 5 5 5 5.00 5 5 0 1.00 1.00 Oracle 43 30 3 3 3.00 3 3 0 1.00 1.00 Oracle 44 36 6 6 6.00 6 6 0 1.00 1.00 Oracle 45 31 3 7 3.45 7 3 4 0.43 2.34 Oracle 46 70 7 7 7.00 7 7 0 1.00 1.00 Oracle 47 31 3 4 3.10 4 3 1 0.75 1.34 Oracle 48 40 3 4 3.08 4 3 1 0.75 1.34 Oracle 49 30 3 3 3.00 3 3 0 1.00 1.00 Oracle 50 29 3 5 3.22 5 3 2 0.60 1.66 Total 1751 166 240 182.45 236 172 64 38.25 71.25 Average 35.02 3.32 4.80 3.65 4.72 3.44 1.28 0.77 1.42 Micro avg ····················· 0.73 1.38 382 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Size Evolution of configuration Shattered Document with Markov Chains with heuristics, Trimmed Tree
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 2 7 4 4 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 3 4 4 3 3 3 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 4 4 4 3 3 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 6 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 8 4 6 4 4 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 9 6 4 3 3 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 10 5 3 3 4 3 3 3 3 3 3 3 3 3 4 3 3 3 4 3 3 Oracle 11 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 12 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 13 3 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 14 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 15 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 16 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 17 7 3 4 3 3 4 3 4 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 18 5 3 4 4 3 3 3 4 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 19 7 4 4 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 20 4 5 4 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Oracle 21 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 22 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 23 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 24 7 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 25 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 26 4 4 3 3 4 3 4 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 27 5 4 3 4 3 3 4 4 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 28 4 3 3 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 29 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 30 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 31 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 32 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 33 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 34 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 35 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Oracle 36 5 4 3 4 3 3 3 3 4 4 3 3 3 3 3 3 3 3 3 3 Oracle 37 6 4 4 4 3 3 3 3 3 4 3 3 3 3 3 3 3 3 3 3 Oracle 38 7 7 3 4 3 4 4 3 4 3 3 3 3 3 3 3 3 3 3 3 Oracle 39 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 40 4 4 5 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 41 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 42 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Oracle 43 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 44 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 Oracle 45 7 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 46 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Oracle 47 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 48 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 49 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Oracle 50 5 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Total 236 186 174 179 176 177 174 175 174 175 172 172 172 173 172 172 172 173 172 172 Average 4.72 3.72 3.48 3.58 3.52 3.54 3.48 3.50 3.48 3.50 3.44 3.44 3.44 3.46 3.44 3.44 3.44 3.46 3.44 3.44 D.20. COMPLETE RESULTS OF CONFIGURATION SHATTERED DOCUMENT WITH MARKOV CHAINS WITH HEURISTICS, TRIMMED TREE383
Cost Results of configuration Shattered Document with Markov Chains with heuristics, Trimmed Tree
Sum Min Max Avg First Last Diff Rate Gain Oracle 1 27.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 2 25.00 3.00 7.00 4.16 7.00 3.00 4.00 0.43 2.34 Oracle 3 28.00 3.00 4.00 3.50 4.00 4.00 0.00 1.00 1.00 Oracle 4 25.00 3.00 4.00 3.57 4.00 3.00 1.00 0.75 1.34 Oracle 5 33.00 4.00 5.00 4.13 5.00 4.00 1.00 0.80 1.25 Oracle 6 33.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 7 34.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 8 40.00 3.00 6.00 3.63 4.00 3.00 1.00 0.75 1.34 Oracle 9 27.00 3.00 6.00 3.86 6.00 3.00 3.00 0.50 2.00 Oracle 10 65.00 3.00 5.00 3.25 5.00 3.00 2.00 0.60 1.66 Oracle 11 37.00 3.00 4.00 3.09 4.00 3.00 1.00 0.75 1.34 Oracle 12 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 13 43.00 2.00 3.00 2.87 3.00 3.00 0.00 1.00 1.00 Oracle 14 33.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 15 25.00 3.00 4.00 3.13 4.00 3.00 1.00 0.75 1.34 Oracle 16 42.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 17 34.00 3.00 7.00 3.78 7.00 3.00 4.00 0.43 2.34 Oracle 18 38.00 3.00 5.00 3.45 5.00 3.00 2.00 0.60 1.66 Oracle 19 36.00 4.00 7.00 6.00 7.00 7.00 0.00 1.00 1.00 Oracle 20 24.00 3.00 5.00 4.00 4.00 4.00 0.00 1.00 1.00 Oracle 21 35.00 3.00 7.00 3.50 7.00 3.00 4.00 0.43 2.34 Oracle 22 31.00 3.00 4.00 3.10 4.00 3.00 1.00 0.75 1.34 Oracle 23 49.00 3.00 7.00 3.27 7.00 3.00 4.00 0.43 2.34 Oracle 24 38.00 3.00 7.00 3.45 7.00 3.00 4.00 0.43 2.34 Oracle 25 18.00 2.00 2.00 2.00 2.00 2.00 0.00 1.00 1.00 Oracle 26 49.00 3.00 4.00 3.27 4.00 3.00 1.00 0.75 1.34 Oracle 27 45.00 3.00 5.00 3.46 5.00 3.00 2.00 0.60 1.66 Oracle 28 25.00 3.00 4.00 3.57 4.00 3.00 1.00 0.75 1.34 Oracle 29 17.00 3.00 5.00 3.40 5.00 3.00 2.00 0.60 1.66 Oracle 30 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 31 35.00 3.00 5.00 3.18 5.00 3.00 2.00 0.60 1.66 Oracle 32 43.00 3.00 4.00 3.07 4.00 3.00 1.00 0.75 1.34 Oracle 33 36.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 34 56.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 35 20.00 2.00 2.00 2.00 2.00 2.00 0.00 1.00 1.00 Oracle 36 45.00 3.00 5.00 3.46 5.00 3.00 2.00 0.60 1.66 Oracle 37 40.00 3.00 6.00 3.63 6.00 3.00 3.00 0.50 2.00 Oracle 38 45.00 3.00 7.00 4.09 7.00 3.00 4.00 0.43 2.34 Oracle 39 21.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 40 47.00 3.00 5.00 3.36 4.00 3.00 1.00 0.75 1.34 Oracle 41 25.00 3.00 4.00 3.13 4.00 3.00 1.00 0.75 1.34 Oracle 42 5.00 5.00 5.00 5.00 5.00 5.00 0.00 1.00 1.00 Oracle 43 30.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 44 36.00 6.00 6.00 6.00 6.00 6.00 0.00 1.00 1.00 Oracle 45 31.00 3.00 7.00 3.45 7.00 3.00 4.00 0.43 2.34 Oracle 46 70.00 7.00 7.00 7.00 7.00 7.00 0.00 1.00 1.00 Oracle 47 31.00 3.00 4.00 3.10 4.00 3.00 1.00 0.75 1.34 Oracle 48 40.00 3.00 4.00 3.08 4.00 3.00 1.00 0.75 1.34 Oracle 49 30.00 3.00 3.00 3.00 3.00 3.00 0.00 1.00 1.00 Oracle 50 29.00 3.00 5.00 3.22 5.00 3.00 2.00 0.60 1.66 Total 1751.00 166.00 240.00 182.45 236.00 172.00 64.00 38.25 71.25 Average 35.02 3.32 4.80 3.65 4.72 3.44 1.28 0.77 1.42 Micro avg ····················· 0.73 1.38 384 APPENDIX D. DETAILED RESULTS OF EXPERIMENTS
Cost Evolution of configuration Shattered Document with Markov Chains with heuristics, Trimmed Tree
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Oracle 1 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 2 7.00 4.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 3 4.00 4.00 3.00 3.00 3.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 4 4.00 4.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 5 5.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 6 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 7 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 8 4.00 6.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 9 6.00 4.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 10 5.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 4.00 3.00 3.00 Oracle 11 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 12 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 13 3.00 2.00 2.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 14 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 15 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 16 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 17 7.00 3.00 4.00 3.00 3.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 18 5.00 3.00 4.00 4.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 19 7.00 4.00 4.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 20 4.00 5.00 4.00 3.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Oracle 21 7.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 22 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 23 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 24 7.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 25 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 26 4.00 4.00 3.00 3.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 27 5.00 4.00 3.00 4.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 28 4.00 3.00 3.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 29 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 30 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 31 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 32 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 33 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 34 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 35 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 2.00 Oracle 36 5.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 37 6.00 4.00 4.00 4.00 3.00 3.00 3.00 3.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 38 7.00 7.00 3.00 4.00 3.00 4.00 4.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 39 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 40 4.00 4.00 5.00 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 41 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 42 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 5.00 Oracle 43 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 44 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 6.00 Oracle 45 7.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 46 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 7.00 Oracle 47 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 48 4.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 49 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Oracle 50 5.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 3.00 Total 236 186 174 179 176 177 174 175 174 175 172 172 172 173 172 172 172 173 172 172 Average 4.72 3.72 3.48 3.58 3.52 3.54 3.48 3.50 3.48 3.50 3.44 3.44 3.44 3.46 3.44 3.44 3.44 3.46 3.44 3.44