KNOWLEDGE ORGANIZATION KO Contents

Knowl. Org. 32(2005)No.1

KNOWLEDGE ORGANIZATION KO

Official Quarterly Journal of the International Society for Knowledge Organization ISSN 0943 – 7444 International Journal devoted to Concept Theory, Classification, Indexing and Knowledge Representation

Contents

Articles Book Review BROUGHTON, VANDA. Essential Classification. Elena García-Barriocanal, Miguel A. Sicilia, New York, NY: Neal-Schuman, 2004. 324 p. and Salvador Sánchez-Alonso. ISBN 1-55570-507-3...... 47 Usability Evaluation of Ontology Editors ...... 1

Hur-Li Lee and Hope A. Olson. Knowledge Organization Literature Hierarchical Navigation: 32 (2005) No.1...... 50 An Exploration of Yahoo! Directories ...... 10 Abdus Sattar Chaudhry and Goh Hui Ling. Personal Author Index Building Taxonomies using Organizational 32 (2005) No.1...... 60 Resources: A Case of Business Consulting Environment...... 25 Index to Volume 31 (2004) ...... 61

Knowl. Org. 32(2005)No.1

KO KNOWLEDGE ORGANIZATION

Contents page

García-Barriocanal, Elena, Miguel A. Sicilia, and Salvador pertise, knowledge of the discipline, and time required to Sanchez-Alonso. Usability Evaluation of Ontology Edi- complete the search. Without a definitive conclusion, we tors. Knowledge Organization, 32(1). 1-9. 27 refs. suggest a number of directions for further research.

ABSTRACT: Ontology editors are software tools that allow the creation and maintenance of ontologies through a graphical user interface. As the semantic web effort grows, a larger community of users for this kind of tool is ex- Chaudhry, Abdus Sattar, and Goh Hui Ling. Building pected. New users include people not specifically skilled in Taxonomies Using Organizational Resources: A Case of the use of ontology formalisms. In consequence, the us- Business Consulting Environment. Knowledge Organiza- ability of ontology editors can be viewed as a key adoption tion, 32(1). 25-46. 33 refs. precondition for semantic web technologies. In this paper, the usability evaluation of several representative ontology ABSTRACT: Taxonomies are becoming an increasingly editors is described. This evaluation is carried out by com- important tool for companies to effectively manage infor- bining a heuristic pre-assessment with a subsequent user- mation, particularly in the business consulting environ- testing phase. The target population is comprised of people ment, where information is considered a main asset and a with no specific ontology-creation skills that have a general key product. This paper describes a case study of develop- knowledge about domain modelling. For this kind of user, ing a taxonomy system for a regional business consulting current editors are adequate for the creation and mainte- company. The taxonomy, consisting of 12 main categories nance of simple ontologies. Also, there is room for im- and approximately 500 terms, was built based on the exist- provement, especially in browsing mechanisms, help sys- ing knowledge structure and information needs of consult- tems, and visualization metaphors. ants in a selected company. This prototype can be conveniently utilised and adapted by other companies in their efforts to develop their own taxonomy system.

Lee, Hur-Li, and Hope A. Olson. Hierarchical Naviga-

tion: An Exploration of Yahoo! Directories. Knowledge

Organization, 32(1). 10-24. 37 refs.

ABSTRACT: Although researchers have theorized the

critical importance of classification in the organization of

information, the classification approach seems to have

given way to the alphabetical subject approach in retrieval

tools widely used in libraries, and research on how users

utilize classification or classification-like arrangements in

information seeking has been scant. To better understand

whether searchers consider classificatory structures a viable

alternative to information retrieval, this article reports on a

study of how 24 library and information science students

used Yahoo! directories, a popular search service resem-

bling classification, in completing an assigned simple task.

Several issues emerged from the students’ reporting of

their search process and a comparison between hierarchical

navigation and keyword searching: citation order of facets,

precision vs. recall, and other factors influencing searchers’ successes and preferences. The latter included search ex- This contents page may be reproduced without charge. Knowl. Org. 32(2005)No.1

KNOWLEDGE ORGANIZATION KO

KNOWLEDGE ORGANIZATION Dr. M.A. GOPINATH, 7604 Sandy Lake Ct., Raleigh, NC- 27613, USA, (919) 788-9099 voice. Email: [email protected] This journal is the organ of the INTERNATIONAL SOCIETY FOR KNOWLEDGE ORGANIZATION (General Secretariat: Dr. Rebecca GREEN, College of Information Studies, Hornbake H. Peter OHLY, Social Science Information Center, Lennestr. 30, Bldg. (So. Wing), Room 4105, University of Maryland, College D-53113 Bonn, Germany. Park, MD 20742-4345 USA. Email: [email protected] Dr. Norbert HENRICHS, Im Luftfeld 80, D-40489 Düsseldorf, Germany Dr. Barbara H. KWASNIK, Professor, School of Information Editors Studies, Syracuse University, Syracuse, NY 13244 USA, (315) 443-4547 voice, (315) 443-4506 fax. Email: [email protected] Dr. Ia MCILWAINE, Director, School of Library, Archive & In- Dr. Richard P. SMIRAGLIA (Editor-in-Chief), Palmer School of formation Studies, University College London, Gower Street, Library and Information Science, Long Island University, 720 London WC1E 6BT U.K. Email: [email protected] Northern Blvd., Brookville NY 11548 USA. Email: [email protected] Prof. Dr. Francis MIKSA, Graduate School of Library and Infor- mation Science, University of Texas at Austin, SZB 564, Austin, Dr. Clément ARSENAULT (Book Review Editor), École de bib- TX 78712-1276 liothéconomie et des sciences de l’information, Université de Montréal, C.P. 6128, succ. Centre-ville, Montréal (QC) H3C 3J7, Ms. Joan S. MITCHELL, Editor in Chief, Dewey Decimal Classi- Canada. Email: [email protected] fication, OCLC Online Computer Library Center, Inc., 6565 Frantz Road, Dublin, OH 43017-3395 USA. Dr. Gerhard RIESTHUIS (Literature Editor), Liendenhof 60, Email: [email protected] NL-1108 HB Amsterdam, Netherlands. Email: [email protected] Dr. Widad MUSTAFA el HADI, URF IDIST, Université Charles de Gaulle Lille 3, BP 149, 59653 Villeneuve D’Ascq, France Dr. Nancy WILLIAMSON (Classification Research News Edi- tor), Faculty of Information Studies, University of Toronto, 140 H. Peter OHLY, IZ Sozialwissenschaften, Lennestr. 30, 53113 St. George Street, Toronto, Ontario M5S 3G6 Canada. Bonn Germany. Email: [email protected] Email: [email protected] Dr. Hope A. OLSON, School of Information Studies, 522 Bolton Hanne ALBRECHTSEN, Institute of Knowledge Sharing, Bu- Hall, University of Wisconsin-Milwaukee, Milwaukee, WI 53201 reauet, Slotsgade 2, 2nd floor DK-2200 Copenhagen N Denmark. USA. Email: [email protected] Email: [email protected] Ms. Annelise Mark PEJTERSEN, Systems Analysis Dept., Risoe Dawn E. GROSS (Editorial Assistant), Palmer School of Library National Laboratory, P.O. Box 49, DK-4000 Roskilde, Denmark and Information Science, Long Island University. Dr. M. P. SATIJA, Guru Nanak Dev University, School of Li- brary and Information Science, Amritsar-143 005, India Dr. J.F. (Jos) SCHREINEMAKERS, Section Business Informat- ics, School of Computer Science, Faculty of Sciences, Vrije Uni- Consulting Editors versiteit Amsterdam, The Netherlands. Email: [email protected] Dr. Otto SECHSER, In der Ey 37, CH-8047 Zürich, Switzerland Prof. Dr. Ulf G. BARANOW, Rua Ubaldino do Amaral 580, Dr. Winfried SCHMITZ-ESSER, Rothenbaumchaussee 3, D – Apto. 51, BR-80060-190, Curitiba - PR, Brazil 20148 Hamburg, Germany Prof. Clare BEGHTOL, Faculty of Information Studies, Univer- Dr. Dagobert SOERGEL, College of Information Studies, Horn- sity of Toronto, 140 St. George Street, Toronto, Ontario M5S bake Bldg. (So. Wing), Room 4105, University of Maryland, Col- 3G6, Canada. Email: [email protected] lege Park, MD 20742 Dr. Gerhard BUDIN, Dept.of Philosophy of Science, University Dr. Eduard R. SUKIASYAN, Editor-in-chief, Library Biblio- of Vienna, Sensengasse 8, A-1090 Wien, Austria. graphical Classification (LBC), National Classification System of Email: [email protected] Russia, Russian State Library. Email: [email protected]. Dr. Jesús GASCÓN GARCÍA, Facultat de Biblioteconomia i Dr. Martin van der WALT, Department of Information Science, Documentació, Universitat de Barcelona, C. Melcior de Palau, University of Stellenbosch, Private Bag X1, Stellenbosch 7602, 140, 08014 Barcelona, Spain. Email: [email protected] South Africa. Email: [email protected] Claudio GNOLI, University of Pavia, Mathematics Department Prof. Dr. Harald ZIMMERMANN, Softex, Schmollerstrasse 31, Library, via Ferrata 1, I-27100 Pavia, Italy. Email: [email protected] D-66111 Saarbrücken, Germany Knowl. Org. 32(2005)No.1

KO KNOWLEDGE ORGANIZATION

Publisher References should be listed alphabetically by author at the ERGON-Verlag, Grombühlstr. 7, GER-97080 Würzburg end of the article. Journal names should not be abbreviated. Phone: +49 (931) 280084; FAX +49 (931) 282872 Multiple citations by the same author should be listed chrono- E-mail: [email protected]; http://www.ergon-verlag.de logically and should each spell out the author’s name. Articles appearing in the same year should have the following format: Editor-in-chief (Editorial office) Jones, Thomas 1990a ... , Jones, Thomas 1990b … Dr. Richard P. SMIRAGLIA (Editor-in-Chief), Palmer School Examples: of Library and Information Science, Long Island University, 720 Dahlberg, I. 1978. A referent-oriented, analytical concept theory Northern Blvd., Brookville NY 11548 USA. for INTERCONCEPT. International Classification, 5(3). Email: [email protected] 142-151. Graesser, A., Person, N. & Huber, J. 1992. Mechanisms that Instructions for Authors generate questions. In T. W. Lauer, E. Peacock, & A. C. Graesser (Eds.). Questions and Information Systems. Hills- Manuscripts should be submitted electronically (in Word, dale, NJ: Lawrence Erlbaum Associates. 167-187. Wordperfect, or RTF format) in English only to the editor-in- Sager, J.C. 1990. A Practical Course in Terminology Processing. chief and should be accompanied by an indicative abstract of Amsterdam: John Benjamins. 100 or 200 words. Submissions via email are preferred; submis- Sukiasyan, E. R. 1996. Change as a problem of classification sys- sions will also be accepted via post provided that submissions tem development. In R. Green (Ed.). Knowledge Organiza- are accompanied by a 3.5” diskette encoded in Word, Wordper- tion and Change: Proceedings of the 4th International ISKO fect, or RTF format. Conference. Frankfurt: Indeks Verlag. 119-122. A separate title page should include the article title and the Footnotes should be kept to a minimum. They should be in- author’s name, postal address, and E-mail address, if available. dicated in the text with numbered superscripts, and the corre- Only the title of the article should appear on the first page of sponding notes should be collected at the end of the article, be- the text. To protect anonymity, the author’s name should not fore the references, under the heading Notes. appear on the manuscript, and all references in the body of the Illustrations should be kept to a necessary minimum and text and in footnotes that might identify the author to the re- should be submitted electronically when possible. Photographs viewer should be removed and cited on a separate page. Articles (including color and half-tone) should be scanned with a mini- that do not conform to these specifications will be returned to mum resolution of 600 dpi and saved as tif files (Tagged Image authors. File Format preferred). Tables and figures should be embedded Criteria for acceptance will be appropriateness to the field of within the document or, alternatively, saved as separate files the journal (see Scope and Aims), taking into account the merit with clear instructions indicating their placement in the text. of the contents and presentation. The manuscript should be Tables should contain a number and title at the top, and all col- concise and should conform as much as possible to professional umns and rows should have headings. All illustrations should be standards of English usage and grammar. Manuscripts are re- cited in the text as Figure 1, Figure 2, etc. or Table 1, Table 2, ceived with the understanding that they have not been previ- etc. Illustrations submitted in hard copy only should be marked ously published, are not being submitted for publication else- to indicate their placement in the text. where, and that if the work received official sponsorship, it has Upon acceptance of a manuscript for publication, authors been duly released for publication. Submissions are refereed, and must provide a wallet-size photo and a one-paragraph bio- authors will usually be notified within 6 to 10 weeks. Unless graphical sketch. The photograph should be scanned with a specifically requested, manuscripts and illustrations will not be minimum resolution of 600 dpi and saved as a tif file (Tagged returned. Image File Format). The text should be structured by numbered subheadings. It should contain an Introduction, giving an overview and stating Advertising the purpose, a main body, describing in sufficient detail the ma- Responsible for advertising: Dr. H.-J. Dietrich, ERGON- terials or methods used and the results or systems developed, Verlag, Grombühlstr. 7, 97080 Würzburg (Germany). and a conclusion or summary. Reference citations within the text should have the following © 2005 by ERGON-Verlag Dr. H.-J. Dietrich. form: (author year). For example, (Jones 1990). Specific page All Rights reserved. numbers are optional, but preferred when applicable, e.g. (Jones Printed in the Federal Republic of Germany 1990, 100). A citation with two authors would read (Jones & by Offizin Hildburghausen GmbH Druckhaus. Smith, 1990); three or more authors would be: (Jones et al., 1990). When the author is mentioned in the text, only the date and optional page number should appear in parenthesis – e.g. KO is published quarterly by ERGON-Verlag. According to Jones (1990), … The price is € 106,00/ann. including airmail delivery. Knowl. Org. 32(2005)No.1

KNOWLEDGE ORGANIZATION KO

Scope Aims

The more scientific data is generated in the impetuous Thus, KNOWLEDGE ORGANIZATION is a forum for present times, the more ordering energy needs to be expended all those interested in the organization of knowledge on a uni- to control these data in a retrievable fashion. With the abun- versal or a domain-specific scale, using concept-analytical or dance of knowledge now available the questions of new solu- concept-synthetical approaches, as well as quantitative and tions to the ordering problem and thus of improved classifica- qualitative methodologies. KNOWLEDGE ORGANIZA- tion systems, methods and procedures have acquired unfore- TION also addresses the intellectual and automatic compila- seen significance. For many years now they have been the fo- tion and use of classification systems and thesauri in all fields cus of interest of information scientists the world over. of knowledge, with special attention being given to the prob- Until recently, the special literature relevant to classifica- lems of terminology. tion was published in piecemeal fashion, scattered over the KNOWLEDGE ORGANIZATION publishes original numerous technical journals serving the experts of the various articles, reports on conferences and similar communications, fields such as the Newsletters of the International Society for Knowledge Organization (ISKO News) and the Committee on Classifi- cation Research of the International Federation for Informa- philosophy and science of science tion and Documentation (FID/CR News) as well as book re- science policy and science organization views, letters to the editor, and an extensive annotated bibliog- mathematics, statistics and computer science raphy of recent classification and indexing literature, covering library and information science some 500 items in each issue. archivistics and museology KNOWLEDGE ORGANIZATION should therefore be journalism and communication science available at every university and research library of every coun- industrial products and commodity science try, at every information center, at colleges and schools of li- terminology, lexicography and linguistics brary and information science, in the hands of everybody interested in the fields mentioned above and thus also at every office for updating information on any topic related to the Beginning in 1974, KNOWLEDGE ORGANIZATION problems of order in our information-flooded times. (formerly INTERNATIONAL CLASSIFICATION) has KNOWLEDGE ORGANIZATION was founded in been serving as a common platform for the discussion of both 1973 by an international group of scholars with a consulting theoretical background questions and practical application board of editors representing the world’s regions, the special problems in many areas of concern. In each issue experts from classification fields, and the subject areas involved. From many countries comment on questions of an adequate struc- 1974-1980 it was published by K.G. Saur Verlag, München. turing and construction of ordering systems and on the prob- Back issues of 1978-1992 are available from ERGON-Verlag, lems of their use in opening the information contents of new too. literature, of data collections and survey, of tabular works and As of 1989, KNOWLEDGE ORGANIZATION has be- of other objects of scientific interest. Their contributions have come the official organ of the INTERNATIONAL SOCI- been concerned with ETY FOR KNOWLEDGE ORGANIZATION (ISKO) and is included for every ISKO-member, personal or institu- tional in the membership fee (US $ 55/US $ 110). (1) clarifying the theoretical foundations (general ordering Rates: From 2004 on for 4 issues/ann. (including indexes) theory/science, theoretical bases of classification, data € 103,00 (forwarding costs included). Membership rates see analysis and reduction) above. (2) describing practical operations connected with index- ERGON-Verlag, Grombühlstr. 7, GER-97080 Würzburg; ing/classification, as well as applications of classification Phone: +49 (931) 280084; FAX +49 (931) 282872; E-mail: systems and thesauri, manual and machine indexing [email protected]; http://www.ergon-verlag.de (3) tracing the history of classification knowledge and methodology The contents of this journal are indexed and abstracted in Refera- (4) discussing questions of education and training in classi- tivnyi Zhurnal Informatika and in the following online databases: fication Information Science Abstracts, INSPEC, Library and Information (5) concerning themselves with the problems of terminol- Science Abstracts (LISA), Library Literature, PASCAL, Sociologi- ogy in general and with respect to special fields. cal Abstracts, and Web Science & Social Sciences Citation Index. Knowl. Org. 32(2005)No.1 1 E. García-Barriocanal, M. A. Sicilia, and S. Sanchez-Alonso: Usability Evaluation of Ontology Editors

Usability Evaluation of Ontology Editors

Elena García-Barriocanal*, Miguel A. Sicilia**, Salvador Sánchez-Alonso***

*/**/***University of Alcalá, Polytechnic School, Computer Science Department, Ctra. Barcelona km. 33.600, Alcalá de Henares, Madrid, Spain, 28871 *[email protected] **[email protected] ***[email protected]

Dr. Elena García-Barriocanal obtained a university degree in computer science from the Pontifical University of Salamanca in Madrid and a PhD from the computer science department of the Univer- sity of Alcalá. From September 1998 to February 1999 she worked as a lecturer in the computer languages and information systems department of the Pontifical University, and in 1999 she joined the computer science department of University of Alcalá where she is associate professor. Her research interests mainly focus on topics related to the role of knowledge representation in fields like human- computer interaction and learning technologies.

Miguel A. Sicilia obtained a university degree in computer science from the Pontifical University of Salamanca in Madrid, Spain and a Phd from Carlos III University in Madrid, Spain. From 1997 to 1999 he worked as assistant professor at the Pontifical University, after which he joined the computer science department of the Carlos III University in Madrid as a lecturer, working simultaneously as a software architect in e-commerce consulting firms. From 2002 to October 2003, he worked as a full- time lecturer at Carlos III University. Currently, he works as an associate professor at the computer science department, University of Alcalá (Madrid). His research interests are primarily adaptive hy- permedia, learning technology, semantic web and human-computer interaction.

Dr. Salvador Sanchez-Alonso obtained a university degree in computer science from the Pontifical University of Salamanca (Spain) in 1997, and a PhD in computing from the Polytechnic University of Madrid (Spain) in 2005. He worked as an assistant professor at the Pontifical University of Salamanca from 1997 to 2000 and from 2002 to 2005. He also worked as a software engineer at a software solutions company during 2000 and 2001. From 2005, he is a professor of the computer science department of the University of Alcala, in Spain. His research interests include learning objects reusability, metadata, object-oriented technologies and web engineering.

García-Barriocanal, Elena, Miguel A. Sicilia, and Salvador Sanchez-Alonso. Usability Evaluation of Ontology Editors. Knowledge Organization, 32(1). 1-9. 27 refs.

ABSTRACT: Ontology editors are software tools that allow the creation and maintenance of ontologies through a graphical user interface. As the semantic web effort grows, a larger community of users for this kind of tool is expected. New users include people not specifically skilled in the use of ontology formalisms. In consequence, the usability of ontology editors can be viewed as a key adoption precondition for semantic web technologies. In this paper, the usability evaluation of several representative ontology editors is described. This evaluation is carried out by combining a heuristic pre-assessment with a subsequent user-testing phase. The target population is comprised of people with no specific ontology-creation skills that have a general knowledge about domain modelling. For this kind of user, current editors are adequate for the creation and maintenance of simple ontologies. Also, there is room for improvement, especially in browsing mechanisms, help systems, and visualization metaphors.

2 Knowl. Org. 32(2005)No.1 E. García-Barriocanal, M. A. Sicilia, and S. Sanchez-Alonso: Usability Evaluation of Ontology Editors

1. Introduction ristic evaluation. Later on, section 3 includes the results. Finally, conclusions and future research direc- 1.1. The Role of Ontology Editors in the Semantic Web tions are sketched in Section 4.

From an Artificial Intelligence perspective, ontolo- 1.2. Overall Description of the Evaluation Method gies can be described as a kind of knowledge representation (Davis, Shrobe and Szolovits 1993) for Usability evaluation is considered an important di- shared conceptualizations of specific domains (De- mension in the evaluation of systems that have some cker et al. 2000), which is considered as a key ena- kind of knowledge acquisition interfaces (Adelman bling technology for e-commerce (Fensel 2001) and and Riedel 1997). Reports on usability evaluation of for the so-called semantic web (Ding et al. 2002). To various knowledge representation systems have ad- date, widely used object-oriented modelling lan- dressed different usability measures, e.g. the time guages, like the UML (Unified Modelling Language) needed to learn specific knowledge entry functional- (Object Management Group 2003) have been used to ities (Shahar et al. 1999), technical aspects that di- represent ontologies (Cranefield and Purvis 1999; rectly affect the user (e.g., explanation), error han- Cranefield, Haustein and Purvis 2001). However, dling, system efficiency, and adequacy of program- current ontology formalisms, like KIF (Knowledge ming interfaces (McGuinness and Patel-Schneider Interchange Format) (NCITS 1998), either exceed 1998). the built-in information representation capabilities of In previous studies (Duineveld et al. 2000), a the core meta-models of those languages (Cranefield comparison between six ontology-engineering tools and Purvis 1999), or make necessary the introduction was made in accordance with three different dimen- of a set of supplementary notational extensions (Ba- sions: the user interface, the ontology-related issues clawski et al. 2001), both of which result in harder-to- found in the tool, and the tool’s capacity to support learn modelling languages. In consequence, it is ex- the construction of an ontology by several people at pected that knowledge representation (KR)-specific different locations. Duineveld et al. (2000) describe tools, like Protégé (Noy et al. 2001), will continue to their opinion about ontology engineering tools by be used for ontology creation and editing in the near using a checklist, but potential users did not take future, taking into account that frame-based mark-up part in the evaluation. languages – such as RDF (Resource Description In this study, a conventional usability evaluation Framework) and its extensions – are intended for has been carried out combining two widespread computer interchange rather than for direct human techniques: heuristic evaluation and user testing. reading and writing. Three groups of users/evaluators were formed, each In this work, the term Ontology editor (OE) is with different backgrounds, to report on the usabil- used to refer to KR-specific software tools, explicitly ity of selected OEs. The main objective was not to based on any ontology formalism, which allow the analyse specific knowledge entry techniques, but, in- interactive creation and updating of ontologies stead, to consider general user interaction issues. through a graphical user interface. The focus of this Gómez-Pérez (1994) proposed an explicit distinc- research is on the specific human-interaction charac- tion between evaluation, and assessment of knowl- teristics of these tools, assuming that efficient and edge sharing technology (KST), including ontology easy to use ontology creation and maintenance ap- editors. “Evaluation means to judge technically the plications are a critical element in the semantic web features of KST, and assessment refers to the usabil- infrastructure. Taking into account that a larger ity and utility of KST in companies.” However, the community of users would include a larger number term “evaluation” has been used for the sake of clar- of non-KR specialists, we aim at investigating ity in the application of the most common methods whether currently available OEs are usable for peo- and techniques to measure system usability, as this is ple without a deep understanding of (or experience a more familiar term in the Human-Computer Inter- in) ontology modelling. action (HCI) community. The rest of this paper is structured as follows. In Preceded by an heuristic evaluation (Nielsen 1994) the rest of this section the general principles and aimed at obtaining the present assessment of usability methods of the evaluation are described. In Section problems, conventional user testing techniques (Du- 2, the test procedure is explained in detail, including mas and Redish 1999) have been selected as the main the findings obtained from the pre-assessment heu- approach. These methods are considered complemen- Knowl. Org. 32(2005)No.1 3 E. García-Barriocanal, M. A. Sicilia, and S. Sanchez-Alonso: Usability Evaluation of Ontology Editors

tary, as each one detects usability problems over- Although several OEs currently exist – an exhaustive looked by the other (Nielsen 1994). Our evaluation is list can be found in Denny (2002) – the stability of mainly formative, in the sense that it is targeted to the versions, as well as platform and licensing con- expose usability problems in current tools. However, straints, have served as a filter in the selection of because of the process, some aspects have also been OEs for this study. From the selected OEs, those identified that could be used as the point of departure that did not allow both edition and creation proc- for a summative evaluation – i.e. an evaluation to de- esses were discarded. Later, practitioners were asked termine which among several alternatives to ontology for their opinions on the most widely used OEs, re- editing is best. In addition, because OEs are far too sulting in the list above. In the overall process of se- complex to test all of their functionalities at once, the lection, the main criterion was that of comparing study was purposefully limited to answering ques- two types of interfaces: HTML-based interfaces and tions relevant the community of OE users that do ‘GUI-desktop’ interfaces. not come from the KR field. The general concern of our study is to determine 2. Evaluation design the ease of use of OEs or, in other words, to be able to provide an answer to the question: “If users have 2.1. Specific Concerns and Measures limited or null experience in ontology creation and maintenance, or if they have an exploratory learning The specific concerns of the evaluation were moti- style, are OEs robust in terms of usability?” For this vated by an heuristic analysis (Nielsen 1994) carried purpose, users will be considered to have an explora- out by experts with at least one year of previous ex- tory learning style if they prefer to learn about the perience in ontology editing. The procedure for the use of the system by investigating it on their own evaluation consisted of three phases: a pre-selection initiative – often in pursuit of a real or artificial task phase, in which some tools could be discarded, the – instead of working through precisely sequenced actual evaluation, and a debriefing and severity-rating training materials. More specifically, the following phase. Although experts were free to take their own two concerns have been raised: “How easy is it to approach, it was suggested that they edit simple on- create a new ontology with current OEs?” and “how tologies taken from the Internet, browse sample on- easy is it to browse, search and perform updating tologies downloaded from the DAML library, and tasks on large ontologies with current OEs?” Re- search in the (KA)2 ontology (Benjamins et al. 1999). lated activities allowed in some OEs like Protégé The latter two tasks were only performed for those (Noy, Fergerson and Musen 2000), such as semantic OEs that included support for loading RDF ontolo- web page annotation, collaborative ontology edition, gies. Four evaluators carried out the study, therefore, or ontology meta-modelling, are not considered according to Nielson’s curve, more than fifty percent here, because they are not directly supported by the of the usability problems are estimated to have been most commonly used OEs. found (Nielsen 1992). The following tools were initially selected for in- In the first phase, after the first three experts’ pre- clusion in the test: evaluations, KADS22 was discarded. This decision was based on the fact that it did not adhere to com- – Protégé 2000 1.6.2 mon platform conventions, and to its clear orienta- (http://protege.stanford.edu/); tion to CML-file editing. It is important to note that – OntoEdit 2.0 KADS22 is considered to be in development. In ad- (http://www.ontoprise.de); dition, WebOde was not evaluated, because it com- – OILEd 2.2a bines HTML forms with graphical interfaces based (http://oiled.man.ac.uk); on applets, and this would have complicated the – the KSL Ontology Editor categorization and comparison of the two estab- (http://www-ksl-svc.stanford.edu:5915/); lished types of tools. Table 1 summarizes the most – WebOde 1.1 relevant results of the second phase of the analysis, (http://kw.dia.fi.upm.es/wpbs/); structured around Nielsen’s heuristics. The experts – WebOnto were advised to use Tognazzini’s (2002) principles as (http://kmi.open.ac.uk/projects/webonto/); and, a checklist. The column marked ‘S’, shows the sever- – KADS22 ity rating estimated by the experts during the third (http://hcs.science.uva.nl/projects/kads22/). phase. According to Nielsen (Nielsen 1994), a scale 4 Knowl. Org. 32(2005)No.1 E. García-Barriocanal, M. A. Sicilia, and S. Sanchez-Alonso: Usability Evaluation of Ontology Editors

Heuristic Problems Found S(0..49) A B C D F Visibility of system status Lack of status bar 2 X - - X X Match between system and the real world Unexplained system-oriented terms 3 - - X - X No printing functionality 3 X X X X X User Control and freedom No “un-do”/“re-do” functionality 3 X - X . X No “replace” functionality 2 X X - X X No “copy & paste” functionality in hierarchies 2 X - X X X No “drag & drop” functionality in hierarchies 2 - X X X X No “cut & paste” functionality in hierarchies 2 X - - X X No tool tips in some elements 2 X - - X X Pop up menu navigation using cursors is not 1 X X - na na permitted No searching slots functionality 2 X X - - - Consistency and standards Does not follow menu platform 1 - - - - X conventions Recognition rather than recall Actions available only through 2 - - X na na Flexibility and efficiency of use Excessive time to launch 3 - - - X X No key accelerator 2 X X - na X

Table 1. Heuristic evaluation results. from 0 to 4 is used, where 0 stands for no problems, ontology and to set the initial basic properties?; and, 1 stands for cosmetic problems, 2 for minor prob- 2) how easy is it for new users to define a new on- lems, 3 for major problems and 4 for problems that tology construct of type X (where X stands for, re- are imperative to fix. When an expert detected a spectively, a class, a property, and an instance)? problem on an OE, the problem was recorded by Next, the following specific issues were derived marking an ‘x’ in the corresponding column. The from the second general concern about the ease with acronym n.a. stands for not applicable. The main which one might browse, search, and perform updat- conclusion of the heuristic evaluation is that major ing tasks on large ontologies with current OEs. usability problems are scarce, except for inadequate These questions are: 1) how easy is it to find a spe- help, and user error reporting systems. cific ontology construct of type X?; 2) how easy is it After the heuristic evaluation, WebOnto was dis- to navigate through the generalization/specialization carded from the user test due to several behaviour hierarchy?; and, 3) how easy is it to update a charac- problems related to the interface (buttons disappear- teristic C (e.g. name, property/slot, instance) of an ing in the toolbar, operations that did not report er- existing ontology construct of type X? rors but did nothing, and the like.) This inconsistent In all cases, the time to complete a task and the performance might have been due to minor issues, number of errors raised in completing that task have such as a non-compatible Web browser version or been selected as a measure for the issue. The applica- any other problem related to the common platform, tion response time has not been included in the but the behavior problems made it impossible to evaluation, as it is easy to verify that some of the carry out a fair comparison with the other tools. current OEs require further improvements in pars- Nonetheless, the graphical editing capabilities of ing and/or caching of large ontologies. An example WebOnto, that provide an appropriate and efficient is the large Universal Standard Products and Services way to edit hierarchies, and the unique collaborative Classification ontology that takes about two minutes editing capabilities, must be highlighted. to load in Protégé 2000 on a Pentium III computer Based on the heuristic evaluation results, the gen- with 1GB of main memory. Loading this particular eral concerns of the test are detailed in specific is- ontology makes the Protégé process grow to 150 MB sues, and the measures used for each of these issues of memory. In addition, Web-based OEs, in some are provided. First of all, the general concern relating cases, do not reach the 1 second limit necessary to to the question “How easy is it to create a new on- keep uninterrupted the user’s flow of thought (Niel- tology with current OEs?” is illuminated by two sen 2000), although those OEs do not violate the 10 questions: 1) how easy is it to create a new empty second response time limit that is considered neces- Knowl. Org. 32(2005)No.1 5 E. García-Barriocanal, M. A. Sicilia, and S. Sanchez-Alonso: Usability Evaluation of Ontology Editors

sary for keeping the user’s attention focused on the were structured around concepts that are not fa- dialogue. miliar to UML users, e.g., the fact that properties are a first-class modelling element (Baclawski et 2.2. Participants al. 2001). – Evaluation. This step was divided into two parts, The target population is made up of individuals who one for each general concern. share the following characteristics: more than five – Post-test. After each part of the evaluation, the years of experience in the use of computers, daily use participants responded to a questionnaire aimed at of complex GUI-based applications, and a minimal measuring their subjective satisfaction. understanding of conceptual models (but capable of understanding, at least simple UML class diagrams). In order to evaluate the specific issues detailed in A pre-test phase allowed for the rejection of users section 2.1, scenarios in step two were set up as fol- not fitting this profile, as current OEs are not con- lows: sidered adequate for their use. Note that the ontologies used in the test are designed for usability rather – Scenario 1. The user creates a small ontology than for reusability – in the sense given in Domingue from scratch. For this purpose, a part of the on- and Motta (1999), and therefore, further testing tology described in Fensel et al. (2000) was then would be required for ontologies designed for reus- written in a language-neutral (from the perspec- ability. Moreover, a number of features that can be tive of ontology languages) textual form, and considered as advanced ontology modelling, like ex- sketched as a UML diagram. A total of ten classes, ploiting inference engines or defining axioms and five properties were used. This scenario was through formulas, were disregarded in the analysis. the same for all the evaluations. An informal experiment with three users who were – Scenario 2. After loading a relatively large ontol- not familiar with KR Internet services demonstrated ogy, the user was requested to search a class and a that including those features was simply not realistic. property, to annotate all the relationships of the The experiment consisted of editing axioms with class (along with the entire generalization hierar- OILEd from natural language descriptions. None of chy), and to perform small updates on either the the users was able to complete the task in a reason- class, or the property, or both. Depending on the able time, which suggests that simpler and more in- OE, the following ontologies were used: Cyc- tuitive interface metaphors are required for those Transportation Ontology, World-Fact-Book and tasks to be carried out by people with no back- UNSPSC. ground in description logics or similar formalisms. From the basic user profile, three subgroups were 2.4. Tools and Environment of the Test considered: 1) users with experience in ontology definition; 2) users with experience in computer- The test team was made up of three of the experts based modelling (e.g. users with experience in UML involved in the heuristic evaluation phase. The envi- modelling tools) but with no experience in ontology ronment in which the test was carried out was an definition; and, 3) users with neither experience in isolated room with a personal computer running the computer-based modelling nor ontology definition, Windows 2000 operating system. The user interac- but accustomed to using computer applications. For tion was recorded with screen capture software, the test, four participants from each group were se- while one of the experts on the test team observed lected. In addition, a participant from each of the the participant’s reactions. Each participant evalu- subgroups was selected to perform a pre-test de- ated each of the OEs, but the order of evaluation signed to detect defects in the test process itself. was different for each participant in order to prevent biases derived from remembering previous scenarios. 2.3. Procedure and Scenarios 3. Results The final test involved three steps: 3.1. Test Results – Learning. Participants in subgroups 2 and 3 were given a brief introduction both to general ontol- Figure 1 summarizes the overall results obtained for ogy concepts and to the specifics of every OE un- each OE. The results were obtained by calculating the der evaluation. For subgroup 2, the explanations arithmetic mean of the time, in minutes, that each 6 Knowl. Org. 32(2005)No.1 E. García-Barriocanal, M. A. Sicilia, and S. Sanchez-Alonso: Usability Evaluation of Ontology Editors

Total time scenario 1 Total time scenario 2

KSL 19 KSL 10,5

OILEd 10,1 OILEd 5,5

OntoEdit 11 OntoEdit 5,2

Protègè 9,5 Protègè 5,2

0 5 10 15 20 024681012 minutes minutes

Figure 1. Overall results classified by ontology editor

Figure 2. Time (in seconds) needed to complete each task in each OE and number of errors recorded. group (represented by the arithmetic mean of the The KSL ontology editor exhibited problems in minutes of its members) used to complete the first both its orientation and navigability functions (e.g. and second scenarios, respectively. Figure 2 shows in frames that hid some functions, errors that did not more detail, for each editor, the time (in seconds) provide links to go back, and difficulties for users in spent by the groups in performing a specific task, and knowing what they were editing). These factors the number of errors each group made before the might account for the significantly higher times and task was completed. Both measures represent the error rates, which also increased significantly in the arithmetic mean of the members of the group. third user group recorded for the KSL ontology edi- In some cases, as for example the ‘create class’ tor. Some participants in this third group were not task, the scenario involved several repetitions, and able to complete the tasks in the estimated maximum thus the time reported is the average time needed to time. In addition, the pages of the KSL editor do not complete a task. The measures should be considered fit the common visualization area of a browser, which approximate, since most of the users did not take a results in scrolling and frame resizing, which, in turn, task-by-task approach, but instead explored the in- significantly increases the time-to-complete measure. terface options, performing partial tasks that were These specific problems prevented a fair comparison then completed later. between HTML-based and desktop-based interfaces. Knowl. Org. 32(2005)No.1 7 E. García-Barriocanal, M. A. Sicilia, and S. Sanchez-Alonso: Usability Evaluation of Ontology Editors

The OntoEdit results show that only property- efficiency), but also user satisfaction. A slightly related operations were problematic for users, per- modified version of the System Usability Scale haps because most of them defined properties at a (Brooke 1986) was used in a simple, five-item Likert global level, thus preventing the users from attaching scale (from 1-completely disagree to 5-completely the properties to previously defined elements. This agree) questionnaire. This questionnaire provided a problem then caused user disorientation. Results for global view of subjective assessments of usability, OILEd are of a similar magnitude, but specific prob- summarized in Figure 3. Note that the help system lems arose in navigating the class hierarchy. Protégé was not evaluated, since it was clearly identified as an results are slightly better than those of OntoEdit, area in need of improvement during the heuristic but no significant conclusions can be drawn from analysis phase. them. The metamodel accessibility in both Protégé The global scores clearly show that the KSL editor and KSL is perceived as a drawback that causes er- is perceived as complex and difficult to use. The notes rors and disorientation, because non-specialists do of the evaluators corroborate this fact, as six of the not understand the need for such functionality. An participants complained about KSL, while four of overall analysis reveals that browsing large hierar- them pointed out that the problem was that HTML- chies in any OE environment is a time-consuming based interfaces are, in general, less usable. The high task, and that creating properties is an error-prone score for KSL in question 1 might correspond to the activity. Errors that occur during the creation of fact that most of the users found themselves lost properties might perhaps be due to the duality be- while navigating in the KSL meta-model, because the tween global and local properties. meta-model is accessible through links in the OE. OILEd and OntoEdit obtain similar satisfaction 3.2. Post-Questionnaire Results scores, and are perceived as significantly easier to use than Protégé, with the exception of the responses to In order to understand usability, it is important not questions 4 and 7, which are directly linked to pre- only to measure user performance (effectiveness and dictability. This result points out that some editing

Figure 3. User satisfaction. 8 Knowl. Org. 32(2005)No.1 E. García-Barriocanal, M. A. Sicilia, and S. Sanchez-Alonso: Usability Evaluation of Ontology Editors

capabilities in Protégé that could be considered as ad- by the evaluations, new visualization metaphors (e.g. vanced – such as the explicit edition of the meta- 3-dimensional, filters on the class hierarchy) should model – are perceived as unnecessarily complex for be explored, because discovering the hierarchy of a non-specialists. Another important conclusion is that specific class has been revealed to be a time- no significant differences exist between the three user consuming task. profiles, apart from a slight increase in the perception A more comprehensive evaluation is needed, both of ease of use in the third group. of the number of OEs (including WebOnto and The global satisfaction results for the three desk- WebOde, which posses interesting user interface top editors show that all of them can be considered characteristics) and the depth of the analysis. During reasonably adequate for their purposes. this study, the authors observed (as it was previously supposed) that users usually prefer learning about 3.3. Summary of Major Problems how to use an ontology editor by directly using the tool, instead of by reading the documentation. For To summarize the study, a list of the most relevant this reason, the cognitive walkthrough technique areas for improvement was elaborated: (Polson et al. 1996), which pays special attention to how well the interface supports exploratory learning, – Integrated, context-aware help systems should be could be an interesting candidate for further evalua- developed. tions. – The meta-model should be considered an advanced feature, and, thus, it should be disabled by References default. In addition, the use of a common meta- model terminology across OEs would be benefi- Adelman, S.L., Riedel, L. 1997. Handbook for evalu- cial (e.g. providing a unified name for the concept ating knowledge-based systems: Conceptual frame- of ‘relation between classes,’ because this is cur- work and compendium of methods. Boston, USA: rently referred to as property, relation or slot de- Kluwer Academic Publishers. pending on the OE), in order to hide the differ- Brooke, J. 1986. System usability scale (SUS), User ences between the underlying ontology formal- information Architecture A/D Group, Digital isms as much as possible. Equipment Co. Ltd. Online at: http://www. – The language used in the tools should be oriented usability.serco.com/trump/ towards a non-specialized user community, thus Baclawski, K., Kokar, M.K., Kogut, P.A., Hart, L., avoiding language-specific constructs and terms. Smith, J.E. , Holmes, W.S., Letkowski, J., Aron- – New interaction mechanisms to browse the gen- son, M.L. 2001. Extending UML to support on- eralization/specialization hierarchy should be ex- tology engineering for the Semantic web. In Mar- plored. In this sense, editing should be based on a tin Gogolla and Cris Kobryn (eds.) UML 2001- hierarchically structured view. Moreover, as in The Unified Modeling Language. Modeling Lan- Protégé, hints should be given to recognizing guages, Concepts, and Tools. 4th International Con- multiple inheritance. ference, Toronto, Canada, October 2001, Proceed- – Richer navigation and filtering mechanisms ings. Springer-Verlag LNCS (2185): 342-360. should be developed according to the user task Benjamins, R. Fensel, D. Decker, S. Gómez-Perez, model. For example, users should have the ability A. 1999. (KA)2: Building ontologies for the Inter- to navigate from a class to its instances, or to filter net: A mid-term report. International journal of the visualization of classes by given criteria. human-computer studies 51:687-712. Cranefield, S., and M. Purvis. 1999. UML as an on- 4. Conclusions tology modelling language. In Proceedings of the Sixteenth International Joint Conference on Artifi- The overall conclusion is that current GUI-desktop- cial Intelligence (IJCAI99), Stockholm, Sweden, based OEs are fairly adequate for new users that pre- July 31-August 6, 1999, ed. Thomas L. Dean. San fer exploratory learning. A number of minor usabil- Francisco: Morgan Kaufmann. Available online at: ity errors, which could be easily fixed, have been re- http://hcs.science.uva.nl/usr/richard/workshops/ ported in this paper. In addition, a number of overall ijcai99/UML_Ontology_Modelling.pdf> improvement areas have been identified, which may Cranefield, S., S. Haustein and M. Purvis. 2001. be the topic of future research work. As suggested UML-based ontology modelling for software Knowl. Org. 32(2005)No.1 9 E. García-Barriocanal, M. A. Sicilia, and S. Sanchez-Alonso: Usability Evaluation of Ontology Editors

agents. In Proceedings of the Fifth International ence on Artificial Intelligence. Madison, Wisconsin, Conference on Autonomous Agents, Montreal, Can- USA: American Association on Artificial Intelli- ada, May 28-June 1, 2001, eds. Jörg Paul Müller, gence, pp. 608-614. Elisabeth Andre, Sandip Sen and Claude Frasson. NCITS (National Committee for Information New York: ACM Press. Technology Standards)-TC T2 (Information In- Davis, R., Shrobe, H., Szolovits, P. 1993. What is a terchange and Interpretation). 1998. American na- knowledge representation? AI magazine 14(1): 17- tional standard for Knowledge Interchange For- 33. mat Proposed Draft, NCITS.T2/98-004. http:// Decker, S.D. Fensel, F., Harmelen, I., Horrocks, S., logic.stanford.edu/kif/dpans.html Melnik, M., Klein, J. B. 2000. In Knowledge repre- Nielsen, J. 1992. Finding usability problems through sentation on the Web, Proceedings of the 2000 Inter- heuristic evaluation. In CHI ’92 conference pro- national Workshop on Description Logics. Aachen, ceedings: Striking a balance, Monterey, California, Germany. CEUR Workshop proceedings. http:// eds. Penny Bauersfeld, John Bennett, and Gene CEUR-WS.org/Vol-33/ Lynch. New York: Association for Computing Denny, M. 2004. Ontology Building: A Survey of Machinery; Reading, MA: Addison-Wesley, pp. Editing Tools. In O’Reilly, XML.com. http:// 373-380. www.xml.com/pub/a/2004/07/14/onto.html Nielsen, J. 1994. Heuristic evaluation. In Usability Ding, Y., D. Fensel, Klein, M., Omelayenko, B. 2002. inspection methods. John Wiley & Sons. The semantic web: Yet another hip?. In Data and Nielsen, J. 2000. Designing web usability: The practice knowledge engineering 41: 205-227. of simplicity. Indianapolis, USA: New Riders Domingue, J., Motta, E. 1999. A knowledge-based Press. news server supporting ontology-driven story en- Noy, N.F., Fergerson, R.W., Musen, M.A. 2000. The richment and knowledge retrieval. In Proceedings knowledge model of Protege-2000: Combining in- of the 11th European Workshop on Knowledge Ac- teroperability and flexibility. In Proceedings of the quisition, Modelling, and Management (EKAW 2nd International Conference on Knowledge Engi- ‘99). Lecture notes in artifical intellegence 1621. neering and Knowledge Management (EKAW’ Berlin, Germany: Springer-Verlag. 2000), Juan-les-Pins, France. Duineveld, A. J., Stoter, R., Weiden, M. R., Kenepa, Noy, N.F., Sintek, M., Decker, S., Crubezy, M., B., Benjamins, V. R. 2000. Wondertools? A com- Fergerson, R.W., Musen, M.A. 2001. Creating se- parative study of ontological engineering tools. In mantic web contents with Protege-2000. In IEEE International journal of human-computer studies intelligent systems 16:60-71. 52:1111-1133. Object Management Group. 2003. The Unified Dumas, J. S., Redish, J.C. 1999. A practical guide for Modeling Language specification, version 1.5. usability testing. Sheffield, UK: Intellect, Ltd. http://www.uml.org Fensel, D., Horrocks, I., Van Harmelen, F., Decker, Polson, P.G., Lewis, C., Rieman, J., Wharton, C. S., Erdmann, M., Klein, M. 2000. OIL in a nut- 1992. Cognitive walkthroughs: a method for the- shell. In Proceedings of the European Knowledge ory-based evaluation of user interfaces. In Inter- Acquisition Conference (EKAW-2000). Lecture no- national journal of man-machine studies 36:741- tes in artificial intelligence. Berlin, Germany: 773. Springer-Verlag. Shahar, Y., Chen, H., Stites, D. P., Basso, L., Kaizer, Fensel, D. 2001. Ontologies: Silver bullet for knowl- H., Wilson D. M., Musen, M. A. 1999. Semiauto- edge management and electronic commerce. Berlin, mated entry of clinical temporal-abstraction Germany: Springer-Verlag. knowledge. In Journal of the American Medical In- Gómez-Pérez, A. 1994. Some ideas and examples to formatics Association 6:494-511. evaluate ontologies. Tech. report # KSL-94-65. Tognazzini, B. 2002. First principles and design Knowledge Systems Laboratory. Stanford Univer- http://www.asktog.com/basics/firstPrinciples.html sity. ftp://ftp.ksl.stanford.edu/pub/KSL_Reports/ KSL-94-65.ps.gz McGuinness, D.L., Patel-Schneider, P.F. 1998. Us- ability issues in knowledge representation systems. In Proceedings of the 15th National Confer-

10 Knowl. Org. 32(2005)No.1 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

Hierarchical Navigation: An Exploration of Yahoo! Directories

Hur-Li Lee* and Hope A. Olson**

*/**University of Wisconsin-Milwaukee, School of Information Studies, P.O. Box 413, Milwaukee, Wisconsin 53201-0413, USA, *[email protected] **[email protected]

Hur-Li Lee is an Associate Professor in the School of Information Studies at the University of Wis- consin-Milwaukee. Her major research interests include the organization of information; users’ interactions with structures of information systems; and social and cultural aspects of information and information services. She and Dr. Hope A. Olson are currently principal investigators for a major grant from the Institute of Museum and Library Services (U.S.) to recruit diverse students to specialize in the organization of information.

Hope A. Olson is a Professor in the School of Information Studies at the University of Wisconsin Milwaukee. Dr. Olson’s research addresses classification theory and problems of bias in subject access to information using feminist, poststructural, and postcolonial perspectives. Among her current interests are the cultural specificity of classificatory structure, structures of indigenous knowledge, searcher navigation of hierarchical structures (with Dr. Hur-Li Lee), and the nature of consistency (with Dr. Dietmar Wolfram). She has published: The Power to Name: Locating the Limits of Subject Representation in Libraries (Kluwer Academic, 2002); Information Resources in Women’s Studies and Feminism as editor (K.G. Saur, 2002); and Subject Analysis in Online Catalogs 2nd ed. with John J. Boll (Libraries Unlimited, 2001) and was editor-in-chief of the journal Knowledge Organization from 2000 to 2004.

Lee, Hur-Li, and Hope A. Olson. Hierarchical Navigation: An Exploration of Yahoo! Directories. Knowledge Organization, 32(1). 10-24. 37 refs.

ABSTRACT: Although researchers have theorized the critical importance of classification in the organization of information, the classification approach seems to have given way to the alphabetical subject approach in retrieval tools widely used in libraries, and research on how users utilize classification or classification-like arrangements in information seeking has been scant. To better understand whether searchers consider classificatory structures a viable alternative to information retrieval, this article reports on a study of how 24 library and information science students used Yahoo! directories, a popular search service resem- bling classification, in completing an assigned simple task. Several issues emerged from the students’ reporting of their search process and a comparison between hierarchical navigation and keyword searching: citation order of facets, precision vs. recall, and other factors influencing searchers’ successes and preferences. The latter included search expertise, knowledge of the discipline, and time required to complete the search. Without a definitive conclusion, we suggest a number of directions for further research.

1. Introduction proach in retrieval tools widely used in libraries. Until the nineteenth century, classified catalogs were the Bibliographic classification has long been a standard norm in most libraries. With Charles Cutter’s Rules device for organizing libraries’ print collections on for a Printed Dictionary Catalogue in 1876, the shift the shelves. However, the classified approach seems to alphabetical subject catalogs became entrenched in to have given way to the alphabetical subject ap- the United States. Libraries in other parts of the Knowl. Org. 32(2005)No.1 11 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

world maintain a mixture of classified and alphabeti- A classificatory structure also facilitates naviga- cal subject dictionary catalogs, although the latter are tion up and down a hierarchy. Charles Cutter (1904, prevalent, perhaps because of the globalization of an 79) recognized the shortcomings of verbal systems American model through the distribution of catalog- to allow this type of browsing: ing copy. A few indexing and abstracting databases utilize classifications as do some individual journals. Subject-entries, individual, general, limited, ex- However, the most widespread use of classification- tensive, thrown together without any logical like arrangements for information retrieval, other arrangement, in most absurd proximity – Ab- than material arrangements on library shelves, is scess followed by Absentee-ism and that by Ab- probably in the directory functions of some internet solution, Club-foot next to Clubs, and Com- search engines. Yahoo! (http://www.yahoo.com/) munion to Communism, while Bibliography and began in 1994, using a hierarchical browsing structure Literary history, Christianity and Theology, are exclusively, and still has one of the deepest hierar- separated by half the length of the catalogue – chies. For some time, Yahoo! has not been alone in are a mass of utterly disconnected particles utilizing this type of structure. Many other search without any relation to one another, each use- engines also have hierarchical directories, including ful in itself but only by itself. Google, the most popular search engine in the United States, Yahoo! being second (Sullivan 2004). It makes He also noted that catalog users of his time tended one wonder: Why do successful enterprises like Ya- to search for broader topics than they actually hoo! and Google invest in something akin to the clas- sought (Cutter 1904, 67). While Cutter attributed sified catalogs that American libraries abandoned this tendency to his public’s previous use of classi- over a century ago? fied catalogs, more recent research suggests that it is When online searching first became readily avail- still the case where users have no prior experience able, popular debate pitted verbal controlled vocabu- with classified catalogs (Cousins 1992; Drabenstott lary against natural language (Svenonius 1986). That and Vizine-Goetz 1994; Larson 1991). It seems logi- debate was more or less dissolved by the widespread cal to deduce that classification-like arrangements provision of both options in many applications. continue to be useful. Keyword searching of both natural language and, The question is: Do searchers consider classifica- where available, controlled vocabulary has now evol- tory structures a viable alternative or supplemental ved as a common approach from Internet search en- approach to information retrieval? That is the gen- gines like Yahoo! to library online catalog interfaces eral question that leads to this specific study. More that frequently make keyword searching the default specifically: How do searchers use these directories basic search. in this age of keyword searching? Answers to this The hierarchical navigation enabled by classifica- question can inform our understanding of how peo- tory structures provides a third search mechanism ple utilize classification as an instrument of informa- often viewed as an alternative to keyword approaches tion retrieval; and this knowledge, in turn, can assist to retrieval whether through controlled or natural in future system design. language (see Mai 2004). These two options, hierar- The study reported here is an exploratory study chical navigation and keyword searching, have some- intended to address the following research questions: times been said to represent different paradigms (Hildreth 1995) or even “distinct patterns of human 1. Do searchers use hierarchical directories success- behavior” (Dodd 1996, 281). Hierarchical navigation fully? Are they able to navigate the hierarchies? may use traditional classifications or it may use clas- 2. Do searchers prefer hierarchical navigation or sification-like schemes such as Yahoo! directories or keyword searching? the structure of files within folders within folders on 3. What factors potentially influence searchers’ suc- one’s computer. Not only does a classificatory struc- cess and preferences? ture gather the resources on a particular subject, it 4. What characteristics of classification are germane puts them in logical proximity to resources on close- to searchers’ performance? ly related subjects in the same class. Such structures help to fulfill the collocation function in organizing This initial project analyzes the work of library and information and allow searchers to browse in a way information studies students assigned to compare di- that verbal systems do not. rectory and keyword searching on a prescribed topic 12 Knowl. Org. 32(2005)No.1 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

in Yahoo! The students, in the first level of their number searching was a successful subject searching study of organization of information, are aware of strategy” and made suggestions for its effective im- basic information retrieval concepts, but have not plementation (1986, 150). Unfortunately, today’s yet attained a professional level of expertise. There- OPAC interfaces have not fulfilled the promise of fore, they combine some remaining naïveté in regard searching via classification in spite of IFLA’s invoca- to organization of information with the ability to tion to “display the hierarchical relationship between describe their search processes and perceptions. a classification number and the entire classification” (principle 22, p.47, in its Guideline for OPAC Dis- 2. Literature Review plays, Yee 1998), which implies the efficacy of hierarchical navigation. Christine Borgman and others Jens-Erik Mai (2004, 93-94) notes that most research (1995) tested a DDC-based hierarchical browsing on improving access to Web documents through or- system for children’s use with similar success. The ganization of information focuses on descriptive as- results of these two studies suggest that further ex- pects such as author and title and what little research amination of users’ navigation using hierarchical has been done on the use of classification relates structures is potentially fruitful. primarily to the use of traditional library classifica- More recent research has explored hierarchical tions. He suggests that “the community of biblio- Web directories, but not in relation to navigation or graphic knowledge organization theory and practice searching. David G. Dodd (1997) compared the has not been able to make its knowledge available to main classes in directories with those in library clas- the Web directory community.” Conversely, we have sifications and compared the results of keyword found that little use has been made of the Web as a searches to hierarchical navigation using examples of laboratory for understanding classificatory struc- both known-item searches and topic searches. His tures as retrieval tools. Alan Wheatley (2000, 140) data are not extensive, but do suggest that hierarchi- notes that “subject trees [in Web search engines] cal navigation using directories was more effective have not been examined by performance testing such for topic searches, at least in the late twentieth- as that applied to many other information retrieval century Web. Marthinus S. Van der Walt (1997 and techniques.” In fact, the entire body of research on 1998) looked at the hierarchical structures of Web how people navigate using classification or classifica- directories including specificity and citation order. tion-like schemes is scant. Michèle Hudon (2003) made more in-depth com- The lack of research on navigation using classifica- parisons in the field of education. Van der Walt’s and tory structures may be at least partly attributable to Hudon’s findings show both similarities and differ- the information retrieval research model established ences between the structure of the directories and by the Cranfield tests as discussed by David Ellis and that of library classifications, suggesting that transfer Ana Vasconcelos (2000, 110). Further, the first of findings between the two should be approached Cranfield study, in comparing the application of the with caution. Universal Decimal Classification, an alphabetical sub- Most recently, a study by Said Mirza Pahlevi and ject catalogue, a faceted classification scheme, and Hiroyuki Kitagawa (2005) demonstrated the poten- Uniterms, found that simple indexing systems such tial of hierarchical directories in the retrieval process. as Uniterms can be effective. The second Cranfield Pahlevi and Kitagawa developed and tested a mecha- study affirmed that “natural language, with slight nism that combined natural language search terms modifications of confounding synonyms and word with directory categories, probed the directories for forms, combined with simple coordination, can give relevant material, derived search modifiers from that a reasonable performance” (Cleverdon and Keen material, and then combined the derived modifiers 1966, 263). This emphasis on natural language has with the original natural language, thus creating a re- drawn attention away from controlled vocabularies, fined search that can be run in any search engine. including classification. Their purpose in deriving search modifiers from di- Fortunately, some research in navigating library rectory categories was to develop more precise catalogs through classifications has been conducted searches. Searchers can take advantage of the hierar- during the intervening decades. Karen Markey chical navigation to identify a category, and the re- (1986) explored the use of the Dewey Decimal Clas- sultant modifiers provide a context apart from the sification (DDC) in a large scale experiment using human-indexed categories. Ellis and Vasconcelos catalogs in four major libraries. She found that “class (1999) agree that the context provided by directories Knowl. Org. 32(2005)No.1 13 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

is important in Web searching and eases the need to Google in use and popularity as documented generate search terms. throughout Danny Sullivan’s Search Engine Watch Both the research and non-research literatures en- website. Yahoo!’s directory is the most specific of dorse the use of hierarchical navigation. Nancy Wil- the widely used search engines (Wheatley 2000, liamson (1997, 24) noted that: “classification aids in 137). Hence, it is a reasonable environment in which retrieval through the use of logical structure and to explore the use of hierarchical navigation. helpful order to facilitate browsing and filtering of large quantities of data. It also has the potential to 3. Research Method make possible multilingual access and improved in- teroperability with other services.” The ease of hier- The purpose of this study was to identify approaches archical navigation in library catalogs was borne out taken by students in navigating across Yahoo! direc- by Borgman et al. who discovered that children aban- tory hierarchies and to compare navigation with doned keyword searches more frequently than hierar- keyword searching. Because of its exploratory na- chical ones because of the need to generate and spell ture, the researchers chose to conduct the study with search terms (1995, 682). Bob Ainsbury (2002) sug- a small, convenient sample. The results, therefore, gests the same advantages for corporate portals, as are limited by the nature of the study and its partici- does Martin White (2001) in the context of intranets. pants. Future research will be required to establish The main process that facilitates retrieval using firm conclusions. hierarchical navigation is browsing. As Julian Warner In 2003, one of the researchers asked students in (2000, 37) noted: “The value of an information sys- two online classes to complete an assignment inde- tem could then be the ability it offers discriminat- pendently over a two-week period by using the Ya- ingly to follow ‘paths and tracks, however slight.’ hoo! services. Both classes were sections of a re- Classification schemes themselves … can then be required course, Organization of Information, in the ceived not as fixed models of stable entities but as Master’s in Library and Information Science (MLIS) valuable exploratory devices.” The value of browsing program at the University of Wisconsin-Milwaukee. is recognized through the DESIRE (Development of Some of the students were beginning MLIS students a European Service for Information on Research and and others had taken other MLIS classes in previous Education) project. Traugott Koch, Michael Day, and semesters. According to information about them- others (1997) elaborate on enabling the broadening selves provided by students in both classes, none of and narrowing of topics and identification of context them had had any prior formal training or knowl- in hierarchical browsing. For DESIRE, the potential edge in information organization in general or in for classification to accommodate multilingual access classification theory specifically. Although as a group is also important. Others have also suggested classi- they might be more experienced searchers than aver- fication as a potential switching language in multilin- age users, their experience with the Internet was gual contexts (McIlwaine 2003). overwhelmingly limited to keyword searching; few Lois Mai Chan, Xia Lin, and Marcia Zeng (1999) indicated that they had used Yahoo! directories in a noted that hierarchical browsing “improves precision substantive way. by first defining and narrowing the domain for When they received instructions for the assign- searching.” It does so by collocating (inclusion) and ment, students were given information about the partitioning (exclusion). They elaborated on earlier study explaining to them that participation in the recognition of hierarchical Web directories as preci- study was voluntary, would not affect their grades, sion devices by Mary Micco (1996), Greg R. Notess and did not require any extra work. An additional (1997), Thomas Pack (1999), and others. step taken to ensure confidentiality was to ask those The directory used in this project is from Yahoo! volunteering to participate to send their consent It has been based on literary warrant with categories forms to the other researcher who was not the in- established as sites on new topics were received (Cal- structor of the classes. Students were assured that lery 1996; Steinberg 1996). It is, as Bella Hass Wein- the instructor did not know which students would berg (1999) points out, an alphabetico-classed sys- or would not participate in the study before grading tem. So although it benefits from a hierarchical and that data extraction would begin only after all structure, it does not include a notation and, there- grades were submitted. Out of a total of 44 students, fore, has the limitations of the alphabet within a 26 agreed to participate in the study. Twenty-four of given category. Yahoo! is generally second only to the submissions were used in the study and two were 14 Knowl. Org. 32(2005)No.1 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

excluded because the two students did not follow the from the texts of students’ papers. Categories were assignment instructions. When the texts were ex- not stipulated in the assignment (see appendix). tracted for data analysis, all students’ names were re- Each researcher independently coded a subset of as- placed by numbers that were assigned in no particu- signments, developing categories on the basis of the lar order. data. These categories were then merged and differ- The assigned task was to find five national or in- ences of interpretation were resolved. The research- ternational scholarly or professional organizations in ers then coded all of the data, recording the prelimi- the field of microbiology by conducting two types of nary subsets. searches in Yahoo!: navigating across directories and searching by keyword (the text of the assignment is 4. Results appended to this article). Due to the fact that these students were beginners in classification theory and 4.1. Hierarchical Paths Taken that they came from varied backgrounds, the topic assigned was intentionally straightforward, not re- As indicated in Table 1, 21 of the 24 students par- quiring any subject knowledge or advanced skill in ticipating in the study took the same path in navi- using classification. To simplify grading, students gating the Yahoo! directories: “Science” to “Biol- were asked to use “Science” as the starting category. ogy” to “Microbiology” to “Organizations” (Path The assignment instructions clearly specified that A). Out of these 21 students, 15 did not attempt navigation using the Yahoo! directories should be any other paths. However, it cannot be concluded completed first because it would be more interesting that the majority of the students saw this as the to see students’ own paths in this kind of search. The only logical path, because once they successfully hierarchical paths along the directories indicated by completed the task required in the assignment Yahoo! in keyword search results might influence there was no reason for them to explore other op- students’ judgment in navigating the hierarchies and tions. On the other hand, three others tried more thus the keyword search had to be conducted after paths, mostly other categories at one particular the navigational search. Although the results of na- level of the hierarchy, after they examined the re- vigation might influence terms chosen in the key- sults at the end of Path A; of these, 2 simply word search, this influence was considered inconse- wanted to see more and 1 gave a specific interpreta- quential in this instance because selection of key- tion of the task and determined that the sites listed words was not a central concern in either the as- at the end of Path A were not completely satisfac- signment or the study. Students were then asked to tory. The other 3 students exploring alternatives to write a two-page comparison of the two distinct Path A considered Path A to be one of the options searches. Those comparative narratives provided the that were all equally logical. Of the remaining 3 data for this study. students who did not take Path A, 1 took Path H, 1 Data analysis was of a qualitative nature. Catego- took Path I, and the last did not indicate the path ries were developed by the researchers as derived taken.

Number of No. of Students Who Took Path Taken Students This Path Exclusively A. Science→Biology→Microbiology→Organizations 21 15 B. Science→Biology→Microbiology→Web Directories 1 0 C. Science→Biology→Parasitology→Organizations 1 0 D. Science→Biology→Organizations 1 0 E. Science→Biology→Organizations→Microbiology 3 0 F. Science→Biology→Organizations→Professional 1 0 G. Science→Medicine 1 0 H. Science→Medicine→Microbiology and virology→Organizations 1 1 I. Science→Organizations→Biology→Microbiology 3 1 J. Science→Research 1 0 Not indicated 1 N/A

Table 1. Hierarchical paths taken by the students. Knowl. Org. 32(2005)No.1 15 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

Keyword Number of Students Number of Keywords Number of Students Microbiology 24 2 keywords 12 Organization(s) 23 3 keywords 11 Professional 8 5 keywords 1 Association(s) 4 Table 3. Number of keywords used National 1 Scholarly 1 Society 1

Table 2. Keywords used by the students.

Search Techniques Number of Students Boolean 3 Quotation marks 3 Advanced search 1 Not indicated 19

Table 4. Keyword search techniques applied

Advantage # of Disadvantage # of Students Students Quick, easy, and not time-consuming 13 Requires more steps 3 High precision 11 Low recall 11 Showing relationships between topics 8 Limited to the structure developed by Yahoo 3 Useful for experts or someone familiar with the 3 Requires knowledge of a particular subject hi- 8 topic erarchy No need to generate search terms 2 Subjectivity in categories and assignment of 1 sites to categories No need to know advanced search techniques 1 Requires knowledge of hierarchical search 4 Updated slowly 1

Table 5. Advantages and disadvantages of hierarchical navigation.

4.2. Keywords and Search Techniques Applied Among the 24 participants, only 5 consciously applied one or more complex techniques in the key- Table 2 lists the words used by the participating stu- word search (Table 4). Judging from other students’ dents in the keyword search. All students but one search results (typically in tens or hundreds of thou- entered either “microbiology organization” or “mi- sands), it was clear that they simply typed in the crobiology organizations.” The last student typed in chosen keywords as one character string and clicked “microbiology associations,” instead. Three students the search button. The two most frequently applied tried both “organization(s)” and “association(s)”; advanced search techniques were Boolean (“and” and another student used both “organization” and “soci- “or”) and quotation marks to search as a phrase. ety.” The data did not indicate why few of the stu- Only one student took the step to go to the ad- dents considered synonyms. Three of the keywords vanced search page. (national, scholarly, and society) came up only once each. According to Table 3, one half of the students 4.3. Advantages and Disadvantages of Hierarchical applied only 2 keywords, 11 others used 3 keywords, Navigation and only 1 student typed in 5 keywords. Among the 3-keyword searchers, 7 used “microbiology,” “or- In comparing the two types of searches, the students ganization(s),” and “professional”. The person who offered 6 advantages of hierarchical navigation (Table used 5 keywords included the string “national schol- 5). First, possibly due to the fact that the assigned arly professional organizations microbiology”. topic was simple and straightforward, 13 students in- 16 Knowl. Org. 32(2005)No.1 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

dicated that navigating through the Yahoo! directo- that sites were assigned to individual categories. ries was quick, easy, and not time-consuming. Some Three students complained that it took many more of them clearly understood that other more complex steps to go through the categories as compared to a topics might require lengthy exploration of numer- keyword search that took only one step: entering ous unfamiliar categories. Eleven students mentioned keywords. None of these students noted that in addi- the high precision rate in the search results: with the tion to steps taken in conducting the search, steps exception of one broken link, all sites retrieved in this taken to evaluate retrieved sites should also be con- search were relevant. To 8 of the students, Yahoo! di- sidered. The last disadvantage, according to 1 student, rectories presented hierarchical relationships between was the slowness of Yahoo! in updating its directories topics and thus could assist users in identifying so that new sites would most likely be excluded. broader or narrower topics that were potentially more appropriate. Three students believed that this 4.4. Advantages and Disadvantages of Keyword type of search was useful for people who were able to Searching follow easily the hierarchical path because of their familiarity with the general area of the search topic. As shown in Table 6, high recall rate in search results On the other hand, 2 other students thought that this was the number one advantage of keyword searching type of search was useful for novices if they were un- mentioned by half of the students (12). They drew able to come up with good keywords or were unfa- this conclusion because they identified many more miliar with advanced keyword search techniques. relevant organizations through keyword searching The disadvantages of hierarchical navigation are than through hierarchical navigation. A second fac- listed in Table 5. Nearly half (11) of the students no- tor seen as an advantage by 9 students was access to ticed the low recall rate in search results. In other other directories, searching aids, and pathfinders. In words, this type of search did not retrieve many of addition, 4 other advantages of keyword searching the relevant sites they found using the keyword were listed by 1 or 2 students: user-friendly, more search. Eight students thought that this type of up-to-date, no need to have subject knowledge, and search would require the searcher to have a certain more useful for an experienced searcher or some one level of knowledge about the hierarchical structure in with subject knowledge. The last one was quite in- the subject area and 4 students believed that it would teresting. The student argued, “through key word require a basic understanding of the concept of a hi- searching one often gets many false drops, but if the erarchical arrangement. Their view was that the aver- searcher is an experienced searcher or has subject ex- age person would have difficulties with hierarchical perience, he or she may know subject terms to aid in navigation. Interestingly, 3 students said that the the search thus eliminating many non-relevant terms structure of categories was developed by Yahoo! and that may result in false drops.” that users were limited by this home-grown struc- A majority of the students (16) saw its time- ture. Another student also expressed a concern over consuming nature as a significant disadvantage of the subjectivity in Yahoo’s categories and in the way keyword searching (Table 6). The second most

Advantage # of Disadvantage # of Students Students High recall 12 Low precision 10 As access to others’ directories, searching aids, 9 or path finders Time-saving 2 Time-consuming 16 User-friendly 2 More up-to-date 1 Best if the user had little subject knowledge 1 Need to know useful keywords 6 Good for experienced user 1 Need to know advanced search techniques to 5 be effective Possibly missing relevant sites if they are not 2 on first few pages

Table 6. Advantages and disadvantages of keyword search. Knowl. Org. 32(2005)No.1 17 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

common complaint about keyword searching given chies and categories in searches. These issues in- by those students (10) was the overwhelming num- cluded (1) the hierarchical relationship; (2) the prin- ber of sites retrieved, a large percentage of which ciple of specificity; and (3) citation order. were irrelevant (i.e., low precision rate). Six students First, the ability to understand a hierarchical rela- pointed out that users needed to come up with use- tionship depends on the knowledge of the concepts ful keywords on their own and not many users involved in the relationship. It is difficult for a per- would be able to do so. In addition, 5 students son to navigate this relationship when he or she has thought that a keyword search would only be effec- inadequate knowledge of the concepts. A useful ex- tive if the user knew how to apply advanced search ample is the relationship between parasitology and techniques to reduce the size of the retrieved results. microbiology. McGraw-Hill Encyclopedia of Science Lastly, 2 students mentioned the possibility of miss- & Technology (2003) defines parasitology as “a ing relevant sites that did not show up on the first branch of biology which deals with those organisms, few screens because few users would go beyond that plant or animal, which have become dependent on point. other living creatures” and microbiology as “the science and study of microorganisms, including proto- 4.5. Preference Between The Two Methods zoans, algae, fungi, bacteria, viruses, and rickettsiae.” The organisms studied in parasitology may or may Table 7 shows that in completing the assignment, 6 not be microorganisms and the microorganisms students preferred hierarchical navigation, 3 pre- studied in microbiology may or may not be depend- ferred keyword searching, 8 indicated that it would ent on other living creatures. The relationship be- depend on a user’s need or situation, and 7 did not tween the two fields of study can be expressed as a give any preference. Venn diagram (Figure 1).

Preference Hierarchical navigation 6 Keyword search 3 Depending on needs/situations 8 Not indicated 7

Table 7. Students’ preferences regarding search methods applied.

5. Discussion

The data collected in this study raise four main factors in comparing hierarchical navigation and keyword searching: the hierarchical structure itself, in- Figure 1. Illustrated relationship between parasitology and cluding issues of specificity and citation order; the microbiology. influence of these two approaches on precision and recall; time consumed in retrieval; and knowledge The circle on the left represents the field of parasitol- and skill required for retrieval. ogy and the circle on the right microbiology. Area 2, the overlapping area between the two circles, repre- 5.1. Hierarchies and Categories sents the branch of science that studies microorganisms dependent on other living creatures. Therefore, Possibly because the assignment was intended to be navigating through Path C, Science-Biology-Para- uncomplicated, most of the participants had little sitology-Organizations, in Yahoo! would lead to or- difficulty navigating Yahoo! directories in complet- ganizations dealing with not only studies within the ing the required task. The majority of them (21 out scope of microbiology (i.e., Area 2) but also those of 24) followed the hierarchical Path A to find the outside microbiology (i.e., Area 1). In other words, desired results. Nevertheless, the descriptions of the the student who followed Path C made a mistake in paths attempted and the students’ reflections on this choosing the subcategory “Parasitology”. type of hierarchical navigation provide possible in- A similar mistake was following Path H, Science- sight into general issues related to the use of hierar- Medicine-Microbiology and virology-Organizations. 18 Knowl. Org. 32(2005)No.1 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

Again two overlapping circles may be drawn, one 5.2. Precision vs. Recall representing medicine and the other microbiology. Each circle has an area that is outside the other cir- When the two search methods were compared, navi- cle. When the student took Path H, the only organi- gating through Yahoo! directories yielded high preci- zations found at the end of the path were those deal- sion but low recall and keyword searching resulted in ing with microbiology and virology as a subfield of low precision but high recall. Applying the former medicine (the overlap of the two circles). Although method, Path A, for example, led to 10 sites with this time no irrelevant items were retrieved, the re- only one of them being a broken link; the other 9 sults were limited, leaving out many relevant organi- sites all met the search criteria. However, many sites zations that dealt with microbiology outside medi- of other relevant organizations in microbiology were cine. missed because they had not been added by Yahoo’s The second issue pertains to the level of specific- human organizers. On the other hand, more than ity – a chief principle in subject analysis at least since 140,000 sites were retrieved as a result of a search by Cutter’s 1876 codification. The case mentioned in the keywords “microbiology organizations.” Many the above paragraph was one example of being too more relevant sites were included in the keyword specific. As a result, the searcher was only able to search result set, but a majority of the sites retrieved identify organizations in a sub-area of microbiology. were irrelevant. Another student had a difficult time finding any A particular feature in Yahoo! directories contrib- relevant sites at first because she did not reach the uting to a higher precision rate was the embedded most specific level, the form subdivision “Organiza- hierarchical structure. In navigating the structure, tions,” in following Path A. The student went from the user was led from a broad class to its narrower “Science” to “Biology” to “Microbiology” and exam- concepts to achieve a desired level of specificity. The ined all sites under “Microbiology” before discover- labor-intensive process of establishing hierarchical ing that “Organizations” was a subcategory under relationships among concepts in the context of each “Microbiology.” Conversely, a third student at- discipline (or area of interest) was meant to gather tempted 4 other less specific paths (D, F, G, and J) together only relevant sites under each concept, thus because she was dissatisfied with the sites listed at maximizing the precision rate. This advantage of di- the end of Path A. None of the 4 less specific paths rectory navigation was articulated by one third of the turned out to be helpful. participating students (8). “Organizations” was a problematic concept in An interesting question to ask is: “Is precision or terms of its placement in the hierarchy. The conven- recall a preference in information seeking?” Twenty tional citation order of facets usually places a general out of the 24 participants in the study mentioned ei- form facet (such as “Organizations”) after topical ther precision or recall in their discussions of the ones. In both Paths E and I taken by the students in two searches. This high percentage was not surpris- the study, “Organizations” came between topical ing because the participants were LIS students who facets. Two students thought that the 3 paths were had already had at least some exposure to theories of equally valid: information retrieval. As shown in Table 8, among the 11 students who cited high precision as one of A. Science→Biology→Microbiology→Organizations the advantages of hierarchical navigation, only 3 E. Science→Biology→Organizations→Microbiology clearly preferred this search method, 1 preferred I. Science→Organizations→Biology→Microbiology keyword searching, 4 stated that it depended on the user’s need or situation, and 3 mentioned no prefer- Another student took only Path I and yet another ence. Among the 11 students who mentioned low chose both Paths A and E but not I. Although these recall as a disadvantage of hierarchical navigation, 1 students were not the majority in the group, their preferred this type of search, 1 preferred keyword approach seemed to suggest that some users would searching, and 9 either said that it depended on the probably behave similarly in navigating hierarchi- user’s need or did not indicate their preference. cally. It may be plausible to think that the average Twelve students cited high recall as an advantage of user, lacking any training in information organiza- keyword searching, of whom 2 preferred hierarchical tion, will more likely choose a form facet before a navigation, 2 preferred keyword searching, and 8 ei- topical one than would LIS students. ther said that it depended on the user’s need or did not indicate a preference. The fact that keyword Knowl. Org. 32(2005)No.1 19 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

Preferred Preferred Depending on Preference Advantage or Disadvantage Hierarchy Keyword Need Not Stated Hierarchical navigation High precision (n= 11) 3 1 4 3 Low recall (n=11) 1 1 5 4 Keyword search High recall (n=12) 2 2 5 3 Low precision (n=10) 1 0 4 5

Table 8. Preferences of search methods. searching resulted in a low precision rate was men- Number of The “Time” Factor tioned as a disadvantage of keyword searching by 10 Students students, of whom 1 preferred hierarchical naviga- Mentioned time 20 tion, none preferred keyword searching, and 9 either Hierarchical navigation was quick, 13 said that it depended on the user’s need or did not easy, and not time-consuming. indicate a preference. Further, among the 3 students Hierarchical navigation was more 3 who preferred keyword searching, 2 mentioned high time-consuming. recall as an advantage of keyword searching but none Keyword search was time-saving. 2 saw low precision as a disadvantage of this type of Keyword search was more time- 16 search. It seemed that neither precision nor recall consuming. appeared to be an overriding concern of the partici- Table 9. The “time” factor mentioned by the students. pants despite their awareness of precision and recall issues. Among the 6 students who preferred hierarchical navigation, 5 mentioned its time-saving advantage 5.3. The “Time” Factor (as compared to only 3 who mentioned its high precision) and 5 said that keyword searching was time- A clear majority of the students (20) mentioned consuming (as compared to only 1 who said that “time” (see Table 9). In comparing the two search keyword searching resulted in a low precision rate). methods, 13 students thought that hierarchical navi- Two out of the 3 who preferred keyword searching gation was quick, easy, and not time-consuming. cited time-saving as one of its advantages and one of Some of them, rightly, pointed out that the task in- those two students also said that hierarchical naviga- volved in this assignment required only a few easy tion took more steps and was, thus, more time- steps and many real-life inquiries might lead to more consuming. It appears that among those who indi- complicated and prolonged exploration of numerous cated their preferences between the two types of paths in each search; in other words, this type of searching, “time” may have been a more significant search could be time-consuming. Three other par- factor than “precision/recall.” ticipants, however, said that following Yahoo! direc- The idea of what is time-saving was somewhat tories in this assignment took more steps and, thus, ambiguous. One of the 3 students who said that hi- was more time-consuming. As for keyword search- erarchical navigation took more steps also acknowl- ing, 2 students said that being time-saving was one edged the fact that keyword searching in this case of its advantages; but 16 others said that it was more was, overall, more time-consuming because more time-consuming. The keyword search was time- time was spent weeding out many irrelevant items. consuming, according to the students, because it re- After reviewing these three students’ descriptions, it sulted in too many relevant as well as irrelevant hits became clear that they were only concerned with the and too much repetition with many sites listed nu- steps taken in searching. In other words, hierarchical merous times. In addition, the large number of re- navigation took more steps and was more time- trieved sites and the low precision rate made it nec- consuming in the searching stage; but keyword sear- essary to take more care in examining the appropri- ching required more time spent on evaluating the ateness of each site. appropriateness of the retrieved sites individually, many of which did not meet the search criteria. In this way, the precision and recall of search results were clearly linked to time taken. 20 Knowl. Org. 32(2005)No.1 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

5.4. The “Knowledge/Skill” Factor useful keywords or advanced search techniques or both to conduct an effective keyword search. Since The degree of difficulty, as seen by the participating the assignment was uncomplicated, the students’ students, seemed to be associated with how much perception of the difficulties involved might not re- knowledge and/or skill the user brought to each flect the complexities in many real-life searches. search (Table 10). On the one hand, subject knowl- Of the 24 students, 17 discussed subject knowledge was a particular concern: 3 students thought hi- edge and search skills. Among the 6 who preferred erarchical navigation was a useful search for subject hierarchical navigation, 1 considered it easier for ex- experts; 8 students said that subject knowledge perts because it would require both subject knowledge and more experience in searching; another men-

Number of tioned the need for subject knowledge; the third The “Knowledge/Skill” Factor Students only listed disadvantages of keyword searching in terms of the need for both advanced search tech- Hierarchical navigation was useful for 3 subject experts. niques and useful keywords; and the rest did not Subject knowledge was necessary for mention this issue. Among the 3 students who pre- 8 successful hierarchical search. ferred keyword searching, only 1 mentioned the Keyword search was best if the searcher need to know useful keywords; the other two did 1 was unfamiliar with the subject. not touch on the issue of subject knowledge or Hierarchical navigation required no search skills. knowledge of useful terms or complex 2 As shown in Table 11, 8 of the 24 students search techniques. thought that the need for subject knowledge was one Users would need to know hierarchical of the disadvantages of hierarchical navigation. 4 arrangements in navigating directories. Among the 8, 2 preferred hierarchical navigation Keyword search was user-friendly. 2 (neither talked about the difficulties in keyword Keyword search required knowledge searching), none preferred keyword searching, and 6 of useful terms and complex search 9 either said that it depended on the user’s need or did techniques not indicate a preference. Four students thought that Table 10. The “knowledge/skill” factor mentioned by the the need to understand hierarchical arrangements students. was one of the disadvantages of hierarchical navigation. Among them, 1 preferred hierarchical naviga- was necessary for conducting a successful hierarchi- tion, another preferred keyword searching, and 2 ei- cal search; and 1 student maintained that keyword ther said that it depended on the user’s need or did searching was the best if the user knew little about not indicate a preference. Six of the 24 participants the subject. Search skills possessed by the user were thought that the need to know useful keywords was also mentioned as a significant factor: 2 students in- a disadvantage of keyword searching. Among the 6, 1 dicated that the user did not need to generate search preferred hierarchical navigation, 1 preferred key- terms or know complex search techniques in navigat- word searching, and 4 either said that it depended on ing the directories; 4 others said that it was necessary the user’s need or did not indicate a preference. Five for the user to understand the concept of hierarchi- students thought that the need to know advanced cal arrangements in order to conduct such a search; 2 search techniques was a disadvantage of keyword stated keyword searching to be user-friendly; and, 9 searching. Among the 5, 1 preferred hierarchical students said that the user needed to know either navigation, none did keyword search, and 4 either

Preferred Preferred Depending Preference Disadvantage Hierarchy Keyword on Need Not Stated

Hierarchical Need subject knowledge (n=8) 2 0 3 3 navigation Need to know hierarchical arrangements (n=4) 1 1 1 1 Need to know useful keywords (n=6) 1 1 2 2 Keyword Need to know advanced search techniques search 1 0 2 2 (n=5)

Table 11. The “knowledge/skill” factor and student preferences Knowl. Org. 32(2005)No.1 21 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

said that it depended on the user’s need or did not searcher must click on “more” search options to find indicate a preference. This pattern did not give the a set of icons that includes “directories” represented impression that knowledge/skills played a significant rather cryptically by a graphic of an open book). Li- role in the students’ preferences even though they brary catalogs and indexing and abstracting databases considered it a factor worth noting. often make classification searching equally obscure, typically, under “call number” search or “numeric” 6. Conclusion search in a drop-down menu if available at all. (Of more use in library catalogs is the option to click on What answers might emerge from this exploratory a call number in an individual bibliographic record study? Is further research merited and, if so, what di- and retrieve a list of other items at that call number. rections might it fruitfully take? Sometimes, the searcher is dropped into an online The first question we asked is: Do searchers use shelflist through which they can scroll. However, hierarchical directories successfully? The students in this approach still does not give the browsability of this study were generally effective in navigating the lists of categories arranged hierarchically.) If options hierarchy of the directories. In this instance, it was a for hierarchical navigation were made more readily hierarchy likely to be familiar to many users. The hi- accessible, would searchers prefer them, at least in erarchy follows the conventional notion common in certain contexts, to keyword searching? Might users bibliographic classifications of classification by dis- employ hybridized techniques that draw on hierar- cipline with form as the last facet. Whether or not chical navigation to assist keyword searching and this capability would be present in other groups of vice versa? These are questions worth further inves- searchers is open to question. Yahoo! does not re- tigation through system and interface development. quire searchers to use traditional citation order, al- What factors potentially influence searchers’ suc- though that is the citation order displayed at the cess and preferences? The results of this study in re- head of the results screen. Searchers can find the lation to such factors are complex. LIS students were same results by using any progression that comes to able to express their perceptions in terms such as mind as long as they select the same categories. So precision and recall and considered issues such as “Science” to “Organizations” to “Biology” to “Mi- search expertise, knowledge of the discipline, and crobiology” will retrieve the same screen as “Sci- user-friendliness. However, no consensus was ence” to “Biology” to “Microbiology” to “Organiza- reached in terms of how these factors would or tions.” Since the former, unconventional citation or- should influence the decision to use hierarchical der was used by three students in this study, it sug- navigation or keyword searching. They did generally gests that exploration of alternative citation orders agree that the hierarchical navigation produced for searching might be worthwhile in other contexts. higher precision and keyword searching produced Bibliographic classification has limited the flexibility higher recall. They did not agree on which approach of citation order, presumably because of its use in is more user-friendly, more or less time-saving or determining the physical location of items. However, time-consuming, or more appropriate for experts or making individual facets searchable in some manner novices. In discussing these factors, the students (perhaps a PRECIS-like arrangement) might be considered their own experience with the two worthwhile if further research suggests that conven- searches and also tried to anticipate the experiences tional citation order is not intuitive. of other searchers. Sorting out these various factors Do searchers prefer hierarchical navigation or will require further research, probably of a qualita- keyword searching? LIS students, schooled in the tive nature, with data collection more targeted to- need to listen to users, leaned toward the answer: it ward the variables raised here than this pedagogical depends. However, few preferred keyword searching exercise allowed. categorically while nearly as many preferred hierar- What characteristics of classification are germane chical navigation as suggested that “it depends.” to searchers’ performance? Related to the question While the numbers in this study are insufficient to of the advantages and disadvantages of hierarchical draw conclusions, they do suggest that further re- navigation are two interesting implications regarding search is merited. Quite a few students were new to the nature of classification. First is the advantage of the Yahoo! directories, which are placed well down showing relationships between topics. While rela- the busy Yahoo! homepage. Google has gone even tionships can also be shown in an alphabetically ar- further to take directories off of the homepage (a ranged controlled vocabulary such as a thesaurus 22 Knowl. Org. 32(2005)No.1 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

(not available in Yahoo!), hierarchical or classifica- Callery, Anne. 1996. Yahoo! Cataloging the web. tion-like structures make relationships more visible, Untangling the Web. Available at: http://www. arranging related topics in proximity to each other library.ucsb.edu/untangle/callery.html Accessed: for browsing. The second implication for classifica- October 6, 2004. tion is that several students referred to the structure Chan, Lois Mai, Xia Lin, and Marcia Zeng. 1999. of the hierarchy as a limitation, requiring knowledge Structural and multilingual approaches to subject of the particular subject hierarchy of hierarchical access. 65th IFLA Council and General Conference, searching and limiting what might be searched. This Bangkok, Thailand, August 20-August 28, 1999. limitation needs testing, perhaps by comparing mul- Available at: http://www.ifla.org/IV/ifla65/papers/ tiple conventional hierarchies with each other or 012-117e.htm Accessed October 8, 2004. with more creative structures such as might allow Cleverdon, Cyril W. 1962. Report on the testing and searchers to determine citation order (e.g. elaborated analysis of an investigation into the comparative ef- “sort by” options). ficiency of indexing systems. Cranfield, Eng.: Col- In sum, the evidence provided by this study closes lege of Aeronautics. no doors to future research. It points to hierarchical Cleverdon, Cyril, and Michael Keen. 1966. Factors navigation as a viable alternative to the ubiquitous determining the performance of indexing systems. keyword searching. It calls for clarification of the Vol. 2: test results. Cranfieldfield, Eng.: College of roles of expertise and time in hierarchical navigation. Aeronautics. It leaves us with an “it depends” conclusion about Cousins, Shirley Anne. 1992. In their own words: the desirable balance between hierarchical navigation An examination of catalogue users’ subject que- and keyword searching, with the preference for pre- ries. Journal of information science 18: 329-41. cision or recall as one continuing factor affecting Cutter, Charles A. 1904. Rules for a dictionary cata- that balance. Two types of research are suggested by log. 4th ed. Washington: Government Printing these conclusions: continuing studies, both quantita- Office. tive and qualitative, of users’ performance and pref- Dodd, David G. 1996. Grass-roots cataloging and erences in hierarchical navigation; and, the develop- classification: Food for thought from World Wide ment and testing of experimental systems and inter- Web subject-oriented hierarchical lists. Library re- faces that offer easier and more effective hierarchical sources & technical services 40: 275-86. navigation than is currently available. The former Drabenstott, Karen Markey, and Diane Vizine- should include more complex user-defined search Goetz. 1994. Using subject headings for online re- topics, other types of classified databases, larger trieval: Theory, practice, and potential. San Diego: samples, different categories of users, and different Academic Press. cultural contexts so as to move beyond the limita- Ellis, David, and Ana Vasconcelos. 1999. Rangana- tions of the present study. Studies of users’ prefer- than and the net: Using facet analysis to search ences in existing systems can inform development and organise the World Wide Web. Aslib proceed- and testing of experimental systems and interfaces. ings 51: 3-10. Only with both research streams can we answer the Ellis, David, and Ana Vasconcelos. 2000. The rele- question: Is classificatory structure a powerful tool vance of facet analysis for World Wide Web sub- for the 21st century? ject organization and searching. Journal of internet cataloging 2n3/4: 97-114. References Hildreth, Charles R. 1995. Online catalog design models: Are we moving in the right direction? Ainsbury, Bob. 2002. Cataloging’s comeback: Classi- Part 5. Browsing and exploring: A new paradigm fying and organizing corporate documents. for IR/OPAC system design. Available at: http:// Online (May/April): 27-31. www.ou.edu/faculty/H/Charles.R.Hildreth/ Borgman, Christine, Sandra G. Hirsch, Virginia A. clr-five.html Accessed January 30, 2005. Walter, and Andrea L. Gallagher. 1995. Children’s Hudon, Michèle. 2003. Expanding audiences for searching behavior on browsing and keyword education-related information and resources: catalogs: The Science Library Catalog Project. Classificatory structures on the World Wide Web. Journal of the American Society for Information Cataloging & classification quarterly 37n1/2: 81-96. Science 46: 663-84. Knowl. Org. 32(2005)No.1 23 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

Koch, Traugott, Michael Day, et al. 1997. Specifica- Van de Walt, Marthinus S. 1997. The role of classifi- tion for resource description methods. Part 3. The cation in information retrieval on the Internet: role of classification schemes in Internet resource some aspects of browsing lists in search engines. description and discovery. http://www.ub.lu.se/ In Knowledge organization for information re- desire/radar/reports/D3.2.3/class_v.10.html Janu- trieval: Proceedings of the Sixth International Study ary 30, 2005. Conference on Classification Research. The Hague: Larson, Ray R. 1991. The decline of subject search- International Federation for Information and ing: Long-term trends and patterns of index use in Documentation (FID), pp. 32-35. an online catalog. Journal of the American Society Van der Walt, Martin. 1998. The structure of classifi- for Information Science 42: 197-215. cation schemes used in Internet search engines. In Mai, Jens-Erik. 2004. Classification of the web: W. Mustafa el Hadi, J. Maniez and S.A. Pollitt, Challenges and inquiries. Knowledge organization eds. Structures and relations in knowledge organiza- 31: 92-97. tion: Proceedings of the Fifth International ISKO Markey, Karen. 1986. Class number searching in an Conference, 25-29 August 1998, Lille, France. experimental online catalog. International classifi- Würzburg, Germany: Ergon Verlag, pp. 379-87. cation no. 3: 142-50. Warner, Julian. 2000. Can classification yield an McGraw-Hill Encyclopedia of Science & Technology evaluative principle for information retrieval? In Online. 6th ed. 2003. New York: McGraw-Hill. R. Marcella and Arthur Maltby, eds. The future of McIlwaine, I.C. 2003. The UDC and the World Wide classification. Aldershot, England: Gower, pp. 33- Web. In I.C. McIlwaine, ed. Subject retrieval in a 41. networked environment: Proceedings of the IFLA Weinberg, Bella Hass. 1999. Improved internet ac- Satellite Meeting Held in Dublin, OH 14-16 Au- cess: Guidance from research on indexing and gust 2001. München: K.G. Saur, pp. 170-76. classification. Bulletin of the American Society for Micco, Mary. 1996. Subject authority control in the Information Science 25n2:. 26-29. world of the internet. Part two. Libres 6n3. Avail- Wheatley, Alan. 2000. Subject trees on the internet: able at: http://libres.curtin.edu.au/libre6n3/ A new role for bibliographic classification? Jour- micco_2.htm Accessed January 17, 2005. nal of Internet cataloging 2n3/4: 115-41. Notess, Greg R. 1997. WEB sites: Directories. Data- White, Martin. 2001. Architecture, search, integra- base magazine 20n1: 61-64. tion: Classification is the common denominator. Pack, Thomas. 1995. WEB search engines: EContent (September): 54-55. LOOKSMART, EBLAST, YAHOO! Inc., SNAP. Williamson, Nancy J. 1997. Knowledge structures Database magazine 22n1: 51-54. and the Internet. In Knowledge organization for Said, Mirza Pahlevi, and Hiroyuki Kitagawa. 2005. information retrieval: Proceedings of the Sixth In- Conveying taxonomy context for topic-focussed ternational Study Conference on Classification Re- web search. Journal of the American Society for In- search. The Hague: International Federation for formation Science & Technology 56: 173-88. Information and Documentation (FID), pp. 23- Steinberg, Steve G. 1996. Seek and ye shall find 27. (maybe). Wired, issue 4.05. Available at: http:// Yee, Martha M. 1998. Guidelines for OPAC displays. www.wired.com/wired/archive/4.05/ Prepared for the IFLA Task Force on Guidelines indexweb_pr.html Accessed January 20, 2005. for OPAC Displays. Available at http:// Sullivan, Danny. Search engine watch. Available at: www.ifla.org/ifla/VII/s13/guide/opac-d.pdf Ac- http://searchenginewatch.com/ Accessed January cessed March 29, 2005. 30, 2005. Sullivan, Danny. July 14, 2004. Nielsen NetRatings Appendix: Assignment Description Search Engine Rating. http://searchenginewatch. com/reports/article.php/2156451 Accessed Au- A user asks you to help her find five national or in- gust 5, 2004. ternational scholarly/professional organizations in Svenonius, Elaine. 1986. Unanswered questions in the field of microbiology. You decide to search on the design of controlled vocabularies. Journal of the Internet by using Yahoo! (Do not use any other the American Society for Information Science 37: search engines) in two different ways: one by navi- 331-40. gating through the categories (Yahoo directories) and the other by typing in the keywords (in this 24 Knowl. Org. 32(2005)No.1 H.-L. Lee, H. A. Olson: Hierarchical Navigation: An Exploration of Yahoo! Directories

order, please). Do not mix these two methods in each search. Compare and contrast the two types of searches (classification vs. subject terms). Write a two-page (double-spaced) summary of your findings, including a discussion of the merits and problems in each. A hint: Start your category search with “Science.”

Knowl. Org. 32(2005)No.1 25 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Building Taxonomies Using Organizational Resources: A Case of Business Consulting Environment

Abdus Sattar Chaudhry and Goh Hui Ling

Nanyang Technological University, Division of Information Studies, School of Communication & Information, 31 Nanyang Link, SCI Building, Singapore 637718 aschaudhry@ntu. edu. sg

Abdus Sattar Chaudhry is Head of the Division of Information Studies at the School of Communica- tion and Information at Nanyang Technological University (NTU) of Singapore. He holds a master’s degree from the University of Hawaii and a PhD from the University of Illinois. Before joining NTU in 1996, Dr. Chaudhry worked in different types of information institutions in the United States, Saudi Arabia, Malaysia, and Pakistan. He teaches in the areas of organization and management of information and knowledge. His current research focuses on building and deploying taxonomies for knowledge management.

Goh Hui Ling graduated with an MSc in Information Studies from the Nanyang Technological Uni- versity. She was previously involved in information and database management for a government agency, before moving on to knowledge management for a global media firm. Her areas of interest include taxonomies and knowledge portals for users in various industries. She is a member of the Infor- mation and Knowledge Management Society of Singapore.

Chaudhry, Abdus Sattar, and Goh Hui Ling. Building Taxonomies Using Organizational Resources: A Case of Business Consulting Environment. Knowledge Organization, 32(1). 25-46. 33 refs.

ABSTRACT: Taxonomies are becoming an increasingly important tool for companies to effectively manage information, particularly in the business consulting environment, where information is considered a main asset and a key product. This paper describes a case study of developing a taxonomy system for a regional business consulting company. The taxonomy, consisting of 12 main categories and approximately 500 terms, was built based on the existing knowledge structure and information needs of consultants in a selected company. This prototype can be conveniently utilised and adapted by other companies in their efforts to develop their own taxonomy system.

1. Introduction companies involves capturing information on real- time news, best practices, industry analyses, etc. and Companies that depend largely on information for making it accessible to their researchers and consult- the success of their business have to make extra ef- ants. According to the Delphi Group’s research forts to ensure that their information resources are (2002), the “lack of organisation of information is in properly organized. It is particularly important for fact the number one problem in the opinion of busi- business consulting companies, since “knowledge ness professionals.” One of the most common prob- and analysis is what [they] sell to clients” (Silverman lems faced is that information databases often be- 2002). A significant proportion of work in such come information dumps as a result of badly classi- 26 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

fied information. The traditional taxonomy, typically In order to benefit from their knowledge pool, it used to organize information, has been found to be is critical for companies to develop an effective tax- an effective solution to this problem. onomy that can be integrated into the information Taxonomy is a term borrowed from biology and database. The taxonomy will provide “a centralised, has been defined differently by scholars in different integrated search-and-browse experience” and de- fields. Some defined it as categorization based on liver “an intuitive browse interface” for their users certain criteria (Hakeem & Shah 2004, Adams 2002) (Potter 2001). A taxonomy that is well-constructed while others defined it as a structure that provides a and properly implemented can help to increase pro- way of classifying things such as living organisms, ductivity by reducing the amount of time needed to products and books, into a series of hierarchical locate the right piece of information at the right groups to make them easier to identify, study or lo- time. The taxonomy will provide the framework cate (Graef 2001, VanDercar 2002). The Delphi necessary for efficient categorisation of information, Group (2004) highlighted the importance of tax- help facilitate the navigation of and access to the in- onomies as a hierarchical listing of topics or subjects formation content, as well as aid in the search proc- or categories, while Warner (2004) defines taxonomy ess of relevant and timely information. from a broader perspective considering it a system of The working environment within the business labels that form a hierarchical navigation scheme and consulting industry is highly dependent upon and in- highlights its emphasis on building intuitive struc- fluenced by changes in the information environment. tures and using familiar terminology (labels) from It is an industry often challenged by the insurgence the users’ perspective to facilitate resource discovery. and overload of information. While many other in- While there are slight variations in the definitions dustries encounter similar information management of taxonomy, there seems to be a consensus about the issues, the key difference for business consulting key features: it is essentially a structure, made up of (being very much an information-related activity) is categories and the relationships that connect them, that these changes are experienced more markedly which enables users to classify things into a hierarchy. and at a much faster pace. Business consulting com- Several papers have used taxonomy interchangeably panies utilise and process a very large amount of in- with other knowledge organization tools like con- formation that covers a wide range of topics. Differ- trolled vocabularies (e.g., thesauri of indexing terms), ent business consulting companies typically have classification schemes, and ontology. While these varying foci. Some tend to specialise in a single in- knowledge organization tools have some similarities, dustry while others choose to have broader industry there are slight differences, particularly in their appli- coverage. Given this observation, it would not be cation. For example, controlled vocabularies focus feasible to develop a taxonomy that could compre- more on terminology and are used to facilitate hensively cover the information needs of every com- searching, and the emphasis in classification schemes pany within the business consulting environment. is on structuring knowledge resources. Gorshal Such a taxonomy would end up either being too gen- (2004) highlights the differences in metadata, eral (lack of depth) for companies that are more spe- thesauri, and taxonomy as knowledge support sys- cialised in their approach, or too specific (lack of tems. Ontology is an explicit and structured concept breadth) for companies that choose to cover many that specifies the set of characteristics of resources industries. In order to develop a taxonomy that can and their relationships that are deemed relevant to a be used as a template for business consulting com- particular community of users or a specific domain of panies in general, the taxonomy would include the interest (Uschold 1998, Soegel 1999, and Jacob major industries covered by most of these compa- 2003). Recently there has been emphasis on dynamic nies. It would also have sufficient depth so that in- taxonomies that are considered more suitable to han- formation can be managed efficiently. dle digital knowledge resources that tend to change For the purposes of this study, MR Business Con- quickly. These are considered particularly helpful in sulting (MRBC) was chosen as the locus for a case enterprise applications. They combine the strengths study in the development of business taxonomy. of different knowledge organization tools and can be (Note: Due to the nature of competitiveness within used to present the same knowledge resources to rep- the business consulting industry, the name of the resent multiple views and paths for diverse users al- company used in this study has been changed. ) lowing assignment of more than one category and MRBC was selected because it is a ‘generalist’ or- updating tagging as information changes. ganization – its business covers most of the major Knowl. Org. 32(2005)No.1 27 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

industries. For the ‘specialist’ consultancies, the se- not be simply “a good idea,” [and that] it must be lect industries could be further developed from the money making, service delivering or stakeholder- appropriate ones in the base template. value creating. By tying the taxonomy to business MRBC is a business consulting company that pro- benefit, it enables the taxonomical choices to be- vides market research, analysis and consulting ser- come interesting to the leaders of the business. This vices to companies in various industries such as food point is vital to the successful creation and imple- and beverage, media, retail, electronics and healthcare, mentation of the taxonomy system, as companies to name a few. The geographical focus of this com- that do not have the backing and support of their pany’s activities and research is the Asia-Pacific coun- top management will often find it an uphill task to tries, including Singapore, Thailand, Vietnam, China, change the working ways of the managers and their India and Australia. The projects and research work staff, and to convince them to embrace the new sys- carried out by the researchers and consultants at tem of information management. MRBC include competitive intelligence, assessment Sykes (2001) observed that while some companies of market sizes and structures, analysis of distribu- already have “sophisticated information and knowl- tion channels, partner evaluation, and identification edge management systems [and] are creating or up- of market opportunities for entry and penetration, dating internal proprietary taxonomies for labelling and business development consulting. documents and reports in their electronic informa- MR Business Consulting was selected for various tion repositories, others are [still] investigating the key reasons: economics of creating an indexing structure/taxonomy or investing in automated tools for this pur- – It is an established business consulting company pose. that is representative of its environment in terms Gilchrist and Kibby (2000) found that the major- of the services it provides and its relevance to the ity of the companies in the survey were looking for fast-changing business consulting landscape; greater business effectiveness as a result of access to – Its business interests include a wide range of in- the right information or else for efficiencies in industries, which covers most of the key industries formation retrieval which free time for better use. within the business consulting environment; Companies were looking to the structuring of in- – The coverage of the industries is not excessively formation as a means to aid their staff in the identifi- in-depth as in single-industry companies (which cation of information that may be critical to their would make the taxonomy too detailed to be rele- competitive position. However, they also discovered vant as a prototype); that while the companies were tackling the problems – It already developed an in-house taxonomy which through the use of business taxonomies, they were has a hierarchy of shared document folders for constructing their own business taxonomies de novo storing relevant files; the structure, usage and and that the working knowledge and theoretical ba- benefits of the in-house system provide a good sis underlying classification practice and the devel- basis for developing a prototypical taxonomy; opment of thesauri was not being fully utilised. and, The taxonomy building process for business tax- – The authors’ access to MRBC’s business envi- onomies differs from traditional process because of ronment, systems, information as well as its per- the different information needs of users in the fast- sonnel. changing business landscape. Different methods have been adopted by various organisations in the 2. Taxonomy Building Principles and Practices development of business taxonomies. These methods can be adapted, in part or whole, to the taxon- Gilchrist and Kibby (2000) and Graef (2001) provide omy building process, depending on the appropri- useful guidelines for capabilities required for busi- ateness of the methods with respect to the type of ness taxonomies. They emphasised that in the busi- business and the information needs of the users. We ness context, it is necessary to have a business- therefore reviewed guidelines developed and steps recognised driver for undertaking the development recommended by different individuals, organiza- of taxonomy. They also highlight that no matter how tions, and groups for developing organizational and small the required effort to create the taxonomy, it domain taxonomies and adopted these to develop a would nonetheless take up time and resources of a step-by-step process for our study. The following number of people. They argued that taxonomy can- papers were particularly helpful in this process: 28 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

– Chaudhry and Saeed (2000) highlighted the im- 2.1. Taxonomy Building Process portance of using organizational resources to build effective taxonomies. They (Saeed and The first step to the building of business taxonomy Chaudhry 2002) also recommended to leverage on is actually the recognition of the need for it. In the classification schemes and controlled vocabularies case of MRBC, it was the realisation of the urgent for building domain specific taxonomies. need to refine/build business taxonomy in order to – Conway & Silgar (2002) stressed that the starting help users with their information search and retrieval point in building a taxonomy is to look for exist- process. The need for a revamped taxonomy for the ing taxonomies, since re-using them will save the company’s information storage was discovered dur- developers many man-hours which would other- ing informal discussions with the users, when it was wise be spent building it from scratch. Besides in- realised that they were not paying attention to the ternal sources, they also suggest looking at exter- system (the categories and structure) implemented nal sources for ideas about how to build the tax- in the server. Instead of utilising the system, users onomy. were constantly approaching the information team – Ramos, L. & Rasmus, D. W. (2003) summarized for information, which was retrievable through a best practices in taxonomy development and man- search of the server’s information database. The rea- agement based on their consulting experience. son given for approaching the team for the informa- Chaudhry and Tan (2005) suggest a 10-step proc- tion was because they were not able to locate it ess for developing taxonomies for organizing re- quickly if they were to do it themselves. This was an sources in the domain of cultural heritage. They indication that the current taxonomy implemented emphasized on the need for content analysis and was not user-friendly or comprehensive enough for survey of users for determining appropriate objec- the users. It also highlighted to the information team tives for taxonomy building exercise. Pahleviand the need for a modification of the current taxonomy Kitagawa (2005) highlighted the importance of (for example, addition of more relevant categories), conveying taxonomy context for topic-focused to explore more options in order to create a more web search. McGregor, B. (2005) described a comprehensive taxonomy to meet the users’ needs. process of constructing a concise medical taxonomy. Gilchrist (2000 2002, and 2004) has high- 2.2. Information Needs of Users in MRBC lighted various principles of developing taxonomies in different writing and emphasised that tax- Finding the right information at the right time can onomy is a mechanism rather than a tool. He also often be a difficult task to achieve. Locating that stressed the need for a strategy for development same piece of information again can also be an and application of taxonomies. equally difficult task, if it is not properly stored. In fact, many of the researchers and consultants in We realized that although business needs and appli- MRBC often find themselves in a situation where cations will largely influence the structuring of the they urgently require access to a particular piece of taxonomy, the process of building the hierarchical information, which they vaguely remember having structure would also require a core set of standards come across before, but cannot recall where it was or rules that can act as a guideline. While these rules found or stored. For the users at MRBC, the most will differ depending on the type of businesses and logical way for them to locate previously stored in- user needs, it should cover the various basic deci- formation is to remember which project the infor- sions that have to be made during the structuring mation was obtained for and to search for it from process, such as the display of the terms, the levels that point. Unfortunately, trying to recall the spe- or depth of the structure etc. The guidelines re- cific project can be difficult as well, either because viewed in the section were helpful to us in deciding they are handling too many projects at one time, or the steps needed for building prototype taxonomy in the project was carried out a long time back. Some our environment. The steps are summarized in the users try to locate information by attempting to re- next section. call the title of the file or document, and subse- quently to search for it either by browsing through the taxonomy or by using the limited search function offered by the Windows Explorer program (which does not allow for free-text search). Knowl. Org. 32(2005)No.1 29 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Informal discussions with the users have revealed chical structure and the labels and terms used were that: solely based on his (the developer’s) perspective of – they would like to be able to have both options of the business consulting environment in the context browsing the business taxonomy as well as to of the company’s goals and directions. While this search the information within the server; method had its advantages (namely, it is a representa- – the majority of them do not browse through the tion of the user’s information needs), it was also, current taxonomy because they find it confusing from an information management’s point of view, and not user-friendly; and, clearly an inadequate and unprofessional way of de- – most of them do not utilise the in-house informa- veloping a taxonomy. tion stored within the internal server as it is not The business taxonomy is maintained and updated searchable; they would rather spend some time re- by Library and Information Services. All files and doing their search again using external sources, documents created and used by the staff of MRBC even though they are aware that it is an inefficient are stored within the shared internal server of the way of information search. company, commonly known as the J drive. The purpose of the J drive is to enforce the storage and shar- As a result of the poorly constructed taxonomy ing of all files and documents so that all information structure, users would rather not attempt to search is made readily available to all staff. Although the for the information that is technically available and company has developed a taxonomy system for clas- retrievable within the corporate collection. They sifying information, there is much room for im- would rather start searching for the information they provement in terms of developing a proper and con- need from scratch, even though it would typically be sistent taxonomy, as well as standardising the termi- time-consuming and often tedious In effect, the ex- nology used. isting taxonomy was becoming almost obsolete, with The taxonomy was implemented using Micro- little or no knowledge-sharing amongst the users. soft’s Windows Explorer software to represent the Similar to other business consulting companies, hierarchies and categories as well as to store the ac- the taxonomy used in MRBC focused on three key tual electronic file. Windows Explorer was used be- areas: company information, country information cause it was freely available and was able to represent and industry information. To create taxonomy for all the taxonomy, albeit in a very simple and ‘primitive’ three areas would be beyond the scope of this pro- format. The company chose not to utilise any of the ject as it would be too large-scale and extensive. In specialised software available in the market because order to bring a focus to the project, the taxonomy of budget constraints. However, at the time of this building will concentrate on the industry informa- study, the company is considering a few enterprise tion, which is most representative of the company’s portal solutions such as Microsoft’s Sharepoint, for activities and is also the most comprehensive in future management of the ever-increasing volume of terms of the amount of information stored. in-house information. While reviewing the taxonomy and its content, it 2.3. Existing Knowledge Structure in the Company was discovered that information was not always placed in the correct category, implying a lack of After identifying the need for a properly developed consistency with the way files and documents have taxonomy structure, the next step undertaken was to been indexed and categorised. To make matters more review the existing taxonomy system. As with most confusing, it was discovered that a few of the users other business consulting companies, MRBC has de- had the tendency to create their own new folders veloped an in-house version for internal use by staff, whenever they were not able to find the appropriate which comprises mainly of researchers and consult- folders to store a particular document or file. Rea- ants. It has attempted to solve the problems of con- sons given for this were either because they did not tent management (or rather the lack of) by “setting have the time to look for the correct folder, or more up systems of folders and subfolders” (Humming- frequently, that the system was not sufficiently well- bird 2000). The existing taxonomy was manually developed to support the company’s information re- created by a top management staff of the company. sources. These feedback and user activities were not Its development was not systematically carried out unreasonable; in fact, they highlight a serious under- and there was no adherence to any particular guide- lying truth about the existing taxonomy – it was line or methodology. The development of its hierar- poorly developed and urgently needed overhauling 30 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

before the situation got out of ontrol and the cate- house business taxonomy; four leading online data- gorisation system collapsed from mismanagement of bases/information providers, namely Profound (by folders. Dialog), Factiva (by Dow Jones and Reuters), Lex- isNexis and OneSource; and two business portals, 2.4. Steps Taken for the Development of the Prototype CorporateInformation and Hoovers. The main cate- Taxonomy gories from each of these seven lists were tabulated within an Excel spreadsheet. Different terms used to 2.4.1. Development of the List of Terms represent a particular category (or industry) were grouped together. Before starting on the construction of the taxonomy Although the seven lists provided a wide range of and its hierarchy of categories, a list of standard industry categories, there were many category- terms (otherwise known as the thesaurus or vocabu- overlaps among the lists. It was decided that this re- lary) has to be compiled. This list of terms should search would concentrate on the main categories of define and recapitulate the content of the collection. industries which MRBC’s business clients come Currently, there are no published taxonomies or from. Although the in-house taxonomy of MRBC thesauri that are available for public use as templates would give a good indication of these industries, the or guidelines to develop customised taxonomies spe- existing taxonomy was not completely relevant. This cifically for the business consulting environment. In is because it was not updated regularly, and there was order to compile a list of terms that will comprehen- a possibility of redundant or outdated industries. As sively represent the business consulting environment such, an informal survey was carried out with indi- of MRBC, several sets of terms were obtained from vidual users within the company to find out which various sources in order to populate the taxonomy categories should be included in the taxonomy. The with relevant, appropriate or frequently used termi- survey was carried out through one-to-one discus- nology. sions about the various categories and their relevance For the selection of the appropriate lists of terms, to the existing consulting work of the company. certain criteria have to be set so that the lists would Categories of industries that are not within the be able to provide a comprehensive perspective of scope of MRBC’s current work were removed from the business environment of the company. A good the spreadsheet. As differing terms were used by the source for the lists would be the actual tools used by various lists to represent the same industry, compari- the business consultants during their information sons are then made across each of the remaining search. These commonly used tools generally fall categories to determine the most suitable term to be into one of the following three types, namely the used. Many factors are taken into account during company’s internal information database, an online this process. These include: database or a portal. From these, three criteria are set for the shortlist: the lists have to be taxonomies, in- 1. the frequency in which a particular term was used dexes or thesauri of one of the following: by the different lists 2. the industry terms used by MRBC, and, 1. a company in the business consulting environ- 3. the opinions of respective industry experts, ob- ment; tained through informal interviews. 2. a business-oriented or business-focused online database, or 2.4.3. Terms for the Subcategories 3. a business-related portal. From the seven lists, four were further selected for In addition, it is important to ensure that the lists se- the compilation of a database that would represent lected are easily accessible. the entire business consulting environment of MRBC. The four lists were MRBC’s in-house tax- 2.4.2. Terms for the Main Categories onomy and three of the online databases, namely Factiva, Profound and LexisNexis. MRBC’s taxon- Based on the criteria above, seven lists were chosen omy was selected, as its terms would best represent to provide the terms for compiling a set of main the company’s information needs and environment. categories which would form the first level hierarchy Factiva, Profound and LexisNexis were chosen be- of the taxonomy. The lists are: MRBC’s current in- cause they were leading online database providers for Knowl. Org. 32(2005)No.1 31 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

many business entities. As such, each of their tax- at a later stage, more emphasis will be placed on onomies would provide good business coverage. terms that appeared in more than one list by virtue OneSource was not used as their list of terms was of its more frequent utilisation by other taxonomies not readily available for review. The other two lists, or indexes. Any duplicate term was then removed CorporateInformation and Hoovers, were not cho- from the list. sen as their taxonomies were more focused on repre- After the list has been culled of any duplication of senting different types of company information. terms, the next step was to remove the terms that The four selected lists were individually reviewed were irrelevant to MRBC’s scope of work. The scope and the appropriate industry-related terms were ex- of industries covered by the three online databases is tracted from each. Approximately 3,000 terms were much broader than that of MRBC’s. These databases compiled from the four lists and entered into an Ex- cater to the needs of a wider spectrum of industries cel spreadsheet. This compiled list of terms would in the business environment. In order to bring more form the base for establishment and development of focus into the taxonomy and to make it relevant to the prototype taxonomy. Each of these terms were the users, industries that are beyond the scope of the tagged with the name of the list from which it was company’s work were removed from the list. extracted, as well as the level of hierarchy or the categorisation it was placed in the original lists. This 2.6. Development of the Hierarchical Structure tagging is necessary so that each term can be traced to its original source, if necessary. The tagging would Having compiled a list of terms that are representa- also give an indication of the importance or hierar- tive of MRBC’s working environment, the next step chical level of each of the terms in relation to other was to develop the hierarchical structure of the tax- terms. For example, a term used at the first or sec- onomy. This involved combing the finalised list of ond level of the hierarchy would be deemed more terms and individually tagging each term with a rele- significant for use in taxonomy building, as opposed vant industry category (taken from the list of cate- to a term appearing in lower levels of the hierarchy. gories compiled as described in the earlier section). Lower level terms typically have a narrower scope In determining the category for each term, some and may not be useful in taxonomies that are more level of general knowledge and common sense was general in nature. Higher level terms are more fre- applied. However, a good understanding of each quently applied as most consist of at least 2-3 levels. category (based on my past project experiences) is However, there are exceptions to this rule, depend- also essential in order to make a calculated decision. ing on the context in which the terms are used and Another useful guideline which was applied during the industry in question. this process was to refer to the original category which the term was linked to in its original database. 2.5. Review and Selection of Appropriate Terms Once each term was appropriately tagged, they were sorted out and separated according to their respec- Having compiled these terms, the next step was to tive categories so that the hierarchical structure can cull the terms into a manageable and workable size. be developed on a category-by-category basis. In her checklist for the design of the tree structure In order to determine the hierarchical position of for Illinois State Library’s thesaurus, Schriar (2000) each of the terms within its respective categories, a indicated that the size of the thesaurus should be not few guidelines were followed: more than 700-1,000 terms. Although the figure would differ depending on the type of taxonomy and – The original taxonomy developed by the company the breadth and depth of the taxonomy, this would was a very useful guideline to the way the re- nonetheless act as a useful guideline. searchers’ minds perceived the way information The list of terms in the Excel spreadsheet were should be organised within each category. first sorted by alphabetical order to aid in the identi- – The more times a particular term appeared in the fication and consolidation of all duplicate terms. initial list of collated terms (as indicated by the Terms that appeared more than once were first tags), the more important is the term for the cate- tagged to indicate the frequency of appearance, gory. which would in turn suggest the suitability of the – Relevant websites relating to each category were term for use in the taxonomy. For example, during studied where available. For example, Business. the selection of appropriate terms for the taxonomy com’s web site (www. business. com) was a useful 32 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

website/portal that categorises the information on relevant sections of the taxonomy, and discussions various industries very clearly and concisely. were carried out based on their comments and que- Other similar useful web sites provided guidelines ries about the taxonomy. The draft taxonomy was on the terms and their hierarchical positions for then revised accordingly, and the final prototype each category. taxonomy was presented to the industry experts for – Additionally, both ‘common sense’ as well as a a final review. ‘business consulting mindset’ application of the meaning of the terms should be applied when de- 3. Prototype Taxonomy ciding on the appropriate layer into which each term should be placed. The main outcome of this study is prototype taxonomy. Implications in different taxonomy building Following these guidelines, the list of terms was or- steps and lessons learned during the process are dis- ganised into a set of categories which was then ar- cussed in the next section. The entire taxonomy is ranged in a hierarchy. This forms the first draft of made up of approximately 500 terms; different cate- the taxonomy for MRBC. gories having different number of terms or subcategories attached to them, depending on the indus- 2.7. Modification and Refinement of Business try in question and its degree of importance to Taxonomy MRBC’s business development. A sample from the prototype is given in the appendix. Although the draft taxonomy was formed from sources that were credible and representative of the 3.1. Consideration in Developing the Prototype current business consulting environment, the selection of terms and the development of its hierarchical 3.1.1. Terms and Hierarchies structure were carried out based on a single person’s perception of the same environment. While some During the process of compiling the list of terms for guidelines were followed, bias and error are inevita- the taxonomy, various sources were used. The terms ble. In order to circumvent this problem, the draft from each source were combined and built into a prototype taxonomy was presented to six industry pool of terms (amounting to over 3000 terms). The experts for their review. These industry experts were process involved culling irrelevant terms and recon- either consultants from MRBC or other persons ex- ciling terms that were similar in meaning. This was a perienced in a specific industry. These experts have difficult task to accomplish as each of the sources se- extensive experience in their respective field so as to lected had different formats with regards to the know the domain and at the same time sufficient terms that were used. work experience in the company to keep in view the In order to overcome these difficulties, it was im- context while suggesting a particular term or label or portant to first identify and develop a concept and assisting in determining the hierarchical level. This understanding of what is needed, and then develop provided a good combination for assuring the liter- guidelines as to which terms to cull or reconcile. It ary as well user warrant in the proposed taxonomy. was also necessary to apply personal professional in- The review was carried out on an informal basis, sight, as well as the opinions of subject experts, be- using face-to-face meetings, email communications, fore eventually deciding what to include and what to and/or telephone interviews. While no questionnaire remove from the compiled list of terms, as well as or interview schedules or forms were given them to how the terms should be arranged in a hierarchical fill (as there was a bit of hesitance on the part of the order. There are no hard and fast rules to follow as experts to provide input using such formal means). this is a subjective process, often depending on the The researcher used worksheets to fill for the pur- individual experiences and perceptions of the people pose of consistency and proper documentation. The involved in the development of the taxonomy. It is opinions of the experts were sought on the validity highly possible that two experts within the same in- of the terms selected as well as their respective hier- dustry will each come up with a taxonomy that is dif- archical levels within the taxonomy structure. Ef- ferent from the other. The differences arise from how forts were taken to ensure that the context of the the individual perceives the industry, which in turn is study and the taxonomy were explained clearly to dependent on their respective circumstances and ex- the experts. The experts were presented with the periences. There is no right or wrong answer as there Knowl. Org. 32(2005)No.1 33 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Figure 1. Main Categories is no perfect taxonomy. It is therefore important to The original taxonomy developed for MRBC con- bear in mind that it would be impossible to come up sists of 15 main categories which represent the core with a taxonomy that will meet every single user’s industries of MRBC’s past business consulting work. needs. What is important is to know how best to uti- The prototype taxonomy is made up of 12 main lise the resources available and to develop a taxonomy categories, which are the key industries that MRBC that would capture the essence of the environment it is currently involved in. Figure 1 shows the use of 12 is meant to represent. It is therefore vital that the main categories representing main industries as the people in the team have a good understanding of the first level of hierarchy. These broad themes were de- specific work environment of the company; that the vised to provide unifying points for diverse re- knowledge of subject experts are tapped upon for a sources rather than division by resource type. deeper and broader insight of the industries; and last A quick comparison between the newly developed but not least, that the opinions and needs of users are taxonomy and the original version would reveal that, incorporated into the process. while most of the industries remain intact, a few have been added or removed, reflecting the changes 3.1.2. Differences between the Original and the in the focus of the consulting world from MRBC’s Prototype Taxonomy Structure perspective. Some main categories have been added as MRBC expands its business to flourishing indus- In order to present the taxonomy as a hierarchical tries, while others have been removed because the structure, with the options to expand and collapse company no longer sees the industry as an area of the tree structure, it is created as a .kbf file using the focus for growth (either due to the economic condi- MyInfo software (a free version of the software is tions or the inability to compete with other competi- available at http://www.milenix.com/download.php). tors in that specific industry). Prototype taxonomy is given at the end of the paper. Figure 2 and 3 show the second level sub- Main categories and different levels of taxonomy are categories of the Banking, Finance & Insurance in- shown in Figures 1-6. dustry and Chemicals.

34 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Figure 2. Level 2 Subcategories for Banking, Finance & Insurance

Figure 3. Level 2 Subcategories for Chemicals

Knowl. Org. 32(2005)No.1 35 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Figure 4. Level 3 Categories for Food and Beverages

Figure 5. Level 4 Subcategories for Healthcare

36 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Most of the main categories in the original taxon- level, so that the time spent browsing each level can omy had only two to three levels in terms of depth. be reduced. Although there was some feedback that Some of them did not even have any subcategories the limit should be set at four or five subcategories, created. This lack of depth resulted in too many this was not applied to the entire taxonomy as it documents being placed in a single subfolder. This would have affected the representation of certain in- slows down the search process for users as they have dustries. to scroll down a long list of documents that have not Figure 4 and 5 are showing sub-categories in dif- been categorised into more specific folders. In order ferent classes. to classify the documents into more specific sub- Figure 6 shows the depth in hierarchy listing part categories, the prototype taxonomy was created with of the second, third, fourth and fifth levels sub- more levels in depth. However, based on some user categories of the Consumer Products category. feedback and other studies on the number of levels Other smaller details, neglected during the devel- optimal for browsing, the number of levels was lim- opment of the original taxonomy, were attended to ited to five. This was to ensure that the taxonomy during the development of the prototype taxonomy. can be used in an efficient manner, as too many lev- These include standardising the form and format of els in depth would make it difficult for users to the terms used to represent the various categories browse through quickly, because they would have to and subcategories, such as the use of capitalisations drill down many levels of the taxonomy before arriv- within each term, and sorting of terms within each ing at the information needed. level in alphabetical order for easy reference. Another key difference and improvement over the While the end product of this study is prototype original taxonomy is the number of terms or sub- taxonomy for a business consulting environment, categories allowed for each level. The original taxon- the development process helped in understanding of omy had as many as 26 subcategories within one information needs of users, as well as greater expo- level, which made the structure difficult to view sure to the various tools available in the market that when browsing or drilling down the structure. The can be used in taxonomy development. prototype taxonomy used 10 subcategories for each

Figure 6. Depth of Hierarchy in Prototype Taxonomy Knowl. Org. 32(2005)No.1 37 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

3.2. Selection of Tools for Taxonomy Development a taxonomy structure that would please all the users. This is because different users have different A search for a standardised, off-the-shelf taxonomy needs as well as different perspective of the various for the business consulting industry did not turn up industries. Some users may also have political agen- anything of relevance. Unlike the medical or engi- das to fulfil in terms of their requirements for the neering industries, where taxonomies are widely ap- taxonomy. For example, users who are experts in a plied, the business consulting environment appears particular industry may protest when they find out to be a laggard and lacking in this area. Without a that their particular industry has been represented published example to use as a guideline, the next al- as being ‘smaller’ than others in terms of the ternative was to find a thesaurus or an index that is breadth and depth of their industry in the taxon- tailored towards business consulting terminology. omy structure. Another area of expectation to be Again, there is a dearth of published lists of this type managed is the inevitability of incompleteness with- available for use or download. However, there were a in the developed taxonomy. The information pro- number of published thesauri or indexes of terms fessional must bear in mind, and must communicate that have been developed for the general business to the users, that the taxonomy is an expandable environment. While these could be used as a guide- and flexible structure, and will be adapted accord- line, most were proprietary and not freely available. ingly when new business interests arise for the Fortunately, a search through these commercial company. As choice of terms was largely based on thesauri and indexes came up with a few that could the existing information content of the company, be readily accessed. The first was LexisNexis, which the taxonomy could be adapted and new subcatego- was an exception to the rest, as they chose to make ries added when documents pertaining to new top- their SmartIndexing Technology Topical Indexing ics are added. Terms available on their web site. Factiva’s Intelli- gent Indexing (FII) was also accessible for the pur- 3.3.2. Manual versus Automatic Development of the pose of this project, as it was made available to its Taxonomy clients, NTU being one of them. Another list that was accessible was Dialog’s Newsroom Sourcebook There were many challenges faced during the taxon- as MRBC was a subscriber of this database. omy creation process. One of the first considera- LexisNexis’, Factiva’s and Dialog’s databases are tions is whether the tasks are to be carried out catered for the business community and their index manually or automatically. Although no formal of terms naturally fit the overall structure of most study has been made on the necessity of manual in- business consulting companies. However, when it tervention for taxonomy development (Hagedorn comes down to the specifics, the availability of suit- 2001), it is generally recognised and agreed amongst able terms for the taxonomy is very much dependent information professionals that manual intervention on the areas or industries in question. As business is necessary, the opinions differ only in where the ex- consulting companies sometimes cover very broad tent of human involvement should be. In most cases, ranging industries, it would therefore be necessary to the extent of manual versus automatic involvement combine the three databases to ensure the industry would be determined on a company-by-company ba- coverage is sufficient. In cases where certain industry sis, depending largely on the type, size and unique coverage is found to be lacking, it may also be neces- characteristics of the company in question. Manual sary to make use of other sources to supplement the involvement could range from the minimum effort terms used. to a full-fledged participation. In the case of MRBC, manual involvement in the 3.3. Lessons Learned during the Development of the taxonomy development ranks high, largely due to Prototype Taxonomy the lack of a taxonomy application or software to utilise and automate some of the processes. As the 3.3.1. Managing Users Expectations and Needs collection size is currently still manageable, it is feasible to carry out most of the development manually. Before embarking on the development of taxonomy However, this will probably not be the case should for a company, it is vital that the information pro- the collection start to expand rapidly. fessional is equipped in managing the expectations of the users. It would be near impossible to develop 38 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

3.3.3. Identification of Main Categories and Terms within the framework of their recently required enterprise portal software that supports developing Another key issue faced during the development knowledge repositories. Among other features, the process was the need to identify the categories and system also has a sub-system for category manage- terms in line with the company’s current business ment. However, the company is still struggling for environment, as well as comprehensively represent developing a mechanism for automatic tagging as no the information within the company. The selection hundred per cent machine solution has yet been of various online databases and business web sites found. As an interim measure they have decided to was an attempt to get a broader view of the types of develop classification rules that should assist in tag- main categories and terms used in the business world ging and categorization using a manual system as a to represent the industries. The frequency of usage stop-gap arrangement with an ultimate objective of a of a particular term by the sources can also give an system that is assisted by automated tools in facili- indication of its appropriateness as a term for busi- tating the indexing and categorization work. They ness taxonomy. have also decided to use Dublin Core as metadata It should, however, be noted that while these schema as there is a strong feeling that without com- sources do provide a useful “database” of terms to prehensive metadata they will not be able to take full choose from, it may sometimes be necessary to se- advantage of the potential of taxonomy. It is esti- lect a less commonly used term or even to modify mated that a reasonable number of knowledge re- the terms available so as to better reflect the specific sources will be linked to selected categories. We in- information needs of the company. This is because tend to do a study to assess the effectiveness of tax- each company has its own unique requirements and onomies once a reasonable level of tagging is the application of an information professional’s un- achieved. derstanding of the company may help contribute to a more accurate and useful taxonomy for the users. 5. Summary And Conclusions Hence, although it is important to include external sources to ensure some form of standardisation and In order to facilitate the discovery of the informa- objectivity to the taxonomy, the process of taxon- tion resources within the company, it was realised omy development would still require input and opin- that there is an urgent need for some clear guidelines ions from individuals who understand the company’s on the development process of taxonomy building. needs. Using the existing taxonomy and business environment of MRBC as a base, prototype taxonomy was 3.3.4. Development of the Hierarchical Structure developed for business consulting companies. The prototype taxonomy consists of 12 main categories The development of the hierarchical structure is car- and approximately 500 terms. The main categories ried out based on two main factors: firstly, the com- represent 12 business industries, which are the key mon sense perspective of the various industries, and industries of the company’s business consulting secondly, the industry experts’ view of the respective work. The taxonomies and indexes used by various industries. This process is highly subjective and de- online database providers and web sites were used to pendent on the views of a few individuals. It is there- provide the building blocks for the prototype taxon- fore crucial that industry experts are involved so that omy, while informal interviews with industry experts the industries are not misrepresented in the taxon- and feedback from users of the company were in- omy. It is also important that the users are consulted strumental in helping to determine the structure of on the hierarchical structure developed for each of the taxonomy. the categories, since they will be the ones who are We have learned in the process that before em- utilising it. barking on the development of a business consulting taxonomy, it is important that the team involved 4. Taxonomy Deployment should make careful plans and decisions about the priorities of the business and the balances between The company has agreed in principle to use the pro- the business goals, the users’ needs, the budget avail- totype taxonomy for categorization of knowledge able for setting up the system as well as the man- resources of linking of exiting documents to differ- agement’s long-term plans for the company. All ent categories. It is intended to be implemented these would differ from company to company, and Knowl. Org. 32(2005)No.1 39 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

would determine the steps to be taken as well as the ently available that can address the problems in its final product of the taxonomy. entirety without involving some extent of manual While it was useful to compile and compare terms inputs. Companies would still need to customise used by different sources to represent their informa- these packaged solutions to tailor them according to tion, it is useful to bear in mind that these terms are their business needs. Hence, the best approach to selected to provide a guideline, rather than a hard- developing business taxonomy would integrate both and-fast rule, to the types of terms that would popu- the manual and automated approaches in the right late the taxonomy structure. This is especially im- proportions as determined by the business nature portant in the context of developing taxonomy to and the company’s specific information needs. The meet the unique requirements of a specific company. taxonomy developed for MRBC can be used as pro- It is also vital that the information professionals totype taxonomy for other business consulting within the company are able to develop a good rap- companies. The guidelines and methods developed port with their colleagues, because their input is during the process can also be similarly applied to needed to ensure that feedback from the users of the modify the prototype taxonomy according to the in- taxonomy themselves is incorporated into the build- formation needs of each company, bearing in mind ing of the taxonomy. While industry experts have that business taxonomies are as unique as the com- helped to provide a good perspective of the various panies they represent. industries, their interpretation may not be easily understood by the users of the taxonomy. Industry ex- References perts are likely to describe the industries to a high degree of depth or detail, which may not be neces- Adams, K. 2002. The semantic web: Differentiating sary for the purposes of the business consulting en- between taxonomies and ontologies. Online 26: vironment. It is therefore important that most staff 20-23. in the company are involved in the development Bruno, D. & Richmond, H. 2003. The truth about process. Without the cooperation of these users, it taxonomies. Information management journal, 37: would not be possible to fine-tune the taxonomy to 44-51. ensure that it meets their information needs. Chaudhry, A. S. and Saeed, H. 2001. Taxonomies ap- The manual development of the prototype taxon- plications for leveraging organisational knowledge omy was a long and time-consuming process, from resources. Singapore journal of library & informa- the compilation and selection of terms, to the formation management 30: 45-52. tion of the hierarchical structure. It would be more Chaudhry, A. S. and Tan, P. J. 2005. Enhancing access efficient if the process can be partially automated, if to digital information resources on heritage: a case budget permits. Although sophisticated tools were of development of taxonomy at the integrated not available for this study, some manual processes museum and archives system in Singapore. Jour- were shortened by making use of available software nal of documentation forthcoming. such as Microsoft Excel, which was used to compile Delphi Group. 2002. Taxonomy & content classifica- and cull out the terms more quickly, and MyInfo, a tion. http://a400.g.akamai.net/7/400/5663/v001/ software that was able to effectively present the tax- www.delphigroup.com/research/whitepapers/ onomy in its hierarchical structure. WP_2002_TAXONOMY.PDF The development of business taxonomy does not Delphi Group. 2004. Information intelligence: Content end with the formation of the taxonomy structure. classification and the enterprise taxonomy practice. The work continues with the constant maintenance Retrieved on November 23 2004, from http:// of the hierarchy. As the company’s goals and direc- www.delphigroup.com/research/whitepapers/ tions change with the times, and as the information 20040601-taxonomy-WP.pdf collection and usage evolve, so must the taxonomy Dialog Corporation. 2000. Automating document adapt and be modified in order for it to remain rele- classification with InfoSort Server. http://www. vant and effective. This is a massive task and a tax- tradespeak.com/processdownload.asp?UX= onomy that relies totally on manual efforts to build 41577&PBX=356&DCX=1701&FT=pdf&pub and maintain will prove to be impractical. Not only id=356&docid=1701&filetype=pdf will it be prohibitively costly, but also it will lack the Dialog Corporation. 1999. Profound for the Internet agility to respond quickly to changing content. user guide. US: Dialog Corporation. However, there is also no packaged solution pres- 40 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Factiva. 2002. Factiva intelligent indexing. Retrieved Pahlevi, S.M. and Kitagawa, H. 2005. Conveying February 25 2002. http://www.factiva.com/ taxonomy context for topic-focused web search. content/intindexing/codes.asp?node=sub-menu4- Journal of the American Society for Information link2 Science and Technology. 56:173-88. Garshol, L. M. 2004. Metadata? thesauri? taxono- Potter, S. 2001. Building an enterprise taxonomy mies? topic maps! making sense of it all. Journal [electronic version]. Special supplement to of information science 30:378-91. KMWorld November/December 2001. http:// Gilchrist, A., & Kibby, P. 2000. Taxonomies for busi- www.kmworld.com/publica- ness: access and connectivity in a wired world. Lon- tions/whitepapers/KM/potter.pdf don: TFPL Ltd. Ramos, L. and Rasmus, D.W. 2003. Best practices in Gilchrist, Alan. 2001. Corporate Taxonomies: Re- taxonomy development and management. http:// port on a Survey of Current Practices. Online in- www.stratify.com/infocenter/download/BestPract formation review 25. icesInTaxDevAndMgmt4.pdf Gilchrist, Alan. 2003. Thesauri, taxonomies and on- Saeed, H. and Chaudhry, A. S. 2002. Using Dewey tologies – an etymological note. Journal of docu- Decimal Classification Scheme (DDC) for build- mentation 59: 7-19. ing taxonomies for knowledge organization. Jour- Gilchrist, A. 2004. The taxonomy: A mechanism, nal of documentation 58: 575-84. rather than a tool, that needs a strategy for devel- Schriar, S. 2000. Find-It! Illinois: Improving access to opment and application. In A. Gilchrist & B. Illinois’ electronic information resources. http:// Mahon, eds. Information architecture: Designing www.cslib.org/stdocconf/hartford. ppt information environments for purpose. London: Silverman, R. E. 2002. McKinsey report shows Facet Publishing, pp. 192-98. growth hindered use of data. The Asian Wall Street Graef, J. (2001, March 30). Managing taxonomies Journal, May 21,p. A9 strategically. Montague Institute Review (abbrevi- Soergel, D. 1999. The rise of ontologies or the rein- ated version). http://www.montague.com/review/ vention of classification. Journal of the American taxonomy3.html Society for Information Science 50: 1119-20. Hagedorn, K. 2001. Extracting value from automated Sykes, J. 2001. The value of indexing. Retrieved July 4 classification tools: The role of manual involvement 2002. http://www.factiva.com/infopro/ and controlled vocabularies http://argusacia.com/ indexingwhitepaper.pdf white_papers/classification.pdf Uschold, M., King, M., Moralee, S., and Zorgios, Y. Hakeem, A. and Shah, M. 2004. Ontology and taxon- 1998. The enterprise ontology. Knowledge engi- omy collaborated framework for meeting classifica- neering review 13: 31-89. tion. http://longwood.cs.ucf.edu/~vision/papers/ VanDercar, A. 2002. What is taxonomy? http:// hakeem_shah_ICPR_2004.pdf md.essortment.com/whatistaxonomy_rfcd.htm Hummingbird Ltd. 2000. Automatic document cate- Warner, A. J. 2004. Information architecture and vo- gorisation. http://www.gms.co.uk/pdf/ cabularies for browse and search. In A. Gilchrist AutoDocWF1.pdf & B. Mahon, eds. Information architecture: De- Jacob, E.K. 2003. Ontologies and the semantic web. signing information environments for purpose. Lon- Bulletin of the American Society for Information don: Facet Publishing, pp. 177-91. Science and Technology 29n4: 19-22 LexisNexis. 2002. LexisNexis SmartIndexing technology topic indexing terms. http://ip.lexis-nexis.com/ products/index/word/Terms%20by% 20Category.doc NewsEdge Corporation. World class taxonomy for business - structured content, in context. http:// www.newsedge.com/products/why_newsedge/tax onomy.asp McGregor, B. 2005. Constructing a concise medical taxonomy. Journal of Medical Library Association 93: 121-23. Knowl. Org. 32(2005)No.1 41 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Appendix Life insurance Property & casualty insurance PROTOTYPE TAXONOMY FOR THE Reinsurance

BUSINESS CONSULTING ENVIRONMENT Taxation Corporate taxes BANKING, FINANCE & INSURANCE Duties & tariffs Personal taxes Accounting Accounting standards Audits BUILDING & CONSTRUCTION

Banking Architectural services Banking & financial services Design Commercial banking Extension to existing building Commercial lending Interior decoration Interbank lending Interior design Retail banking New building design Retail mortgages Design & build Small business lending Consumer banking Building renovations Consumer loans & deposits Home improvements Credit, debit & stored value cards Home remodelling Financial planning Investments Construction Private & priority banking Building materials Electronic banking Insulation products Internet banking Masonry products Banking technology Metal construction products Automated banking Plastic construction products Phone banking Building permits Electronic funds transfer (EFTPOS) Construction equipment Banks Construction machinery Central banks Construction tools Commercial banks Construction spending Investment banks Home construction

Investment services & products Bonds CHEMICALS Commodities Derivatives Adhesive, fillers & sealants Equities Initial public offerings Agricultural chemicals Stock options Fertilisers Online trading Herbicides Real estate investments Unit trusts Lubricants Industrial lubricants Financial regulations Banking & financial regulations Paints & coatings Insurance regulations Coatings Securities regulations Colourants Monetary policies Dyes & pigments Taxation laws Pesticides Insurance Insecticides Insurance agencies & brokerages Insurance products Pharmaceuticals see “Types of drugs” Automobile insurance Health insurance see also “Healthcare Plastics policies” Resins 42 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Rubber Fragrances Hair care Solvents Oral care Sanitaryware Specialty chemicals Household appliances Volatile organic compounds Domestic appliances Air conditioners Clothes dryers CONSUMER PRODUCTS Dishwashers Microwave ovens Apparel & accessories see also “Apparel Refrigerators retailing” Tools Clothing Washing machines Childrenswear Vacuum cleaners Menswear Small electrical appliances Sleepwear Coffee makers Sportswear Toasters Underwear Womenswear Home furnishings Fashion Furniture Fashion accessories Built in furniture Leathergoods Home furniture Footwear Office furniture Unassembled furniture Baby products Household goods Baby care Kitchenware Baby toiletries Tableware Disposable diapers Lighting products Baby clothings Paper products Consumer electronics Disposable paper products Audio discs & tapes Compact discs Personal goods Minidiscs Jewellery Audio equipment Watches and clocks Compact disc players Minidisc players Pet products Radios Pet foods Record players Cameras & photography equipment Toys & games Cameras Games Digital cameras Toys Video cameras Photographic film ELECTRONICS Televisions Colour televisions Electronic components Digital televisions Capacitors High-definition televisions Connectors Video discs & tapes Diodes Video discs Fuses Video tapes Integrated circuits Video equipment LEDs Tape recorders Relays Video recorders Resistors Switches Cosmetics & toiletries Transistors Cosmetics Toiletries Electronics production Body care Board level component manufactur- Face care ing Knowl. Org. 32(2005)No.1 43 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Cable assembly manufacturing Take-aways Contract manufacturing Food packaging Magnetic products manufacturing Canned foods Printed circuit board fabrication Food labelling Food processing Office automation & supply Food producers Office equipment Food products Facsimiles Bakery products Multifunctional products Breakfast cereals Photocopiers Condiments Printers Dairy products Typ e w r i t e r s Desserts Oils & fats Optoelectronics Organic foods Amplifiers Snacks Couplers Vegetarian foods Detectors Food safety Fibers Fresh, chilled & frozen foods Integrated optical circuits Chilled foods Modulators Fresh foods Sources Frozen foods Switches Genetically modified foods Health foods Semiconductors Non-single-crystal silicon technology Tobacco industry Single-crystal silicon technology Cigarettes Tobacco

FOOD, BEVERAGE & TOBACCO HEALTHCARE Beverage industry Alcoholic beverages Biotechnology Beer DNA testing Liquers & aperitifs Genetics Spirits Genetics research Wines Medical research & technology Wine production Tel e m e d i c i n e Beverage packaging Bottled packaging Health promotion Canned packaging Health & fitness PET packaging Health care facilities Carbonated beverages Nutrition & dietetics Distillers & brewers Health drinks Medical Liquid diet supplements Bioethics Milk processing Clinical medicine Non-alcoholic drinks Anesthesiology Beverages Cardiology Cocoa beverages Dermatology Coffee Epidemiology & public health Hot beverages Obstetrics & gynecology Tea Oncology Juices Optometry & ophthalmology Milk Orthopedics Mineral water Pediatrics Surgery & transplantation Food industry Healthcare policies see also “Health Baby foods insurance” Convenience foods Occupational compensation Fast foods Public & health policies Pre-packed foods Healthcare providers 44 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Health care services Gastroenterology Hospitals & clinics drugs Managed care organisations Haem-onco drugs Medical professions Nervous system drugs Nurses & nursing Prescription analgesics Physicians & surgeons Psychiatric drugs Nursing homes Renal drugs Medical education Respiratory drugs Medical equipment & supplies Vaccines Diagnostic products Diagnostic imaging Supporting healthcare services Immunodiagnostics Alternative medicine Medical devices & equipment Chiropractic Cardiovascular Domiciliary care devices Rehabilitation & occupational therapy Dialysis Ophthalmic equipment MEDIA & ENTERTAINMENT Medical electronics Medical supplies Advertising Medical disposables Advertising rates Medical waste & biohazards Broadcast advertising Medical research Radio advertising Mode of treatment Television advertising Lasers Outdoor advertising Print advertising Pharmaceuticals Drug companies Broadcasting Direct to consumer drug mar- Broadcast equipment keting Public broadcasting Drug distribution wholesale Satellite broadcasting Drug manufacturers Transmissioons Drug labelling Television & radio broadcasting Pharmaceutical pack- Cable television aging Radio broadcasting Pharmaceutical associations Television broadcasting Pharmaceutical product development Film, music & video Clinical research Movie industry Clinical trials Film & video production Retail pharmaceuticals Movies Drug interactions & side effects Theaters Pharmaceuticals regulations Music industry Types of drugs Audio recordings Drugs by mode of delivery Recorded music Aerosol drugs Recording industry Oral drugs Video industry Perental drugs Topi c a l dr u g s Intellectual property OTC drugs Censorship Antiseptics Copyright Cough & cold reme- Intellectual assets dies Intellectual property law Nutritional pharma- Privacy rights ceuticals Simple analgesics New media Prescription drugs New media consultancy Allergy & immunol- New media services ogy drugs Antiinfectives Publishing & printing Cardiovascular drugs Printing Knowl. Org. 32(2005)No.1 45 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Publishing Food retailing Book publishing Beer, wine & liquor stores Editorials & columns Grocery stores Educational publishing Health food stores Electronic publishing Retail bakeries Magazine publishing Newspaper publishing Housewares & furnishings retailing

Online retailing REAL ESTATE Supermarkets & hypermarkets Auctions Hypermarkets Supermarkets Commercial property Commercial property leasing Commercial property sales SERVICES

Industrial property Business services Industrial property leasing Accounting & auditing services Industrial property sales Consulting services

Land use & development Design services Land use planning Graphic design services Land use zoning Interior design services

Property management Educational services Commercial property management Professional continuing education Residential property management services

Real estate development Employment services Executive-search services Residential property Public housing Food services Residential property leasing Catering services Residential property sales Information & online services Retail property Information services Retail property leasing Online legal research Retail property sales Legal services Valuations Marketing & advertising services

RETAIL Personal services Child care services Apparel retailing see also”Apparel & accesso- Cleaning services ries” Dry cleaning & laundry services Convenience stores Household cleaning services Personal communications services Cosmetics & beauty supply stores Printing services Department stores & shopping malls Department stores Shopping malls TRAVEL, LEISURE & HOSPITALITY

Drug-based retailing see “Retail pharmaceu- Entertainment & leisure ticals” Bars & nightclubs Casinos & gambling Electronics & appliance retailing Gaming Recreational services & attractions 46 Knowl. Org. 32(2005)No.1 A. S. Chaudhry, and G. Hui Ling. Building Taxonomies Using Organizational Resources

Amusement parks Museums & galleries National parks Theme parks Zoos & aquariums Sports Golf

Hospitality Drinking places Lodgings Bed & breakfast inns Hotels & motels Resorts Restaurants & food services Catering services Fast food restaurants Restaurants

Travel & tourism To u r i s m Ecotourism Tourist information offices Tr a v e l Air fares Cruises Package tours Travel agencies Vacations

Knowl. Org. 32(2005)No.1 47 Book Review

Book Review

Edited by Clément Arsenault

Book Review Editor

BROUGHTON, Vanda. Essential Classification. that LCC doesn’t provide clearer instructions, but if New York, NY: Neal-Schuman, 2004. 324 p. ISBN 1- you keep your head and take them one step at a time 55570-507-3. [i.e. the tables] they’re fairly straightforward” (p. 174)). Vanda Broughton’s Essential Classification is the Chapters 1 to 7 present the essential theoretical most recent addition to a very small set of classifica- concepts relating to knowledge organization and to tion textbooks published over the past few years. bibliographic classification. The author is adept at The book’s 21 chapters are based very closely on the making and explaining distinctions: known-item re- cataloguing and classification module at the School trieval versus subject retrieval, personal versus pub- of Library, Archive, and Information studies at Uni- lic/shared/official classification systems, scientific versity College, London. The author’s main objec- versus folk classification systems, object versus as- tive is clear: this is “first and foremost a book about pect classification systems, semantic versus syntactic how to classify. The emphasis throughout is on the relationships, and so on. Chapters 8 and 9 discuss the activity of classification rather than the theory, the practice of classification, through content analysis practical problems of the organization of collections, and subject description. A short discussion of diffi- and the needs of the users” (p. 1). This is not a theo- cult subjects, namely the treatment of unique con- retical work, but a basic course in classification and cepts (persons, places, etc.) as subjects seems a little classification scheme application. For this reviewer, advanced for a beginners’ class. who also teaches “Classification 101,” this is also a In Chapter 10, “Controlled indexing languages,” fascinating peek into how a colleague organizes con- Professor Broughton states that a classification tent and structures her course. “Classification is eve- scheme is truly a language “since it permits commu- rywhere” (p. 1): the first sentence of this book is also nication and the exchange of information” (p. 89), a one of the first statements in my own course, and statement with which this reviewer wholly agrees. Professor Broughton’s metaphors – the supermarket, Chapter 11, however, “Word-based approaches to re- canned peas, flowers, etc. – are those that are used by trieval,” moves us to a different field altogether, of- our colleagues around the world. fering only a narrow view of the whole world of con- The combination of tone, writing style and controlled indexing languages such as thesauri, and pre- tent display are reader-friendly; they are in fact what senting disconnected discussions of alphabetical fil- make this book remarkable and what distinguishes it ing, form and structure of subject headings, modern from more “formal” textbooks, such as The Organi- developments in alphabetical subject indexing, etc. zation of Information, the superb text written and re- Chapters 12 and 13 focus on the Library of Congress cently updated (2004) by Professor Arlene Taylor Subject Headings (LCSH), without even a passing (2nd ed. Westport, Conn.: Libraries Unlimited, reference to existing subject headings lists in other 2004). Reading Essential Classification, at times, feels languages (French RAMEAU, German SWK, etc.). If like being in a classroom, facing a teacher who as- it is not surprising to see a section on subject head- sures you that “you don’t need to worry about this at ings in a book on classification, the two subjects be- this stage” (p. 104), and reassures you that, although ing taught together in most library schools, the loca- you now spend a long time looking for things, “you tion of this section in the middle of this particular will soon speed up when you get to know the scheme book is more difficult to understand. better” (p. 137). This teacher uses redundancy in a Chapter 14 brings the reader back to classifica- productive fashion, and she is not afraid to express tion, for a discussion of essentials of classification her own opinions (“I think that if these concepts are scheme application. The following five chapters pre- helpful they may be used” (p. 245); “It’s annoying sent in turn each one of the three major and cur- 48 Knowl. Org. 32(2005)No.1 Book Review

rently used bibliographic classification schemes, in time they are used in the text. A short, very short, order of increasing complexity and difficulty of ap- annotated bibliography of standard classification plication. The Library of Congress Classification textbooks and of manuals for the use of major classi- (LCC), the easiest to use, is covered in chapters 15 fication schemes is provided. A detailed 11-page in- and 16. The Dewey Decimal Classification (DDC) dex completes the set of learning aids which will be deserves only a one-chapter treatment (Chapter 17), useful to an audience of students in their effort to while the functionalities of the Universal Decimal grasp the basic concepts of the theory and the prac- Classification (UDC), which Professor Broughton tice of document classification in a traditional envi- knows extremely well, are described in chapters 18 ronment. and 19. Essential Classification is a fine textbook. How- Chapter 20 is a general discussion of faceted clas- ever, this reviewer deplores the fact that it presents sification, on par with the first seven chapters for its only a very “traditional” view of classification, with- theoretical content. Chapter 21, an interesting last out much reference to newer environments such as chapter on managing classification, addresses down- the Internet where classification also manifests itself to-earth matters such as the cost of classification, the in various forms. In Essential Classification, books need for re-classification, advantages and disadvan- are always used as examples, and we have to take the tages of using print versions or e-versions of classifi- author’s word that traditional classification practices cation schemes, choice of classification scheme, gen- and tools can also be applied to other types of docu- eral versus special scheme. But although the ques- ments and elsewhere than in the traditional library. tions are interesting, the chapter provides only a very Vanda Broughton writes, for example, that “Subject general overview of what appropriate answers might headings can’t be used for physical arrangement” (p. be. 101), but this is not entirely true. Subject headings To facilitate reading and learning, summaries are can be used for physical arrangement of vertical files, strategically located at various places in the text, and for example, with each folder bearing a simple or always before switching to a related subject. Profes- complex heading which is then used for internal or- sor Broughton’s choice of examples is always inter- ganization. And if it is true that subject headings esting, and sometimes even entertaining (see for ex- cannot be reproduced on the spine of [physical] ample “Inside out: A brief history of underwear” (p. books (p. 93), the situation is certainly different on 71)). With many examples, however, and particularly the World Wide Web where subject headings as those that appear in the five chapters on classifica- metadata can be most useful in ordering a collection tion scheme applications, the novice reader would of hot links. have benefited from more detailed explanations. On The emphasis is also on the traditional paper- page 221, for example, “The history and social influ- based, rather than on the electronic version of classi- ence of the potato” results in this analysis of con- fication schemes, with excellent justifications of cepts: Potato – Sociology, and in the UDC class course. The reality is, however, that supporting or- number: 635.21:316. What happened to the “history” ganizations (LC, OCLC, etc.) are now providing aspect? Some examples are not very convincing: in great quality services online, and that updates are Animals RT Reproduction and Art RT Reproduction now available only in an electronic format and not (p. 102), the associative relationship is not appropri- anymore on paper. E-based versions of classification ate as it is used to distinguish homographs and would schemes could be safely ignored in a theoretical text, do nothing to help either the indexer or the user at but they have to be described and explained in a the retrieval stage. textbook published in 2005. Essential Classification is also an exercise book. One last comment: Professor Broughton tends to Indeed, it contains a number of practical exercises use the same term, “classification” to represent the and activities in every chapter, along with suggested process (as in classification is grouping) and the tool answers. Unfortunately, the answers are too often (as in constructing a classification, using a classifica- provided without the justifications and explanations tion, etc.). Even in the Glossary, where classification that students would no doubt demand. is first well-defined as a process, and classification The author has taken great care to explain all scheme as “a set of classes …”, the definition of clas- technical terms in her text, but formal definitions are sification scheme continues: “the classification con- also gathered in an extensive 172-term Glossary; ap- sists of a vocabulary (…) and syntax…” (p. 296– propriately, these terms appear in bold type the first 297). Such an ambiguous use of the term classifica- Knowl. Org. 32(2005)No.1 49 Book Review

tion seems unfortunate and unnecessarily confusing in an otherwise very good basic textbook on categorization of concepts and subjects, document organization and subject representation.

M. Hudon

Dr. Michèle Hudon, Associate Professor, École de bibliothéconomie et des sciences de l’information, Université de Montréal, Montréal, Québec, H3C 3J7, Canada. E-mail: [email protected].

50 Knowl. Org. 32(2005)No.1 Knowledge Organization Literature

Knowledge Organization Literature Gerhard Riesthuis: Literature Editor

0 Form division In: Journal of the Medical Library Association, 93(2005)1, p.133. 02 Literature reviews 0006 07.1; 111 See 0001, 0004, 0005, 0008-0010, 0062 Lancaster, F.W. - Indexing and abstracting in theory and practice (Lang.: eng). - 3. ed.. - London : Facet, 2003. - 464 03 Glossaries, Vocabularies, Terminologies in p.. - ca 500 refs.. - ISBN 1-85604-482-3 Knowledge Organization »» Covers: indexing principles and practice; pre- coordinate indexes; consistency and quality of indexing; 0001 031; 02 types and functions of abstracts; writing an abstract; Hudon, M. - (Book review of) Satija, M.P.: A Dictionary if evaluation theory and practice; approaches used in index- knowledge organisation. - Amritsar, India : Guru Namak ing and abstracting services; indexing enhancement; natu- Dev University, 2004. - 248 p. - ISBN 81-7770-101-0 ral language in information retrieval; indexing and ab- (Lang.: eng). - In: Knowledge Organization, 31(2004)3, stracting of imaginative works; databases of images and p.196-198 sound; automatic indexing and abstracting; the future of indexing and abstracting services 06 Conference Reports & Proceedings 0007 07.21 0002 06.04-07-13/16; 926 Broughton, V. - Essential classification (Lang.: eng). - New Williamson, N.J. - Report : Eighth International ISKO Con- York : Neal-Schuman, 2004. - 272 p. - 20 refs. - ISBN 1- ference, London, UK, 13-16 July, 2004 (Lang.: eng). - In: 555-70507-3 Knowledge Organization, 31(2004)3, p.188-195. - 1 refs. 0008 07.21; 02 0003 06.2001-03-21/23 Poulter, A. - (Book review of) Broughton, V.: Essential Lehner, C., Ohly, H.P., Rahmstorf, G. - Wissensorganisati- Classification. - London : Facet Publishing, 2004. - 272p. - on und Edutainment: Wissen in Spannungsfeld von Gesell- ISBN: 1-85604-514-5 (Lang.: eng). - In: Center for infor- schaft, Gestaltung und Industrie : Proceedings der 7. Tagung mation and computer sciences - book reviews [electron.] der Deutschen Sektion der Internationalen Gesellschaft für * http://www.ics.ltsn.ac.uk/books/reviewed/Essential_ Wissensorganisation, Berlin 21. - 23. März 2001/ Hrsg von Classification.htm C. Lehner, H.P. Ohly, G. Rahmstorf. [Knowledge organization and Edutainment: proceedings of the 7th meeting of 0009 07.21; 02 the German Section of the International Society for Smith, N. - (Book review of) Batley, S.: Classification in Knowledge Organization, Berlin, 21 23 March 2001] theory and practice. - Chandos Publishing, 2003. - 200 p. - 1- (Lang.: ger). - Würzburg : Ergon Verlag, 2004. - xii, 283 p.. 8433-4038-6 (Lang.: eng). - In: Center for information and - (Fortschritte in der Wissensorganisation) ; Band 7). - computer sciences - book reviews [electron.] ISBN 3-89913-410-9 * http://www.ics.ltsn.ac.uk/books/reviewed/Classification_ * For the separate papers, see 0012, 0014-0016, 0022, 0053, in_theory_and_practice.htm 0054, 0061, 0063, 0073, 0074, 0077, 0078, 0082, 0095, 0096, 0102, 0104-0106, 0139 0010 07.75; 02; 751 Smith, D.G. - (Book review of) Bourne, C.P., Berlando 07 Textbooks in Knowledge Organization Hahn, T.: A history of online information services, 1963- 1976. - Cambridge, MA: MIT Press, 2003. - 492 p. - ISBN 0004 07.1; 02; 11 0-262-02538-8 (Lang.: eng). - In: Journal of the American Halsall, J. - (Book review of) Lancaster, F.W.: Indexing and Society for Information Science and Technology, abstracting in theory and practice. - 3rd ed. - London: Face, 55(2004)7, p.652-653 2003. - 464 p. - ISBN 1-85604-482-3 (Lang.: eng). - 3. ed.. - In: Learned publishing, 17(2004)2, p. 172 0011 07.75; 751 Bourne, C.P., Berlando Hahn, T. - A history of online in- 0005 07.1; 02; 11 formation services, 1963-1976 (Lang.: eng). - Cambridge, Lingle, V.A. - (Book review of) Lancaster, F.W.: Indexing MT : MIT Press, 2003. - 493 p. . - Refs.. - ISBN 0-262- and abstracting in theory and practice. - 3rd ed. - London: 02538-8 Face, 2003. - 464 p. - ISBN 1-85604-482-3 (Lang.: eng). - Knowl. Org. 32(2005)No.1 51 Knowledge Organization Literature

1 Theoretical Foundations and General Problems 0019 149 Metaxiotis, K., Ergazakis, K., Psarras, J. - Exploring the 11 Order and Knowledge Organization world of knowledge management : agreements and disagree- ments in the academic/practitioner community (Lang.: eng). See 0004-0006, 0033 - In: Journal of Knowledge Management, 9(2005)2, p. 6- 18. - 13 refs 12 Conceptology in Knowledge Organization 0020 149 0012 124 Call, D. - Knowledge management - not rocket science Figge, U.L. - Technische Anleitungen und der Erwerb kohä- (Lang.: eng). - In: Journal of Knowledge Management, renten Wissens. [Technical manuals and the acquisition of 9(2005)2, p. 19-30. - 19 refs coherent knowledge] (Lang.: ger). - In: Wissensorganisa- tion und Edutainment [see 0003], p.116-122. - 6 refs. 0021 149 Branin, J.J. - Knowledge Management in Academic Libraries 14 Systens Theory in Knowledge Organization : Building the Knowledge Bank at the Ohio State University. (Lang.: eng). - In: Journal of Library Administration, See also 0031, 0053, 0054, 0095 39(2003)4, p.41-56. - Refs

0013 146 16 Science and Knowledge Organization Gnoli, C., Poli, R. - Levels of reality and levels of representation (Lang.: eng). - In: Knowledge Organization, 0022 161 31(2004)3, p.151-160. - 45 refs. Budin, G. - Wissensorganisation als Gestaltungsprinzip virtuellen Lernens - epistemische, kommunikative und methodi- 0014 149 sche Anforderungen. [Knowledge organization as design Swertz, C. - Webdidaktik : effiziente Inhaltsproduktion für principle of virtual learning - epistemic, communicative netzbasierte Trainings. [Web didactics: efficient production and methodic requirements] (Lang.: ger). - In: Wissensor- of content for network based training] (Lang.: ger). - In: ganisation und Edutainment [see 0003], p.39-48. - 15 refs. Wissensorganisation und Edutainment [see 0003], p.49-64. - 17 refs. 18 Classification and Indexing Research

0015 149 0023 181; 191 Jorna, R. - Organizational forms and knowledge types López-Huertas, M.J., Jiménez Contreras, E. - Spanish re- (Lang.: eng). - In: Wissensorganisation und Edutainment search in knowledge organization : (1992-2001) (Lang.: [see 0003], p.149-159. - 27 refs eng). - In: Knowledge Organization, 31(2004)3, p.136-150. - 15 refs. 0016 149 Bosch-Sijtsema, P. - The virtual organisation and knowledge 19 History of Knowledge Organization dev elopments : a case of expectations (Lang.: eng). - In: Wissensorganisation und Edutainment [see 0003], p.160- See 0023 170. - 22 refs 2 Classification Systems and Thesauri, Structure 0016a 149 and Construction Gerstheimer, O., Lupp, C. - Systemdesign - Wissen um den Menschen : Bedürfnisorientierte Produktenentwicklung im 21 General Questions of CS & T Mobile Business. [Systemdesign - Knowledge about the people: Development of new products in the mobile tele- 0024 211 phon business based on wishes of customers] (Lang.: ger). Fox, R. - Cataloging our information architecture (Lang.: - In: Wissensorganisation und Edutainment [see 003], eng). - In: OCLC Systems and Services, 21(2004)1, p.23- p.135-140. - 13 refs. 29. - 7 refs

0017 149 0025 211 Herschel, R.T., Nemati, H., Steiger, D. - Tacit to explicit Nicolaisen, .J., Hjorland, B., - A rejoinder to Beghtol knowledge conversion : knowledge exchange protocols (2004) (Lang.: eng). - In: Knowledge Organization, (Lang.: eng). - In: Journal of Knowledge Management, 30(2003)3, p.199-201. - 6 refs. 5(2001)1, p.107-116. - Refs 22 Structure and elements of CS & T 0018 149 Taylor, H. - A Critical Decision Interview Approach to Cap- 0026 221 turing Tacit Knowledge : Principles and Application (Lang.: Will, L. - Taxonomy: Classification by Another Name eng). - In: International Journal of Knowledge Manage- (Lang.: eng). - In: Legal Information Management, ment, 1(2005)3, p.25-39. - Refs 4(2004)2, p.125-130 . - Refs. 52 Knowl. Org. 32(2005)No.1 Knowledge Organization Literature

0026a 221 3 Classing and Indexing (Methodology) Zerweck, P., Lupp, C. - Mehrdimensionale Ordnungssysteme im virtuellen Raum anhand eines 31 Theory of Classing and Indexing. (Methodology) Desktops. [Multidimensional ordering systems in the virtual space on the basis of a Desktop system as example] 0033 311; 113; 922 (Lang.: ger). - In: Wissensorganisation und Edutainment Coleman, A.S. - A Code for Classifiers : Whatever happened [see 0003], p.141-146 to Merrill's Code (Lang.: eng). - In: Knowledge Organiza- tion, 31(2004)3, p.161-176. - 32 refs. 23 Construction of CS & T »» On the work of William Stetson Merrill

0027 235 32 Subject Analysis Amaeshi, B. - Subject indexing in the Nigerian National Corporation Library : design of a instrument (Lang.: eng). - 0034 321; 754 In: Library Review, 50(2001)9, p.457-460. - Refs. Borlund, P. - The Concept of Relevance in IR (Lang.: eng). - In: Journal of the American Society for Information Sci- 0028 235 ence and Technology, 54(2003)10, p.913-925. - Refs. Nielsen, M.L. - A framework for work task based thesaurus design (Lang.: eng). - In: Journal of Documentation, 0035 321; 758-82 57(2001)6, p.774-497. - Refs. Menne-Haritz, A. - Wissensmanagement und Archive : An- gebote der Archivwissenschaft für ein neues Wissenskonzept. 0029 237 [Knowledge management and archives: offers of archival Schneider, J.W., Borlund, P. - A Bibliometric-based semi- sciences to the new concepts of knowledge] (Lang.: ger). - automatic approach to identification of candidate thesaurus In: Archivar, 54(2001)4, p.303-309. - Refs terms : Parsing and filtering of noun phrases from citation contexts (Lang.: eng). - In: Context; Nature, Impact and 0036 323 Role. 5th International Conference on Conceptions of Li- Sauperl, A. - Catalogers' common ground and shared knowl- brary and Information Sciences, CoLIS 2005 Glasgow, edge (Lang.: eng). - In: Journal of the American Society for UK, June 2005 / ed. by F. Crestani, I. Ruthven, I. - Ber- Information Science and Technology, 55(2004)1, p.55 - 63. lin/Heidelberg : Springer Verlag, 2005, p.226-237.. - Refs. - Refs.

24 Relationships 0037 323 Mai, J.E. - Analysis in indexing : document and domain cen- 0030 246; 754 tered approaches (Lang.: eng). - In: Information Processing Wilbur, W.J. - Global term weights for document retrieval and Management, 41(2005), p.599-611. - Refs. learned from TREC data (Lang.: eng). - In: Journal of In- formation Science, 27(2001)5, p.303-310. - Refs. 34 Classing and Indexing (see also 81)

25 Numerical Taxonomy 0038 348 Kwon, O.W., Lee, J.H. - Text categorization based on k- 0031 252; 147 nearest neighbor approach for web site classification (Lang.: Bote, V.P.G., de Moya Anegon, ., Solana, V.H. - Document eng). - In: Information Processing and Management, organization using Kohonen's algorithm (Lang.: eng). - In: 39(2003)1, p.25-44. - Refs. Information processing and management, 38(2002)1, p.79- 89. - Refs. 0039 348 Denoyer, L., Gallinari, P. - Bayesian network model for 28 Compatibility and Concordances between Indexing semi-structured document classification (Lang.: eng). - In: Languages Information Processing and Management, 40(2003), p.807- 827. - 30 refs 0032 281 Wake, S., Nicholson, D. - HILT : subject access across do- 0040 348; 757 mains (Lang.: eng). - In: SCONUL Newsletter, (2001)23, Miyamoto, S. - Information clustering based on fuzzy mul- p.14-18. - Refs. tisets (Lang.: eng). - In: Information Processing and Man- agement, 39(2003)2, p.195-213. - 16 refs

0041 349 Figuerola, C., Rodriquez, A.F.Z., Berrocal, J.L.A. - Auto- matic vs. manual categorisation of documents in Spanish (Lang.: eng). - In: Journal of Documentation, 57(2001)6, p.763-773. - Refs.

Knowl. Org. 32(2005)No.1 53 Knowledge Organization Literature

35 Manual and Automatic Order Techniques 6 On Special Subjects Classifications and Thesauri

0042 357 62 On Classification Systems and Thesauri in Physics, Cantara, L. - The text-encoding initiative (Lang.: eng). - In: Chemistry, Electronics, Energy OCLC Systems and Services, 21(2004)1, p.36-39; 2, p.110- 113. - 8 refs 0048 626 Fugmann , R. - Learning the Lessons of the Past (Lang.: 39 Evaluation of Classing and Indexing eng). - In: The History and Heritage of Scientific and Technological Information Systems: Proceedings of the 0043 391 2002 Conference / ed. by W. Boyd Rayward and Mary Della Mea, V., Mizzaro, S. - Measuring retrieval effectiveness Ellen Bowden. - Information Today, Inc., Medford, New : a new proposal and a first experimental validation (Lang.: Jersey, 2004. - (ASIS & Monograph Series), p.168-181. - 54 eng). - In: Journal of the American Society for Informa- refs. tion Science and Technology, 55(2004)6, p.530-543. - Refs. * http://www.chemheritage.org/events/asist2002/ 14-fugmann .pdf 4 On Universal Classification Systems and Thesauri 0049 626 Davis , C.H. - Indexing and Index Editing at Chemical Ab- 43 On the Dewey Decimal Classification stracts before the Registry System (Lang.: eng). - In: The History and Heritage of Scientific and Technological In- See also 0065 formation Systems: Proceedings of the 2002 Conference / ed. by W. Boyd Rayward and Mary Ellen Bowden. - Infor- 0044 43 mation Today, Inc., Medford, New Jersey, 2004. - (ASIS & Heiner-Freiling, .M. - DDC Deutsch 22 - formale, termino- T Monograph Series), p.182-189. - 19 refs. logische und inhaltliche Aspekte einer deutschen DDC- * http://www.chemheritage.org/events/asist2002/ Ausgabe. [DDC German 22 - formal, terminological and 15-davis.pdf subject aspects of a German DDC edition] (Lang.: ger). - In: Zeitschrift für Bibliothekswesen und Bibliographie, 0050 626; 855 48(2001)6, p.333-339. - Refs. Lynch, M.F. - Introduction of Computers in Chemical Struc- ture Information Systems, or What Is Not Recorded in the 0045 43 Annals (Lang.: eng). - In: The History and Heritage of Sci- Heiner-Freiling, .M. - Die DDC in der Deutsche National- entific and Technological Information Systems: Proceed- bibliographie. [The DDC in the German National Biblio- ings of the 2002 Conference / ed. by W. Boyd Rayward and graphy] (Lang.: ger). - In: Dialog mit Bibliotheken, Mary Ellen Bowden. - Information Today, Inc., Medford, 15(2003)3, p.8-13 New Jersey, 2004. - (ASIS & Monograph Series), p.137- * Also: http://www.ddc-deutsch.de/literature/Heiner-Freiling 148. - 103 refs. _3_2003.pdf * http://www.chemheritage.org/events/asist2002/ 12-lynch.pdf 0046 43 Bee, .G. - Abschlussworkshop zu Project DDC Deutsch. 65 On Classification Systems and Thesauri in Human [Closing workshop of the Project DDC German] (Lang.: Biology, Medicine, Psychology ger). - In: Dialog mit Bibliotheken, 17(2003)7, p.3-6 * Also: http://www.ddc-deutsch.de/literature/2005_2bee.pdf See 0094

0047 43 66 On Classification Systems and Thesauri in Sociology, Gödert, W. - Die Welt ist groß - Wir bringen Ordnung in Politics, Social Policy, Law, Military Science, History diese Welt : Das DFG Projekt DDC Deutsch. [The World is big - we bring order in this world: The DFG Project DDC See 0063 German] (Lang.: ger). - In: Information, Wissenschaft und Praxis, 53(2002), p.395-400 68 On Classification Systems and Thesauri in Science of * Also: http://www.fbi.fh-koeln.de/institut/projekte/ddc/ Science, Information Science, Computer Science, Com- DDCde/DDC_Deutsch_nfd.pdf munication Science, Semiotics

48 On other Universal Classification Systems and 0051 682 Thesauri Holsapple, C.W., Joshi, K.D. - A formal knowledge management ontology : Conduct, activities, resources, and influ- See 0068 ences (Lang.: eng). - In: Journal of the American Society for Information Science and Technology, 55(2004)7, p.593- 612. - Refs.

54 Knowl. Org. 32(2005)No.1 Knowledge Organization Literature

7 Knowledge Representation by Language and 0058 752 Terminology Li, L.C. - Online cultural heritage exhibitions: a survey of information retrieval features (Lang.: eng). - In: Program: 71 General Problems of Natural Language in Relation to electronic library and information systems, 39(2004)1, p.4- Knowledge Organization 24. - 21 refs.

0052 714 0059 752; 947 Thellefsen, T.L., Thellefsen, M.M. - Pragmatic semiotics French, J.C., Chapin, A.C., Martin, W.N. - Multiple view- and knowledge organization (Lang.: eng). - In: Knowledge points as an approach to digital library interfaces (Lang.: Organization, 31(2004)3, p.177-187. - 18 refs. eng). - In: Journal of the American Society for Informa- »» On the work of William Stetson Merrill tion Science and Technology, 55(2004)10, p.911-922. - Refs. 72 Semantics 0060 752.2 0053 725; 149 Hogg, M., Field, J. - Using Z39.50 to build a virtual union Smolnik, S., Nastansky, L. - K-Discovery : Identifikation catalogue Music Libraries Online : a subject clump (Lang.: von verteilten Wissensstrukturen in einer prozessorientierten eng). - In: Catalogue and Index, (2001)139, p.1-4. - Refs. Groupware-Umgebung. [K-discovery: Identification of divided knowledge structures in a process oriented group- 0061 752.3 ware environment] (Lang.: ger). - In: Wissensorganisation Quandt, M. - Sowinet.de - ein Internetjournal mit Berichten und Edutainment [see 0003], p.171-184. - 13 refs aus den Sozialwissenschaften : Ziele und Konzept einer neu- en Informationsplattform . [Sowinet.d - an Internet Journal 0054 725; 149 with reports from the social sciences] (Lang.: ger). - In: Rahmstorf, G. - Strukturierung von inhaltlichen Daten : To- Wissensorganisation und Edutainment [see 0003], p.247- pic maps und Concepto. [Structuring of data concerning the 263. - 15 refs. content: Topic maps and Concepto] (Lang.: ger). - In: Wissensorganisation und Edutainment [see 0003], p.194- 0062 752.3; 02 205. - 2 refs Bellarby, L. - (Book review of) Chowdhury, G.G.: Introduc- tion to modern information retrieval. - 2. Ed. - London: 75 On-Line Retrieval System and Technology Facet Publishing, 2003. - 480 p. - ISBN 1-85604-480-7 (Lang.: eng). - In: Center for information and computer 751 General and Theoretical Problems sciences - book reviews [electron.] * http://www.ics.ltsn.ac.uk/books/reviewed/075.htm See also 0010, 0011 0063 752.3; 66 0055 751 Ohly, H.P. - Gestaltungsprinzipien bei sozialwissenschaftli- Wildemuth, B.M. - The effects of domain knowledge on chen Wissensportalen im Internet. [Design principles of search tactic formulation (Lang.: eng). - In: Journal of the Internet portals for the Social Sciences] (Lang.: ger). - In: American Society for Information Science and Technology, Wissensorganisation und Edutainment [see 0003], p.234- 55(2004)3, p.246-258. - Refs. 246. - Refs.

0056 751 753 On-line access, query optimization, full text search- Bates, M.J. - After the Dot-Bomb : Getting Information Re- ing trieval Right This Time (Lang.: eng). - In: Hastings Re- search [electron.]. - 21 refs 0064 753 * http://www.hastingsresearch.com/net/08-net-information- Radev, D., Fan, W.G., Qi, H., Wu, H., Grewal, A. - Prob- retrieval.shtml#1 abilistic question answering on the Web (Lang.: eng). - In: Journal of the American Society for Information Science 0057 751 and Technology, 56(2005)6, p.571-583. - Refs. Van Rijsbergen, K. - The geometry of information retrieval (Lang.: eng). - Cambridge : Cambridge University Press, 0065 753; 43 2004. - xii, 150 p.. - Ca 200 refs. Svensson, L.G. - Sacherschliessung als Basis für intelligente »» Content: 1. Introduction; 2. On sets and kinds for IR; Navigation ausgehend von der DDC : Konzepte - Realisie- 3. Vector and Hilbert spaces; 4. Linear transformations, rung - Visionen. [Subject indexing as basis for navigation operators and matrices; 5. Conditional logic in IR; 6. The using the DDC: concepts - realization - visions] (Lang.: geometry of IR. ger). - In: Bibliotheksdienst, 38(2004)10, p.1283-1294. - Refs. 752 Dialogue Systems. Interactive Catalogues