Disrupting the Metanarrative: a Little History of Image Indexing and Retrieval

4 Knowl. Org. 46(2019)No.1 P. Rafferty. Disrupting the Metanarrative: A Little History of Image Indexing and Retrieval

Disrupting the Metanarrative: A Little History of Image Indexing and Retrieval Pauline Rafferty Aberystwyth University, Department of Information Management, Libraries and Archives, Aberystwyth SY23 3UX,

Pauline Rafferty is Senior Lecturer at Aberystwyth University, teaching knowledge organization and representation, information architectures, and qualitative approaches to research. Her research interests include knowledge organisation and cultural documentation, and critical communication and information studies. Specific areas of interest include popular culture in and through the web, the democratisation of critical authority, and participative digital cultural production. She holds a PhD in critical theory and cultural studies. Pauline is Joint Editor of Journal of Information Science and Joint Regional Editor of The Electronic Library. She has co-edited books for Facet and co-written a book for Ashgate.

Rafferty, Pauline. 2019. “Disrupting the Metanarrative: A Little History of Image Indexing and Retrieval.” Knowledge Organization 46(1): 4-14. 83 references. DOI:10.5771/0943-7444-2019-1-4.

Abstract: The aims of this paper are twofold: to offer a short history of image retrieval, and secondly and relatedly, to critique the metanarrative of modernity emerging in the literature of knowledge organization and information retrieval. The paper reviews the emerging grand narrative in relation to knowledge organization and information retrieval that sees them as specific aspects of modernity and technological efficiency. This grand narrative is particularly interested in technology even when it is contextualising technology. A more nuanced history emerges when the focus moves to the representation, organization, and retrieval of images. This literature foregrounds not only the technology but also issues relating to definitions of the “subject” and issues relating to interpretation and meaning- making.

Received: 18 September 2018; Revised: 26 November 2018; Accepted: 10 December 2018

Keywords: information systems, image retrieval, image indexing, images

1.0 The technological grand narrative the broader metanarrative of modernity, so that modern information management is characterised as emerging In the introduction to their special issue on the history of from a worldview that privileges facts and science, assumes information systems, Bryant et al. (2013, 1) remind us that stability in textual meaning, and believes in the disinter- histories are made as much as they are discovered and refer ested and objective authority of the professional indexer. back to Carr (1961) and Collingwood’s (1994) view that Such histories point to the pursuit of facts in documents that the relationship between the past itself and the histo- that underpins Paul Otlet’s work (Rayward 2014), the no- rian’s thought is key. There is no one definitive and author- tion of the informational unit, comprising facts and opin- itative information systems history, “on the contrary, the ions that unpins the systematic indexing of Julius Kaiser process of articulating the history of IS needs to be a (Dousa 2007), Suzanne Briet and the post Second World wide-ranging, continuous effort, encompassing different War documentalists, and the development of computer- perspectives and agendas.” That the history of infor- ised information retrieval and the distributed power of the mation science should be of interest is not surprising given World Wide Web. the importance of digital information systems in our mod- Rayward (2014, 683-84), acknowledging the need for ern world. For some scholars working in critical infor- simplification to impose [narrative] order, identifies three mation science contexts (e.g., Day 2008), it is the discursive “information revolutions:” the Gutenberg revolution, the formations of the present that become the drivers toward pre-digital post-Second World War age, and the digital age. a critical interrogation of generally accepted stories of the Whilst emphasising the driver technologies for each of past. Recently there has been considerable scholarly inter- these revolutions, Rayward contextualises them with refer- est in historicising and critiquing information with reference to the broader socio-economic parameters within ence to the metanarrative of modernity (see for example which they have been designed, developed and used. Print Day 2008 and Hayles 2008). In this retelling, information technologies developed within the emerging capitalism of science and information systems are contextualised within Europe and the systems that were designed to the manage

https://doi.org/10.5771/0943-7444-2019-1-4 Generiert durch IP '170.106.202.8', am 03.10.2021, 02:33:44. Das Erstellen und Weitergeben von Kopien dieses PDFs ist nicht zulässig. Knowl. Org. 46(2019)No.1 5 P. Rafferty. Disrupting the Metanarrative: A Little History of Image Indexing and Retrieval information were responding to the needs of competitive mation technology facilitates access to knowledge, at the companies, organisations and governments who looked to same time humans are becoming known “subjects” in and expand while at the same time they sought to communi- through retrieval systems (see, for example, debates about cate with others on an international level. Information, Facebook and its methods of collecting information about textual, graphical and statistical, delivered by print, by tel- its users in Bennett and Livingstone (2018) and Schou and egraph or other means, underpinned capitalist growth. Farkas (2016)). Hayles (2008) distinguishes between the Alongside the technological developments and the expan- humanity of the modern Enlightenment, and the posthu- sion of informational genres came information manage- man, which derives from a view that privileges an abstract ment developments such as the growth of bibliographies, idea of information and sees human identity as being es- national and professional, classification tools and the de- sentially an informational pattern. Hayles traces an histor- velopment of faceted approaches to classification. ical trajectory of this cyborgian worldview from Shannon The information revolution of the post Second World and Weaver through cybernetic theory, and in the process War period focused on scientific and technological infor- points to the skim reading mode of perception that comes mation that grew out of the activities of the war years but with the screen reading information overload of the digital shifted to commercial industrial and medical information age, arguing for the literary method of “close reading” as as well as military information. The indexing of this doc- a strategy to offset the surface of the machine. umentation was crucial and conventional solutions were Within this broad historical framework sit a variety of considered to be too slow and not fine-grained enough for knowledge organization tools designed and developed the work (692). Speed of access, precision in retrieval and within specific epistemological frameworks dominant at technical expertise in indexing were seen to be crucial ele- the cultural-historical moment of their creation and initial ments in information management. The arrival of com- development. Discussions about the great universal classi- puterised systems from the 1950s drove the design of fication schemes of the late nineteenth and early twentieth computerised indexing systems such as KWIC and centuries often include reference to modernist assump- KWOC indexes before the emergence of database systems tions regarding fact, objectivity, and science that underpin and the subsequent development of cataloguing, indexing their creation (Mai 1999 and 2004; Rafferty 2001). Special- and thesaural standards and cooperative cataloguing initi- ist indexing tools such as professional, domain specific atives. thesauri developed to support the increasing specialisation The third, digital revolution, is a revolution of ubiquity, and commercialisation of information post-1945 (for an as access to digital information is possible through a pleth- historical overview, see Rowley 1994 and Sanderson and ora of devices accessible on the go to anyone who can af- Croft 2012). In addition, the development of thesauri of- ford to access them and knows how to use them. Docu- fered a solution at a time when interdisciplinary research ment creation and reproduction techniques have been threatened to make the rigid discipline-oriented structures freed from the restraints of older technologies, and docu- of the classification scheme seem obsolete or at least cum- ment creation in the public sphere of the web is now open bersome, and they offered a way of reconciling subject to the general public without the quality constraints of headings and classification schemes that makes it easier to conventional publishing (704). New communicative gen- switch between both (Garcia-Marco 2016, 6). res such as blogs, emails and tweets have developed, while Within this metanarrative, social tagging and user gen- social networking sites such as Facebook, Instagram, erated content are sometimes seen as postmodern and dis- YouTube and Flickr allow for the creation and dissemina- ruptive approaches to knowledge representation. In this tion of non-textual documents such as images and music. story, social tagging has the potential to unlock the eman- There is some debate about whether the changes that have cipatory possibilities of digital technologies and systems come with the digital age point to some fundamental shifts and counteract the potential gloom of technological tyr- towards a post-modern, surveillance driven digital eco- anny and the indexing of the subject. Champions (e.g., nomic age or whether these shifts, while clearly significant Kroski 2005; Shirky 2005; Merholz 2004) laud the flexible, in their velocity, their convergence and their technologies, participative and collaborative nature of social tagging, are essentially a continuation of the information age in that which is democratic (Rafferty and Hidderley 2007) in that all societies and human ages have been, and are infor- it involves all users, and emergent in that the tags can mation societies (704-705). change rapidly in response to new content (Feinberg That, broadly and generally speaking, is the story. There 2006). Early proponents of social tagging took inspiration are variations on the story, such as Day’s Indexing it All from Surowiecki’s (2005) notion of the “hive mind” or the (2014), that take a slightly darker view about issues regard- “wisdom of crowds” or “social intelligence” to explain the ing ubiquity and the human “subject” in relation to infor- advantages and richness that they claimed for social tag- mation technology. In this darker version, while infor- ging. The idea is that the combined intelligence of a group

https://doi.org/10.5771/0943-7444-2019-1-4 Generiert durch IP '170.106.202.8', am 03.10.2021, 02:33:44. Das Erstellen und Weitergeben von Kopien dieses PDFs ist nicht zulässig. 6 Knowl. Org. 46(2019)No.1 P. Rafferty. Disrupting the Metanarrative: A Little History of Image Indexing and Retrieval of people will be greater than the knowledge of an indi- ing the course of the late twentieth century and early vidual, even an expert individual. Although even early on twenty-first century, content-based retrieval solutions were in its history there were critics of tagging (see for example, developed for managing images. Content-based systems Kroski 2005; Guy and Tonkin 2006; Rafferty and Hidder- often focus on the retrieval of specific images, a large set ley 2007), in this story, tagging is seen as inclusive, incor- of images stored in a digitised database by interrogation porating no imposed cultural or political bias; its language using some form of indexical surrogate, often a specific is current, fluid and capable of incorporating terminology attribute, for example shape, colour, texture. Historically, and neologisms (Garcia-Marco 2016); it is non-binary, issues relating to ambiguity and human interpretation were democratic and self-moderating, follows desire lines challenges that early research programs had to overcome, (Mathes 2004) and engenders community. to bury, or better still, avoid altogether by focusing research on images whose function is monologic in nature, 2.0 An image retrieval story in enquiry situations that are relatively unambiguous systems for retrieving logos, maps, images of textiles (Hid- Image retrieval has its own technologically-oriented story, derley and Rafferty 2007). which moves from domain specific thesauri through to Social media and mobile computing technology have content-based information retrieval (see Benson 2015 for developed at such a rate that there has been a tremendous an historical overview of some of the issues relating to growth in the availability of online images, and this devel- image indexing). In this story, information retrieval solu- opment has been accompanied by an increasing interest in tions in the pre-digital library environment generally de- designing smart content-based systems that can incorpo- rived from the development of specialist knowledge or- rate semantic search (see for example Zhu et al. 2017, and ganization tools such as the Library of Congress Thesau- Ristoski and Paulheim 2016 for overviews), and while the rus for Graphic Materials (LCTGM)( https://www.loc. discourse of user-based indexing often foregrounds de- gov/rr/print/tgm1/), the Getty Institute’s Art and Archi- mocracy and user emancipation, approaches to post-hoc tecture Thesaurus (AAT) and the Netherlands Institute for disciplining of tagging practice remain popular (see for ex- Art History’s Iconclass (http://www.iconclass.nl/home), ample tag recommender systems, and the display of “in- tools that grew out of the pre-digital, post-Second World teresting” or trending tags reported by Dubinko et al. War golden age of specialist controlled vocabularies and 2007). Related approaches include incorporating user be- thesauri. The LCTGM, which started in 1980, was a con- haviour into image retrieval systems, for example, brows- solidation project drawing together subject terms that had ing behaviour (Trevisiol et al. 2012), sentiment analysis been used for fifty years in the division’s manual files and (Chen et al. 2014) and click-through behaviour (Pan et al. the local subject headings lists. Its development coincided 2014). Computational approaches to the challenges of with the publication of the ANSI Guidelines for Thesaurus very large web-based image repositories have also included Structure, Construction and Use and the thesaurus software the development of computational aesthetics solutions, package, Lexico (Alexander and Meehleib 2001). Work on though the developers of such systems differentiate be- the Art and Architecture Thesaurus began in the late 1970s as tween the “true aesthetics value,” determined by genre, art libraries and art journal indexing services were comput- context and semantics of the artwork, and also possibly by erizing their catalogues. The AAT’s own history page the sophistication of the viewer, and which can perhaps (http://www.getty.edu/research/tools/vocabular- only be fully identifiable when the sample size is infinitely ies/aat/about.html#history) notes that while thesauri and large and when there is no noise in the observation, and controlled vocabularies were developed in sciences before the “observed aesthetic” of artworks, which can be ob- this point, “the use of a thesaurus for indexing was not tained from a pool of values drawn from experts and gen- welcomed by art catalogers prior to the advent of comput- eral viewers (see, for example Datta et al. 2006 and Joshi erized cataloguing.” The principles of the AAT thesaurus et al. 2011). Such systems rely on a baseline set of inter- were established by 1981. Iconclass is an iconographic clas- pretations drawn from human feedback, perhaps through sification system devised by Henri van de Waal of Leiden tagging, that could then be used to develop automated rec- University. It took forty years for this system to be devel- ommender systems. oped resulting in seventeen printed volumes published be- Overviews of the history of content-based image re- tween 1973 and 1985. trieval (CBIR) generally survey such systems before engag- So far, this story resembles the more general narrative ing with the issue of the “semantic gap” and the develop- in that specialist controlled vocabularies are created to im- ment of semantic CBIR systems (see Alzu’bi Amira and pose order on to the image information chaos, and the Ramzan 2015 for an overview). The semantic gap in image standardisation of information management allows for co- retrieval refers to the difference between the low-level in- operation and control. As computerisation developed dur- formation retrieved by a computer and the high-level se-

https://doi.org/10.5771/0943-7444-2019-1-4 Generiert durch IP '170.106.202.8', am 03.10.2021, 02:33:44. Das Erstellen und Weitergeben von Kopien dieses PDFs ist nicht zulässig. Knowl. Org. 46(2019)No.1 7 P. Rafferty. Disrupting the Metanarrative: A Little History of Image Indexing and Retrieval mantic image information understood by the user. The se- language and visual perception to analyse the levels of sig- mantic gap is interesting, because emerging early on in the nification inherent in and through pictorial images. history of computerised information retrieval, it points to Shatford (1986), drawing on Erwin Panosky’s three lev- a disruption in the story of smooth efficiency (even when els of meaning in art images, constructed a conceptual the efficiency might contain dark and sinister undertones) framework through which to identify the semantic ele- that drives the metanarrative of computerised information ments within images that might be useful in constructing management. Interpretation matters in relation to images image indexing solutions. The original conceptual frame- and the challenge of meaning-making in image retrieval work constructed by Panofsky to help with the interpreta- foregrounds more general knowledge representation con- tion of Renaissance art consists of three levels of “mean- cerns relating to hermeneutic interpretation, connotation ing,” which are: and associative meanings, and allows for an interrogation of the monologic authority of the curatorial keyword. – Primary level of meaning or sphere of phenomenal meaning: subdivided into factual (a depiction of a hu- 3.0 Images, “subjects” and phenomenological de- man being) and expressional subject matter (Panofsky scription 1993, 54). This is the pre-iconographical level of art. – Secondary level of meaning, dependent on knowledge That the “semantic gap” was recognised and drives the his- of codes, culture and conventions: identifying the male tory of the information management of images within figure in the painting with the knife as St. Bartholomew. both computer science and information science (see for This level of subject matter is called the iconographical example, Gudivada and Raghavan 1995 and Hare et al. level of art. 2006) shows the metanarrative of the efficient computer- – Intrinsic meaning or content: depends on the viewer ised retrieval system being unpacked earlier in this sub- synthesising information gathered at the first two levels field than the more general critique of monologic of meaning with additional information, which might knowledge organization tools that sometimes informs dis- include information about the artist and the socio-po- cussions of web-based digital information management litical cultural moment of production. Achieving icon- (see for example, Shirky 2005). The recognition of the se- ological interpretation depends on “synthetic intui- mantic gap points to the telling of another story focused tion,” an attribute which might be more often found in on subject representation, phenomenal description and in- the talented layman than the erudite scholar (Rafferty terpretation. In this story, it is perhaps useful to distinguish 2011, 283). between the knowledge-based linguistic written signs that have traditionally been the focus of conventional The iconological has sometimes been interpreted as the knowledge organisation research and development, and subjective element in the interpretation of images (see for other kinds of signs that are not necessarily knowledge- example, Enser 2000). Panofsky, however, relates the icon- based signs, in relation to which semantic and affective ological to the broader socio-economic context, the logo- meanings might be more open to reader, listener or viewer nomic parameters in social semiotic terms (Hodge and interpretation. The image retrieval disruption comes with Kress 1988), or psychological interpretations about the the acknowledgement that the interpretation of visual artist, which suggests that Panofsky envisages certain signs and images is not quite the same as the interpretation kinds of viewers or “readers,” whether scholars or talented of textual signs and language, particularly when the visual laymen, undertaking the interpretation of the Renaissance signs are not accompanied by text to anchor the meaning art image. Some of this assumption about the indexer as of the image. expert or talented reader underpins the literature of image The issue of subject indexing in relation to images subject indexing that followed Shatford-Layne, Enser and starts to become a significant one in the literature from the Jörgensen’s leads (see, for example, Bohnsack 2008). 1980s onwards, with writers such as Michael Krause (1988) Shatford interprets Panofsky’s pre-iconographic level as and Sara Shatford Layne (1994) exploring meaning and in- “generic of;” the iconographic level is treated as “specific terpretation in relation to images, Krause distinguishing of;” and the iconological level is treated as “about” and between “hard meaning” or what is observed in a picture, includes four facets of indexing description (who, what, and “soft meaning,” or the subjective meaning, while Shat- where and when) to produce a matrix of indexing possi- ford Layne distinguished between “ofness” and bilities (Shatford 1986, 43). Armitage and Enser (1997, “aboutness.” Shatford Layne, who has played a major role 287) adapted the framework to include four main catego- in exploring issues around the interpretation and significa- ries (who, what, where and when), and three levels of ab- tion of images in the specific context of image indexing, straction: generic (pre-iconographic), specific (icono- draws on the literature of philosophy of art, meaning in graphic) and abstract (iconological). Collins (1998) argued

https://doi.org/10.5771/0943-7444-2019-1-4 Generiert durch IP '170.106.202.8', am 03.10.2021, 02:33:44. Das Erstellen und Weitergeben von Kopien dieses PDFs ist nicht zulässig. 8 Knowl. Org. 46(2019)No.1 P. Rafferty. Disrupting the Metanarrative: A Little History of Image Indexing and Retrieval for more pre-iconological or generic level description of construct superior computerised image retrieval systems. the content of images, an approach that Jörgensen (1998) In many regards this move mirrors the more general move also highlights, focusing on the ways in which users search towards user behaviour research from the 1980s onwards. for images. In the project reported in the 1995 and 1998 The research generally suggested that the further away the papers, Jörgensen sought to discover the range and types information retrieval scenario was from the professionally of features needed to describe the contents of an image. mediated archival context, the greater the significance of She sorted terms that were spontaneously added to images browsing features (Enser 2008, 534). Feedback and user through describing, searching and sorting tasks undertaken interaction features also enhance the browsing approach by research subjects under quasi-experimental conditions. to image retrieval. The results led to the development of her baseline frame- Fidel (1997) captures this insight in the development of work of twelve attributes and three categories (perceptual, the image seeking continuum, which has the “object pole” interpretative and reactive), which have been used in other and the “data pole” as the extreme points of the contin- studies (for example Jansen 2008, and Rorissa 2010). The uum. Fidel explains that images can be used in different perceptual attributes come from looking at the image, the ways, so that an icon representing wheelchair accessibility interpretative attributes require some personal interpreta- is very different from the use of a colourful picture on a tion and may be in the eyes of the specific viewer, while wall (187). The icon is the image as the source of infor- the reactive may include conjecture and emotional re- mation (there are many assumptions in this observation of sponses. interest to semioticians) and the picture is the image as ob- When the Shatford matrix was used by Choi and Ras- ject. The “object pole” refers to image searches in which mussen (2003) to study queries from students of American the interest is in retrieving a specific image, so that speci- history, they discovered greater use of generic terms, with ficity and relevance are of considerable importance. The generic people or things, events and locations occurring “data pole” refers to the need to retrieve information or most frequently. It would seem that different user groups objects that the image portrays. Relevance feedback be- within different domains have different image information comes increasingly important towards this pole. needs. Conduit and Rafferty (2007) attempted to draw to- Smeulders et al. (2000) divided image searching for im- gether the different facet frameworks in a study that also ages into three categories: search by “association,” “target” mapped archivists’ views into a meta-framework but it is search and “category” search. Search by association aims very difficult to construct frameworks for mapping inter- to find interesting things. The results can be manipulated pretation of images, even if the interpreter is a domain ex- interactively by providing user feedback. Systems that sup- pert trained in indexing, not least because, as Armitage and port this category are highly interactive and support Enser (1997) point out, it is difficult to know the right cat- browsing. Target search aims the search at a specific image. egory in which to place things. Category search aims to find an arbitrary image repre- These approaches to image indexing highlight the com- sentative of a specific class (1351). They explain that this plexity of dealing with the subject in non-textual infor- is not the whole story however, and distinguish between mation objects whether we are concerned with analogue narrow and broad domains, arguing that in the broad do- or digital systems. The technologies may allow us to con- main, images are polysemic and their semantics can be de- struct more efficient ways of producing and reproducing scribed only partially (1352). images, of storing, transmitting and accessing images. Yet, And this takes us to another story, about stories and the subject problem is still there to be addressed, and the storytellers, about interpretative hermeneutics and the phenomenal descriptions that these systems envisage, writerly nature of images. In the early 1990s, when com- while interrogating the metanarrative of the efficiency of puterised information retrieval was still relatively specialist, computerized systems, still depend on the description be- some writers speculated about whether digital information ing undertaken by the expert indexer/reader on behalf of retrieval systems could facilitate a multi-voiced approach the information seekers. to indexing images (for example, Hidderley and Rafferty 1997). They took the view that conventional approaches 4.0 User generated content, tagging, landscapes to image indexing both facilitate and discipline access and and flaneurs discovery, and suggested that information system design- ers should encourage more democratic approaches to im- Alongside the development of frameworks for phenome- age indexing. One might critique the use of the term nal descriptions of images and the associative meanings of “democratic” in this early work, and point to the auto- images, a considerable amount of research from the 1990s mated and instrumental means by which a consensus view onwards focused on how and why people searched for im- was to be sought in the theoretical framework (see Mai ages, exploring whether this knowledge could help us to 2011 for a constructive critique), but it is worth recalling

https://doi.org/10.5771/0943-7444-2019-1-4 Generiert durch IP '170.106.202.8', am 03.10.2021, 02:33:44. Das Erstellen und Weitergeben von Kopien dieses PDFs ist nicht zulässig. Knowl. Org. 46(2019)No.1 9 P. Rafferty. Disrupting the Metanarrative: A Little History of Image Indexing and Retrieval that these theoretical musings were undertaken at a time be more appropriate in relation to some, if not all, infor- before the web became ubiquitous, when relatively rigid mation seeking activities around digital cultural infor- frameworks for designing computerised information re- mation, including images. In their research, the “infor- trieval dominated. The early work was necessarily limited, mation flaneur” is a metaphor that inspires a new way of but it opened up some space for further discourse. thinking about information seeking. For Dörk, Carpendale The work done on analysing user requests for images and Williamson, the “information flaneur” is “an urban by Enser, Jorgensen and Fidel, amongst others, revealed wanderer, who leisurely walks through the streets and the importance of browsing and the more general search squares interpreting and re-imaging the city” (1). The “in- by association and also category types of searches that formation flaneur” sees beauty and meaning in growing in- people undertake in large, unmediated image collections. formation spaces. Their flaneur is: This revelation pointed towards the development of browsing and enhanced discovery and access mechanisms – Curious Explorer: passing through squares and in and through web systems; we can see this also in relation crowds, making sense of the city without becom- to web-based text searching, for example in the developing fully part of it. He keeps his own leisurely ment of tagging, but it was image information seeking re- pace in resistance to the growing pace of capital- search that revealed the importance of browsing, feedback ism. and interpretative and reactive categories within unmedi- – Critical Spectator: fascinated by the commercial ated and ubiquitous digital information environments (i.e., spectacle but also aware of the social realities ac- the World Wide Web) as far back as the early 1990s. companying modern life. Something of the desire for a more democratic ap- – Creative Mind: viewing the urban story as epic proach to retrieval that acknowledges and values feedback, heterogeneity, the flaneur is an interpreter making user engagement and browsing can be seen nowadays in “the urban landscape legible and meaningful. He social tagging-based systems, and image tagging, which has the ability to relate to the world through mul- along with music tagging, emerged as an important re- tiple facades” (3) search front in the early 2000s (see for example, Trant 2006). Matusiak (2006) compared and contrasted profes- Envisioned as the information flaneur, this information sionals’ metadata with image creators tags on Flickr, find- seeker’s seeking activities are embedded into everyday ing that although the tags were unstructured and “sloppy,” practice, he uses many information tools: sometimes he they are also richer, more current and multilingual. More moves towards information targets, sometimes uses visu- recently, Baldoni et al. (2012) combined affective compu- alisations. As a critical user, he avoids excessive filtering, ting, social tagging and ontologies in relation to artworks enjoys bumping into information and cultivates an open with the end goal of representing the emotional tags de- mind to find hidden connections. He enjoys exploring un- rived from user interactions as emoticons. While the tag- familiar information spaces, new information discoveries ging and folksonomy approach might allow for a wide and makes personal meaning. The “information flaneur” range of voices to be heard, the burden of judging rele- follows clues and links and hunches to read the stories in vance is then on the information seeker. Recent ap- the information space. proaches to information system design have focused on Based on this persona, Dörk, Carpendale and William- the figure of the information seeker and the goal of de- son suggest a model of information design that privileges: signing systems that can deal with the information overload generated by digital information. – Orientation: including situated navigation, con- Drawing on the literature of information seeking, with textualisation, faceted navigation and visualisa- its focus on the user experience, the human-centred un- tion; derstanding of the research process, its interest in the eve- – Visual Momentum and information visualisation: ryday, the serendipitous and the exploratory, and including for example, animated transitions, zoomable in- the notion of the flaneur, found in literature dating back terfaces and detail-on-demand; and, to 1840s Paris and associated along the way with literary – Serendipity: juxtapositioning researchers who figures such as Baudelaire, Simmel, Walter and Benjamin, share unusual facets or relate to previous interac- Dörk, Carpendale and Wiliamson (2011) sought to create tions. new interface models that offer various and varied pathways through information spaces. They distinguish be- This move towards the human-centred information seek- tween the conventional utilitarian and task-based ap- ing and the serendipitous pleasure oriented exploratory as- proaches to conceptualising information seeking and the pects of information seeking has influenced the develop- casual, playful and pleasurable perspectives that might well ment of the “generous interface” (Whitelaw 2012 and

2015), which offers rich, browsable views, provides sam- mation flaneur” as the information seeker; the web, with its ples of the content and supports contextualised displays. teeming, sprawling, labyrinthine information sites, is the Whitelaw (2015, 7) suggests some general principles for equivalent of the dreadful delights of the city, but in the the creation of more generous cultural information inter- literary metaphor, the flaneur moves within a city of people, faces: not just a city of buildings, architectures, arcades and glit- tering objects, and in the peopled city, the flaneur moves – Show first, don’t ask: volunteer information that through fragments of story: newspapers stories, gossip, supports the audiences’ understanding; chattering crowds, language, image and information. – Provide rich overviews: and help to orient the us- Moreover, the peopled city has its history, fragments of ers’ explorations; which not only survive but form and inform the contem- – Provide samples: to provide rich contextualised porary moment. The flaneur reads the peopled city for in- clues and invite explorations. formation in and through crowds. The “information flaneur” inspired designs that we have Another recent information metaphor that draws on some to date are still curatorially centred, but there may be space of the “information flaneur” approach is the notion of the to create approaches to discovery that draw on the rich and “pathway” through cultural content as used, for example, serendipitous encounters in the virtual cityscape of the by Wray, Eklund and Kautz (2013) to design retrieval sys- web. Such spaces might allow us to build into search and tems for digital art collections. The focus here is on con- discovery approaches the stories and fragments and gossip nectedness while designing approaches that follow visitors’ and reminiscences of the crowd, so that, in relation to at interests and perspectives rather than imposing only one least some types of image search, we might move towards authoritative, curatorially informed view. They point to multi-voiced descriptions of images, as tagging allows, but two emerging trends in interactions of digital objects in go beyond the paradigmatic plane of single word or information spaces: mashed up tags, to the syntagmatic descriptions that storytelling allows. 1. Expansion of information spaces: Massive scale The notion of the “writerly” text and the “readerly” and federated availability that allows sharing of text (Barthes 1974) is perhaps of some interest here. The knowledge, avenues for exploration and meaning writerly text points to texts in which the reader is given making (Cairns 2013) and encourages personal space to interpret the text, while the readerly text is a text exploration and appropriation of collections in which meaning is more clearly determined by the au- from institutions to communities of interest. thor, and beyond the specific author, by the codes and con- 2. Recognition of pleasure, aesthetics and play: re- ventions relating to the genre or form within which the lating the experience of engaging with digital art- author is working. Images, particularly those that do not works back to idea of Homo Ludens (Huizinga have text to anchor meaning, allow us to focus on the writ- 1971). erly text. The writerly text lets us connect human to human, in the historical present, across historical time and Related to the “information flaneur” and the “information their interpretative potential is open to shifts and changes pathway” (and in tagging literature, desire lines) is the met- and multi-various interpretations. In the era of tagging, aphor of the “information landscape,” which calls for ex- Rafferty and Albinfalah (2014) undertook an exploratory ploratory searching, feedback and innovative approaches study that investigated storytelling as an approach to de- to mapping, shifting and managing “information hori- veloping a template for the input side of image description zons.” The phrase information horizons is suggestive of through digital collections. Their study was necessarily Jauss’ (1970) horizons of expectation and the possibilities small and narrow and took a quasi-experimental approach this framework offers for an historicised understanding of that asked research subjects to provide narrative descrip- the reader-author dialectic in meaning making and the di- tions of two images, which while they were highly modal, achronically shifting nature of genre. Something of that were arguably writerly images that invited interpretation. same acknowledgement of the diachronic transformations The storytelling approach produced a broad range of con- in information search and discovery resides in the notion notational responses that could be used to construct rich of the “information flaneur.” discovery tools. In the literature of the 1840s and in Benjamin’s work, The interest in developing rich user-generated descrip- the flaneur walks through the city of sometimes dreadful tions in image retrieval stretches back several years. In a delights, with a critical eye, sometimes a sardonic eye on paper published in 1998, O’Connor, O’Connor and Abbas the consumer society of modernity. We could, following analysed viewers’ descriptions of images and noted the Dörk, Carpendale and Williamson re-envision, the “infor- tendency towards narrative description where narrative re-

https://doi.org/10.5771/0943-7444-2019-1-4 Generiert durch IP '170.106.202.8', am 03.10.2021, 02:33:44. Das Erstellen und Weitergeben von Kopien dieses PDFs ist nicht zulässig. Knowl. Org. 46(2019)No.1 11 P. Rafferty. Disrupting the Metanarrative: A Little History of Image Indexing and Retrieval fers to instances where the viewers told little stories about formation management of signifying documents is diffi- the images, alluded to aspects of their own life story (“this cult. reminds me of …”) or expressed frustration that they As for the way ahead, there are a number of paths that could not situate the image within a narrative. This study could be, and are being pursued, including the notion of was carried out before widespread tagging through web 2.0 the iconic thesaurus, or perhaps more accurately the index- applications and is of interest in relation to this paper, be- ical iconic sign-based thesaurus for images, enhanced cause it is another instance of the image indexing story CBIR practices that incorporate recommender systems of pointing relatively early on to new approaches to docu- various types and viewer-user oriented HCI design ap- ment description, acknowledging the polysemic nature of proaches, such as Dörk’s flaneur-inspired approach. But the sign and recognising the range of information seeking wherever we go, it would seem that the certainty, uni- activities undertaken in relation to images. One of the formity and conventionality that tended to characterise in- challenges in social tagging has been to encourage creative formation retrieval approaches of the mid-twentieth cen- input while at the same time disciplining input, for exam- tury, and which were interrogated so often in the research ple, through the development of tag ontologies (see for literature of image retrieval, have truly been superseded. example, Gruber 2007; Kim et al. 2008; Ding et al. 2010). Earlier templates (Rafferty and Hidderley 2005) have had References limited success in practice. It might be that the ubiquitous and intuitive form of the story would allow the develop- Alzu’bi, Ahmad, Abbes Amira, and Naeem Ramzan. 2015. ment of a relatively structured but intuitive method of in- “Semantic Content-based Image Retrieval: A Compre- putting interpretative content. hensive Study.” Journal of Visual Communication and Image Representation 32: 20-54. 5.0 Final comments Alexander, Arden and Tracy Meehleib. 2001. “The The- saurus for Graphic Materials: Its History, Use, and Fu- History might be characterised as the construction of or- ture.” Cataloging & Classification Quarterly 31, no. 3/4: ganised narrative about human-experienced time, and the 189-212. tendency of the historian is to smooth the stories so that Armitage, Linda H. and Peter G.B .Enser. 1997. “Analysis they fit specific worldviews or theoretical frameworks. But, of User Need in Image Archives.” Journal of Information sometimes there are blips, disruptions, discordances and Science 23: 287-99. difficulties that disrupt the smoothness of historical nar- Baldoni, Matteo, Cristina Baroglio, Viviana Patti, and rative (see for example, discussions about history as narra- Paolo Rena. 2012. “From Tags to Emotions: Ontology- tive in White 1973; Stone 1979; Bruner 1991; Carr 1991). driven Sentiment Analysis in the Social Semantic Web.” Information science and information retrieval is now at an Intelligenza Artificiale 6, no. 1: 41-54. historical stage that allows for broad brushed and sweep- Barthes, Roland, 1974. S/Z. New York: Hill and Wang. ing historical narrative; however, this paper argues that the Bennett, W. Lance and Livingston, Steven. 2018. “The Dis- history of image retrieval presents the metanarrative of in- information Order: Disruptive Communication and the formation retrieval, a metanarrative based on modernity Decline of Democratic Institutions.” European Journal and efficiency, with a little element of disruption. of Communication 33: 122-39. Whilst accepting the general thrust of the modern Benson, Allen C. 2015. “Image Descriptions and their Re- metanarrative of modernity that emerges in the literature lational Expressions: A Review of the Literature and of knowledge organization and information retrieval, this the Issues.” Journal of Documentation 71: 143-64. paper offers a gentle reminder that image retrieval has al- Bohnsack, Ralf. 2009. “The Interpretation of Pictures and ways brought with it challenges regarding interpretation, the Documentary Method.” Historical Social Re- decoding and readership that in many ways have only taken search/Historische Sozialforschung 34: 296-321. a central role in text retrieval solutions in the age of par- Bruner, Jerome. 1991. “The Narrative Construction of Re- ticipatory digital culture and social media. With social me- ality.” Critical Inquiry 18, no 1: 1-21. dia and the democratisation of authorship and readership, Bryant, Antony, Alistair Black, Frank Land, and Jaana Porra. there have been moves towards the democratisation of in- 2013. “Information Systems History: What is History? dexing, but images posed their own knowledge represen- What is IS History? What IS History? … And Why Even tation and information management challenges before the Bother with History?” Journal of Information Technology 28, widespread adoption of the web. And while conventional no. 1: 1-17, https://link.springer.com/article/10.1057/ image retrieval tools have been developed in parallel with jit.2013.3 text retrieval tools, the literature of image retrieval has for many years provided reminders that the indexing and in-

Cairns, Susan. 2013. “Mutualizing Museum Knowledge: 2007. “Visualizing Tags over Time.” ACM Transactions on Folksonomies and the Changing Shape of Expertise.” the Web 1, no. 2, article 7. doi:10.1145/1255438.1255439 Curator: The Museum Journal 56: 107-19. Enser, Peter. G. B. 1995. “Progress in Documentation: Pic- Carr, David. 1991. Time, Narrative, and History. Indianap- torial Information Retrieval.” Journal of Documentation olis: Indiana University Press. 51: 126-70. Carr, Edward Hallett. 1961. What is History? London: Mac- Enser, Peter. G. B. 2000. “Visual Image Retrieval: Seeking millan. the Alliance of Concept-based and Content-based Par- Chen, Tao, Damian Borth, Trevor Darrell, and Shih-Fu adigms.” Journal of Information Science 26: 199-210. Chang. 2014. “Deepsentibank: Visual Sentiment Con- Enser, Peter. G. B. 2008a “The Evolution of Visual Infor- cept Classification with Deep Convolutional Neural Net- mation Retrieval.” Journal of Information Science 3: 531-46. works.” CoRR, arXiv http://arxiv.org/abs/1410.8586 Enser, Peter. G. B. 2008b. “Visual Image Retrieval.” Annual Choi, Youngok and Edie M. Rasmussen. 2003. “Searching Review of Information Science and Technology 42: 1-42. for Images: The Analysis of Users’ Queries for Image Feinberg, Melanie. 2006. An Examination of Authority in Retrieval in American History.” Journal of the Association Social Classification Systems.” Advances in Classification for Information Science and Technology 54: 498-511. Research Online 17, no. 1: 1-11. Collingwood, Robin George. 1994. The Idea of History. Ox- Fidel, Raya. 1997. “The Image Retrieval Task: Implications ford: Oxford University Press. for the Design and Evaluation of Image Databases.” New Collins, Karen. 1998. “Providing Subject Access to Im- Review of Hypermedia and Multimedia 3, no. 1: 181-99. ages: A Study of User Queries.” The American Archivist Garcia-Marco, Francisco-Javier. 2016. “Enhancing the 61: 36-55. Visibility and Relevance of Thesauri in the Web: Conduit, Neil and Pauline Rafferty. 2007. “Constructing Searching for a Hub in the Linked Data Environment.” an Image Indexing Template for The Children’s Society: Knowledge Organization 43: 193-202. Users’ Queries and Archivists’ Practice. Journal of Docu- Godoy, Daniela and Alejandro Corbellini. 2016. “Folkson- mentation 63: 898-919. omy‐Based Recommender Systems: A State‐of‐the‐Art Datta, Ritendra, Dhiraj Joshi, Jia Li, and James Z. Wang. Review.” International Journal of Intelligent Systems 31: 314- 2006. “Studying aesthetics in Photographic Images us- 46. ing a Computational Approach.” In 9th European Confer- Gruber, Thomas. 2007. “Ontology of Folksonomy: A ence on Computer Vision, Graz, Austria, May 7-13, 2006, Mash-up of Apples and Oranges.” International Journal Proceedings, ed. A. Leonardis, H. Bischof, and A. Pinz. on Semantic Web and Information Systems 3, no. 1: 1-11. Lecture Notes in Computer Science 3953. Berlin: Gudivada, Venkat. N. and Vijay. V. Raghavan. 1995. Picture Springer, 288-301. doi:10.1007/11744078_23 Retrieval systems: A Unified Perspective and Research Issues. Day, Ronald E. 2008. The Modern Invention of Information: Computer Science Technical Report CS9503. Athens: Discourse, History, and Power. Carbondale, IL: SIU Press. Ohio University. Day, Ronald E. 2014. Indexing It All: The Subject in the Age Guy, Marieke and Emma Tonkin, 2006, “Folksonomies: of Documentation, Information, and Data. Cambridge, MA: Tidying up Tags?” D-Lib Magazine 12, no. 1. http:// MIT Press. www.dlib.org/dlib/january06/guy/01guy.html Ding, Ying, Elin K. Jacob, Michael Fried, Ioan Toma, Erjia Hare, Jonathon S., Paul H. Lewis, Peter G. B. Enser, and Yan, Schubert Foo, and Staša Milojević. 2010. “Upper Christine J. Sandom. 2006. “Mind the Gap: Another Tag Ontology for Integrating Social Tagging Data.” Look at the Problem of the Semantic Gap in Image Re- Journal of the Association for Information Science and Technol- trieval.” Paper presented at IS&T [Society for Imaging ogy 61: 505-21. Science and Technology)/SPIE [International Society Dörk, Marian, Sheelagh Carpendale, and Carey William- for Optics and Photonics] 18th Annual Symposium, son. 2011. “The Information Flaneur: A Fresh Look at Electronic Imagin Science and Technology, 15-19 Janu- Information Seeking.” Proceedings of the SIGCHI Confer- ary 2007, San Jose, Calif. ence on Human Factors in Computing Systems. New York: Hayles, N. Katherine. 2008. How we became Posthuman: Vir- ACM, 1215-24. tual Bodies in Cybernetics, Literature, and Informatics. Chi- Dousa, Thomas. 2007. “Everything Old is New Again: cago: University of Chicago Press. Perspectivism and Polyhierarchy in Julius O. Kaiser’s Hidderley, Rob and Pauline Rafferty. 1997. “Democratic Theory of Systematic Indexing.” Paper delivered at the Indexing: An Approach to the Retrieval of Fiction.” In- 2007 ASIST SIG/CR Meeting, Milwaukee, Wisconsin. formation Services & Use 17: 101-9. https://repository.arizona.edu/handle/10150/105780 Hidderley, Rob and Pauline Rafferty. 2005. Indexing Multi- Dubinko, Micah, Ravi Kumar, Joseph Magnani, Jasmine media and Creative Works: The Problems of Meaning and In- Novak, Prabhakar Raghavan, and Andrew Tomkins. terpretation. Farnham: Ashgate Publishing.

Hodge, Bob and Gunther. Kress, 1988. Social Semiotics. Nam, Hyoryung and Pallassana Krishnan Kannan. 2014. Cambridge: Polity. “The Informational Value of Social Tagging Net- Huizinga, Johan. 1971. Homo Ludens: A Study of the Play- works.” Journal of Marketing 78, no. 4: 21 –40. Element in Culture. Boston: Beacon. O’Connor, Brian C, Mary K O’Connor, and June M. Abbas. Jansen, Bernard J. 2008. “Searching for Digital Images on 1999. “User Reactions as Access Mechanism: An Explo- the Web.” Journal of Documentation 64: 81-101. ration Based on Captions for Images.” Journal of the Asso- Jauss, Hans Robert; Elizabeth Benzinger. 1970. “Literary ciation for Information Science and Technology 50: 681-97. History as a Challenge to Literary Theory.” New Literary Pan, Yingwei, Ting Yao, Tao Mei, Houqiang Li, Chong- History 2: 7–37. Wah Ngo, and Yong Rui. 2014. “Click-through-based Jorgensen, Corinne. 1995. “Classifying Images: Criteria for cross-view Learning for Image Search.” In Proceedings Grouping as Revealed in a Sorting Task.” Advances in Of The 37th International ACM SIGIR Conference On Re- Classification Research Online 6, no. 1: 65-78. search & Development In Information Retrieval. New York: Jorgensen, Corinne. 1998. “Attributes of Images in De- ACM, 717-26. scribing Tasks.” Information Processing & Management 34: Panofsky, Erwin. 1993. Meaning in the Visual Arts. Har- 161-74. mondsworth: Penguin. Joshi, Dhiraj, Ritendra Datta, Elena Fedorovskaya, Quang- Rafferty, Pauline. 2001. “The Representation of Knowledge Tuan Luong, James Z. Wang, Jia Li, and Jiebo Luo. in Library Classification Schemes.” Knowledge Organization 2011. “Aesthetics and Emotions in Images.” IEEE Sig- 28: 180-91. nal Processing Magazine 28, no. 5: 94-115. Rafferty, Pauline. 2011. “Informative Tagging of Images: Kim, Hak Lae, Alexandre Passant, John G. Breslin, Simon The Importance of Modality in Interpretation”. Scerri, and Stefan Decker. 2008. “Review and Align- Knowledge Organization 38: 283-98. ment of Tag Ontologies for Semantically-linked Data Rafferty, Pauline and Fawaz Albinfalah. 2014. “A Tale of in Collaborative Tagging Spaces.” Proceedings IEEE In- Two Images: The Quest to Create a Story-based Image ternational Conference on Semantic Computing 2008 4-7 Au- Indexing System.” Journal of Documentation 70: 605-21. gust 2008 Santa Clara, California. Los Alamitos, CA: Rafferty, Pauline and Rob Hidderley. 2007. “Flickr and IEEE Computer Society, 315-22. Democratic Indexing: Dialogic Approaches to Index- Krause, Michael. G. 1988. “Intellectual Problems of Index- ing.” Aslib Proceedings 59: 397-410. ing Picture Collections.” Audiovisual Librarian 14: 73-81. Rayward, W. Boyd. 2014. “Information Revolutions, The Kroski, Ellyssa. 2005. “The Hive Mind: Folksonomies and Information Society, and the Future of the History of User-based Tagging.” InfoTangle (blog), December 7. Information Science.” Library Trends 62, no. 3: 681-713. Mai, Jens-Erik. 1999. “A Postmodern Theory of Know- Ristoski, Peta, and Heiko Paulheim. 2016. “Semantic Web ledge Organization.” In ASIS ‘99: Proceedings of the 62nd in Data Mining and Knowledge Discovery: A Compre- ASIS Annual Meeting, Washington, DC, October 31-November hensive Survey.” Web Semantics: Science, Services and Agents 4, 1999; Knowledge, Creation, Organization and Use, ed. Larry on the World Wide Web 36: 1-22. Woods. Proceedings of the ASIS Annual Meeting 36. Rorissa, Abebe. 2010. “A Comparative Study of Flickr Medford, N.J.: Information Today, 547-56. Tags and Index Terms in a General Image Collection.” Mai, Jens-Erik. 2004. “Classification in Context: Relativity, Journal of the Association for Information Science and Technol- Reality, and Representation.” Knowledge Organization 31: ogy 61: 2230-42. 39-48. Rowley, Jennifer. 1994. “The Controlled Versus Natural Mai, Jens-Erik. 2011. “Folksonomies and the New Order: Indexing Languages Debate Revisited: A Perspective on Authority in the Digital Disorder.” Knowledge Organiza- Information Retrieval Practice and Research.” Journal of tion 38: 114-22. Information Science 20: 108-18. Mathes, Adam. 2004. “Folksonomies-Cooperative Classifi- Sanderson, Mark and W. Bruce Croft. 2012. “The History cation and Communication through Shared Metadata.” of Information Retrieval Research.” Proceedings of the htttp://www.adammathes.com/academic/computer- IEEE 100: 1444-51. doi:10.1109/JPROC.2012.2189916 mediated-communication/folksonomies.html Shatford, Sara. 1986. “Analyzing the Subject of a Picture: Matusiak, Krystyna K. 2006. “Towards User-Centered In- A Theoretical Approach.” Cataloging & Classification dexing in Digital Image Collections.” OCLC Systems & Quarterly 6, no 3: 39-62. Services: International Digital Library Perspectives 22: 283-98. Shatford Layne, Sara. 1994. “Some Issues in the Indexing Merholz, Peter. 2005. “Metadata for the Masses.” Adaptive- of Images.” Journal of the American Society for Information Path.org (blog), Oct. 19. http://adaptivepath.com/ideas/ Science 45: 583-88. e000361

Shirky, Clay. 2005. “Ontology is Overrated: Categories, User Browsing Behavior.” In Proceedings of the 35th Inter- Links, Tags.” Clay Shirky’s Writings About the Internet (blog). national ACM SIGIR Conference on Research and Develop- http://shirky.com/writings/ontology_overrated.html ment in Information Retrieval. New York: ACM, 445-54. Schou, Jannick and Johan Farkas. 2016. “Algorithms, In- White, Hayden. 1973. Metahistory: The Historical Imagination terfaces, and the Circulation of Information: Interro- in Nineteenth-Century Europe. Baltimore: Johns Hopkins gating the Epistemological Challenges of Facebook.” University Press. KOME: An International Journal of Pure Communication In- Whitelaw, Mitchell. 2012. “Towards Generous Interfaces quiry 4: 36-49. for Archival Collections.” Comma 2012, no. 2: 123-132. Smeulders, Arnold W.M., Marcel Worring, Simone Santini, Whitelaw, Mitchell. 2015. “Generous Interfaces for Digital Amarnath Gupta, and Ramesh Jain. 2000. “Content- Cultural Collections.” Digital Humanities Quarterly 9, no. based Image Retrieval at the End of the Early Years.” 1. http://www.digitalhumanities.org/dhq/vol/9/1/00 IEEE Transactions on Pattern Analysis and Machine Intelli- 0205/000205.html gence 22: 1349-80. Wray, Tim, Peter Eklund, and Karlheinz Kautz. 2013 Stone, Lawrence. 1979. “The Revival of Narrative: Reflec- “Pathways through Information Landscapes: Alterna- tions on a New Old History.” Past & Present 85: 3-24. tive Design Criteria for Digital Art Collections.” In ICIS Surowiecki, James. 2005. The Wisdom of Crowds. New York: 2013 Proceedings. https://aisel.aisnet.org/icis2013/pro- Anchor. ceedings/GlobalIssues/1/ Trant, Jennifer. 2006. “Exploring the Potential for Social Zhu, Lei, Jialie Shen, Liang Xie, and Zhiyong Cheng. 2017. Tagging and Folksonomy in Art Museums: Proof of “Unsupervised Visual Hashing with Semantic Assistant Concept.” New Review of Hypermedia and Multimedia 12, for Content-based Image Retrieval.” In IEEE Transac- no. 1: 83-105. tions on Knowledge and Data Engineering 29: 472-86. Trevisiol, Michele, Luca Chiarandini, Luca Maria Aiello, and Alejandro Jaimes. 2012. “Image Ranking Based on

https://doi.org/10.5771/0943-7444-2019-1-4 Generiert durch IP '170.106.202.8', am 03.10.2021, 02:33:44. Das Erstellen und Weitergeben von Kopien dieses PDFs ist nicht zulässig.