Big Image Data within the Big Picture of Art History Harald Klinke Abstract: The use of the computer in Art History is changing the approach towards our objects of research. Now, we are able to compute more images than a human can see in a lifetime. That, in turn, calls for a new definition of the role of the researcher and the tools being used. The access to large amounts of visual data stands in a tradition of conventional methods of Art History, but also augments them with quantity. This article proposes a theoretical model on which to build an understanding of the meta image with which we interactively derive our conclusions. Keywords: Big Data, Digital Humanities, Distant Viewing, Information, Media, Meta Image, Methodology, Qualitative/quantiative Research, Semantic Gap he digitalization of Art History is meaning layer by layer. The goal here Tcreating enormous opportunities. is to embed the individual object into Among the most striking is that we its cultural and historic context. Art now have the opportunity to not only History is, thus, an empirical science look at a single work of art—or the that gathers and interprets visual data. comparison of two—but that we are This is one reason why Art History is, also able to compare more pictures in German-speaking countries, also than a human can look at in a lifetime. called “Bildwissenschaft,” the science This concept is called Big Image Data of pictures. It should be noted that (BID). Panofsky knew that starting with the Taking a look at our discipline’s visual phenomenon does not allow the methodology, the first systematic ap- exclusion of any written records. It is proach that comes to mind is Erwin only natural that scholars in front of Panofsky’s iconography and iconolo- a work of art will acquire knowledge gy.1 According to him, and in contrast from books and compare their findings to all other disciplines in the Human- with those of other scholars, and then ities, art historians take the visual re- add to this shared knowledge through cord seriously and derive knowledge future publications. from it by reconstructing an artwork’s Figure 1: About 60.000 images taken from the Artemis database (LMU Munich) sorted by average brightness using ImagePlot developed by the Software Studies Initiative Big Image Data Another art-historical method This capability is exactly what the comes to mind: the side-by-side com- Digital Humanities promise. As the parison of pictures as established by Natural Sciences have used comput- Heinrich Wölfflin.2 What has been ers to yield unprecedented insights done with originals and painted or that were not possible before, such as printed copies before the advent of the sequencing and analysis of the hu- slide projections turned into a central man genome, the Humanities are able method in our discipline through the to make the computer a tool for their use of double slide projectors. Com- research, too. Ever since Roberto Busa parisons make clear the differences as created a lemmatization of the works well as the similarities of the studied of Thomas Aquinas using IBM com- objects. The findings can then be dis- puters in 1951,4 text-based Humanities cussed and converted into insights, have developed many software tools to such as the question of the object being obtain an overview of text corpora em- an original or a copy, questions of dat- ploying statistical methods, to uncover ing (Datierung), or authorship attribu- semantic structures via text mining, tion (Handscheidung).3 A slide library and to develop new insights into lan- containing hundreds of thousands of guage using Corpus Linguistics. The high resolution color images is a pow- visual sciences, the foremost being Art erful means of navigation through the History, started to integrate computa- history of art. tional methods in the early 1980s,5 but progressed slowly for several reasons.6 Both methods show that no other The most important reason is that im- discipline looks at pictures as system- ages are much harder for a computer to atically as Art History. They also show process. This might seem to be an odd that individual scholars and their visu- statement, especially since digital im- al experiences acquired over the years ages consist of nothing beyond a string are at the center of this kind of re- of information units called bits like all search. In both cases, researchers need other digital media. But this matrix of a mental archive of images in order to pixels does not “mean” anything to the identify figures by their attributes and computer at first. to select objects for comparison. Would it not be great to go a step further and While one string of ASCII charac- use computers to be able to consult a ters7 in a text can be compared to an- much greater variety of artworks, to other (search), they can be summed up use algorithms to recommend further in a corpus (frequency distribution) or works that might be of interest to the become the basis of statistical calcu- researcher? Would it not be great to lations (text mining). A string of pix- let the computer do the work of sifting els on a display is nothing more than through a great number of artworks in a sequence of brightness values on order to deliver meta information on the visible light spectrum (red, green an enormous corpus of images? and blue) at first. However, we are 16 DAH-Journal Big Image Data able to calculate and use these color of images at once and deriving meta in- values as the criterion to sort images8 formation from that corpus of images.12 in a process involving these “low lev- Dealing with a massive amount of data el features” of images (see figure 1). is a tremendous challenge for Art His- This process, however, is limited in its tory. The good news is that there is no epistemic potential.9 In contrast, “high need to start from scratch. Other disci- level features” are what those pixels plines have dealt with images before: represent; the content of the images. medical applications deal with images, Between both of these lies the “se- such as high quality x-rays. Biology mantic gap,” where high level features has developed software to handle pic- are perceived by humans with ease, tures produced by microscopy in order whereas the computer still struggles. to quickly discover clusters or count Information Science—Computer Vi- cells. Astronomy, Geology, and other sion in particular—is working on this similar disciplines incorporated such problem via pattern recognition, deep tools into their daily practices long learning, and other approaches, but ago. In particular, one field of Informa- there is still much research to do.10 As tion Science has advanced quickly of research is carried out, it will increas- late: Big Data. ingly provide the means to work with images that Art History can use for its Big Data has been a buzzword for own purposes. a few years now. In 2012, it was iden- tified as the biggest trend by Bitkom— At this point, we have to distin- the German IT Trade Association—and guish two concepts. Reading a book of in 2014, Big Data was the theme of the literature and writing about it has been CeBit trade fair. The reason for this called “close reading.” Having all works recognition is that it promises unprec- of an author in a digital format and edented insights into social behavior comparing them to those of another and new possibilities for business mod- author by means of statistical analysis els. At the same time, it stokes fears of is an example of “distant reading.”11 It surveillance and espionage. Big Data offers the possibility to use computers seems to be defined as the collecting to work with data in order to analyze a and analyzing of data, with the ulti- greater number of books in a matter of mate result and goal to make money; seconds than an individual could hope Facebook, Apple, and Google come to to read in a lifetime. These concepts can mind when discussing Big Data. With also be applied as standard methods data, companies not only know what of study for Art History, where “close we do, but also things that we might viewing” describes the study of indi- do.13 Some say that data is the new raw vidual reproductions of artwork. In the material.14 And the massive amount of same vein, “distant viewing” describes data is not the problem, but the solu- what algorithms are increasingly able tion. to offer: examining an infinite number DAH-Journal 17 Big Image Data Big Data, first of all, is an Informa- a quantity unprecedented in human tion Science concept. Only in combi- history (and that is truer than ever nation with business does it become a today, 22 years later). Since we have concept for profit maximization. It has sciences for language, there must also the potential, in combination with sci- be a science for images (Bildwissen- ence, for “knowledge maximization.” schaft).15 So again, what is an image?16 Moreover, it can become a fruitful An initial approach to answering this tool in the Digital Humanities. While question could be to name the objects “data” is generally used here to denote that we include under the term “image.” values, such as sensor results, the chal- The history of art knows many such lenge for disciplines like Art History objects, and paintings are only one of that use images is figuring out how to them.