An Early Framework for Determining Artistic Influence Kanako Abe, Babak Saleh, and Ahmed Elgammal Department of Computer Science Rutgers University kanaabe,[email protected], [email protected] Abstract. Considering the huge amount of art pieces that exist, there is valuable information to be discovered. Focusing on paintings as one kind of artistic creature that is printed on a surface, artists can determine its genre and the time period that paintings can belong to. In this work we are proposing the interesting problem of automatic influence determina- tion between painters which has not been explored well. We answer the question “Who influenced this artist?” by looking at his masterpieces and comparing them to others. We pose this interesting question as a knowledge discovery problem. We presented a novel dataset of paintings for the interdisciplinary field of computer science and art and showed interesting results for the task of influence finding. 1 Introduction How do artists describe their paintings? They talk about their works using sev- eral different concepts. The elements of art are the basic ways in which artists talk about their works. Some of the elements of art include space, texture, form, shape, color, tone and line [7]. Each work of art can, in the most general sense, be described using these seven concepts. Another important descriptive set is the principles of art. These include movement, unity, harmony, variety, balance, con- trast, proportion, and pattern. Other topics may include subject matter, brush stroke, meaning, and historical context. As seen, there are many descriptive attributes in which works of art can be talked about. One important task for art historians is to find influences and connections between artists. By doing so, the conversation of art continues and new intuitions about art can be made. An artist might be inspired by one painting, a body of work, or even an entire genre of art is this influence. Which paintings influence each other? Which artists influence each other? Art historians are able to find which artists influence each other by examining the same descriptive attributes of art which were mentioned above. Similarities are noted and inferences are suggested. It must be mentioned that determining influence is always a subjective deci- sion. We will not know if an artist was ever truly inspired by a work unless he or she has said so. However, for the sake of finding connections and progressing A. Petrosino, L. Maddalena, P. Pala (Eds.): ICIAP 2013 Workshops, LNCS 8158, pp. 198–207, 2013. c Springer-Verlag Berlin Heidelberg 2013 An Early Framework for Determining Artistic Influence 199 Fig. 1. An example of an often cited comparison in the context of influence. Diego Vel´azquez’s Portrait of Pope Innocent X (left) and Francis Bacon’s Study After Vel´azquez’s Portrait of Pope Innocent X (right). Similar composition, pose, and subject matter but a different view of the work. through movements of art, a general consensus is agreed upon if the argument is convincing enough. Figure 1 represents a commonly cited comparison for study- ing influence. Is influence a task that a computer can measure? In Computer Vision, there has been extensive research on the object-recognition in images, similarity be- tween images. Also there has been united research on automated classification of paintings [1,2,3,9,8]. However, there is very little research done on measuring and deter- mining influence between artists ,e.g. [9]. Measuring influence is a very difficult task because of the broad criteria for what influence between artists can mean. As mentioned earlier, there are many different ways in which paintings can be described. Some of these descriptions can be translated to a computer. Some re- search includes brushwork analysis [9] and color analysis to determine a painting style. For the purpose of this project, we do not focus on a specific element of art or principle of art but instead we focus on finding new comparisons by ex- perimenting with different similarity measures. Although the meaning of a painting is unique to each artist and is com- pletely subjective, it can somewhat be measured by the symbols and objects in the painting. Symbols are visual words that often express something about the meaning of a work as well. For example, the works of Renaissance artists such as Giovanni Bellini and Jan Van-Eyck use religious symbols such as a cross, wings, and animals to tell stories in the Bible. This shows the need for an object-based representation of images. We should be able to describe the painting from a list of many different object classes. By having an object-based representation, the image is described in a high-level semantic as opposed to low-level seman- tics such as color and texture. By using the Classemes [11] feature, we are able to capture both high-level and low-level semantics. For example, Figure 2 may not look like similar images, but when considering the objects placed in each of the paintings, similarity becomes clear. This comparison is a result from our experiments which we describe later. 200 K. Abe, B. Saleh, and A. Elgammal Fig. 2. Fr´ed´eric Bazille’s Studio 9 Rue de la Condamine (left) and Norman Rockwell’s Shuffleton’s Barber Shop (right). The composition of both paintings is divided in a similar way. Yellow circles indicate similar objects, red circles indicate composition, and the blue square represents similar structural element. The objects seen – a fire stove, three men clustered, chairs, and window are seen in both paintings along with a similar position in the paintings. After browsing through many publications and websites, we conclude that this comparison has not been made by an art historian before. One important factor of finding influence is therefore having a good measure of similarity. Paintings do not necessarily have to look alike but if they do or have reoccurring objects (high-level semantics), then they will be considered similar. If influence is found by looking at similar characteristics of paintings, the importance of finding a good similarity measure becomes prominent. Time is also a necessary factor in determining influence. An artist cannot influence another artist in the past. Therefore the linearity of paintings cuts down the possibilities of influence. By including a computer’s intuition about which artists and paintings may have similarities, it not only finds new knowledge about which paintings are connected in a mathematical criteria but also keeps the conversation going for artists. It challenges people to consider possible connections in the timeline of art history that may have never been seen before. We are not asserting truths but instead suggesting a possible path towards a difficult task of measuring influence. The main contribution of this paper is proposing the interesting task of determining influence between artist as a knowledge discovery problem and proposing a new relevant dataset. To the best of our knowledge, Carneiro et al[4] recently published the ”PRINTART” on paintings along with primarily experiments on image retrieval and genre classification. However this dataset contains only monochromatic artistic images. Our dataset have chromatic images and its size is about double the ”PRINTART” dataset. An Early Framework for Determining Artistic Influence 201 2 Dataset Our novel dataset contains a total of 1710 works by 66 artists chosen from Mark Harden’s Artchive database of fine-art. Each image is annotated with the artist’s first name, last name, title of work, year made, and genre. The majority of the images are of the full work while a few are details of the work. We are primarily dealing with paintings but we have included very few images of sculptures as well. The artist with the most images is Paul C´ezanne with 140 images and the artist with the least number of works is Hans Hoffmann with 1 image. The artists themselves ranged from 13 different genres throughout art history. These include Expressionism (10 artists), Impressionism (10), Renaissance (12), Romanticism (5), Cubism (4), Baroque (5), Pop (4), Abstract Contemporary (7), Surrealism (2), American Modernism (2), Post-Impressionism (3), Symbol- ism (1), and Neoclassical (1). The number in the parenthesis refers to the number of artists in each genre. Some genres were condensed such as Abstract Contem- porary, which includes works in the Abstract Expressionism, Contemporary, and De Stijl periods. The Renaissance period has the most images (336 images) while American Modernism has the least (23 images). The average number of images per genre is 132. The earliest work is a piece by Donatello in 1412 while the Fig. 3. Examples of paintings from thirteen genres: Renaissance, Baroque, Neoclassical, Romanticism, Impressionism, Post-Impressionism, Expressionism, Cubism, Surrealism, Symbolism, American Modernism, Pop, and Abstract Contemporary. most recent work is a self portrait by Gerhard Richter done in 1996. The earliest genre is the Renaissance period with artists like Titian and Michelangelo during the 14th to 17th century. As for the most recent genre, art movements tend to overlap more in recent years. Richter’s painting from 1996 is in the Abstract Contemporary genre. 3 Influence Discovery Framework Consider a set of artists, X. For each artist, Xi, we have a ground truth time period ti that artist Xi has performed his work. Also consider a set of images Ii, for each artist Xi. We extract Classeme features [11] as visual features for each 202 K. Abe, B. Saleh, and A. Elgammal image and represent it by a vector called C =[c1, ..., cN ]whereN represents the dimension of the feature space. We represent the problem of influence as similarity following time. For the statement Xi ⇒ Xj to be true, where the arrow indicates the left side influencing the right side, two requirements must be met.