sign in sign up
<<
Home , Art history, Divisionism, History of art, Model (art), Painting, The arts

Downloaded by guest on September 27, 2021 www.pnas.org/cgi/doi/10.1073/pnas.1800083115 Sigaki D. Y. entropy Higor of lens complexity the and through of ino rwrsi uhmr eet mn h eia works, seminal the Among the recent. characteriza- more However, much the is ). to artworks of techniques an tion quantitative and in such image) of ratio elements an application the in of found as (number regularities defined complexity of is book (number measure order 1933 aesthetic between the quantitative to a has back where art traced of be Measure can the Aesthetic in Efforts techniques precedent. quantitative some of use the ically, and image as as well to such as contribute matters, classification. evolution, can practical cultural more to of visual understanding con- of better often characterization approaches is a Quantitative the art qualitative. of at study intrinsically aimed the because be because only to also not sidered but goals, shortage research would data unimaginable ago. of arts decade such visual a among of even be characterization unimaginable quantitative been as at large-scale have (14–18), well A would review as that peer (10–12), human scale and a communication production, of and study (13), researchers 9), the impressive enables (8, to deluge an mobility methods data (5–7), at quantitative digital subjects bring This that diverse to this detail. information most of of digital the degree of popularity about part amount available and large unprecedented in is impact years, the recent The to in steadily due (4). finding growing been of has research goal these the of dynam- share the ics govern that usually universalities and studies principles fundamental Such (1–3). systems P history art of classification effective complexity– an identi- applied for to the the used used be artworks. and that content can styles textual measures verified entropy the the further with describe according have well qualitatively styles correspond We dis- of groups metrics. fied a clustering allowing these have and thus complexity, to styles organization and art. hierarchical artistic entropy of of a different periods degree historical that average by main tinct shows marked these the art, research of with of Our agree behavior evolution temporal that dynamical clear transitions The and a order–disorder . reveals of measures art categories scale qualitative by a degree reflects proposed into the locally artworks map that statistical of measures simplicity–complexity the order These and visual . - entropy of each these permutation of of the complexity the history. estimate in art we almost of spatial ings, local of millennium the a analysis on evolution. nearly Based quantitative therefore spanning large-scale cultural paintings, is a human 140,000 art present understanding we of better Here, analysis corresponding for the quantitative of essential The deeply culture is epoch. and that historical 2018) the human 3, January of by review expression for influenced (received ultimate 2018 the 19, July is approved Art and TX, , University, Rice Levine, Herbert by Edited , 1080 Hub, Science Slovenia; Maribor, 2000 Maribor, of a eatmnod F de Departamento hl h cl fsm urn tde a hne dramat- changed has studies current some of scale the While ierneo icpie,icuigeooi n social and economic including disciplines, of range to wide applied a successfully been have approaches hysics-inspired | paintings sc,Uiesdd sauld Maring de Estadual Universidade ´ ısica, yteAeia ahmtca ikof(19), Birkhoff mathematician American the by a | Matja , complexity Perc z ˇ | c colo lcrncadIfrainEgneig ehn nvriy ejn 011 hn;and ; 100191, University, Beihang , Information and Electronic of School entropy b,c,d,1 | n aod .Ribeiro V. Haroldo and , pta patterns spatial ,Maring a, ´ ,P 72-0,; 87020-900, PR a, ´ a,1 1073/pnas.1800083115/-/DCSupplemental. at online information supporting contains article This 1 the under Published Submission. Direct PNAS a is article This interest. of conflict no . declare the authors wrote The and data, research, analyzed performed tools, research, reagents/analytic designed new H.V.R. contributed and M.P., H.Y.D.S., contributions: Author datasets large-scale of and analysis Manovich the by concerning done (41–43) research same styles. coworkers innovative artistic the also about that is information also there capture Notably, showed to used Among and be 1850, contrast. can after quantity diver- contrast the color in the increase of sudden of sity a observed evolution have paint- they the 180,000 findings, other almost on more analyzed a have focusing In (40) ings, al. years. et the Lee over these work, trend recent of increasing an grayscale displays roughness paintings the the with the that associated moreover, that historical and, exponent finding among paintings, western images, different of 29,000 remarkably periods analyzed is (39) distribution In al. color-use perceptive. historical et art Kim large-scale 2014, a from paintings study to assessment stylometry and procedures. problems, restora- authentication artwork on tools, also tion focusing been (36–38), have journals contributions scientific where of confer- issues several special in and field proceedings documented growing ence comprehensively rapidly are and research emerging art this of of (29–31), The advances (33–35). statistical recent expressions most visual the and other 27), many (28) and (26, (32), paintings movements artists particular specific of of and properties evolution (21–25) the paintings of study be authenticity to the can arts, determine related visual article and to analysis of methods fractal research study of applications quantitative This further the many career. inspiring for over artistic landmark dimension a his paint- fractal considered Pollock’s of increasing where course an (20), by al. the et characterized Taylor are by ings article the have we owo orsodnemyb drse.Eal ajzpr@n-bs or [email protected] Email: addressed. be may correspondence hvr@dfi.uem.br. whom To ritcsye n rwrsbsdo h ereo oa order local of degree paintings. the the different on in distinguish based to artworks us and evolutionary allow styles art measurable also artistic They in and arts. visual historical in canonical a trend present to reveal We well and map artwork. history paintings that the in scales about numerical patterns information ordering crucial are spatial encode that local metrics from of physics-inspired scale estimated simple Our the that methods. in of computational shows of shift research availability use a the recent through by enables The analysis investigated collections such time. art be reasonable digitized can in large that expert artworks art of an number This the comparative. essentially limits is paintings of inquiry critical The Significance odt,rltvl e eerhefrshv endedicated been have efforts research few relatively date, To b aut fNtrlSine n ahmtc,University , and Natural of Faculty y NSlicense. PNAS www.pnas.org/lookup/suppl/doi:10. NSLts Articles Latest PNAS d Complexity | f10 of 1

APPLIED PHYSICAL SCIENCES of paintings and other visual art expressions by means of the tal in this regard. They have proposed to distinguish artworks estimation of their average brightness and saturation. from different periods through a few visual categories and quali- However, except for the introduction of the roughness expo- tative descriptors. Visual comparison is undoubtedly a useful tool nent, preceding research along similar lines has been predom- for evaluating artistic . However, it is impractical to apply inantly focused on the evolution of color profiles, while the at scale. This is when computational methods show their great- spatial patterns associated with the pixels in remain est advantage. Nevertheless, to be useful, it is important that poorly understood. Here, we present a large-scale investiga- derived metrics are still easily interpreted in terms of familiar tion of local order patterns over almost 140,000 visual artwork and disciplinary-relevant categories. images that span several hundred years of . By calcu- We note that the complexity–entropy plane partially (and lating two complexity measures associated with the local order locally) reflects Wolfflin’s¨ dual concepts of linear versus painterly of the pixel arrangement in these artworks, we observe a clear and Riegl’s dichotomy of haptic versus optic artworks. According and robust temporal evolution. This evolution is characterized to Wolfflin,¨ “linear artworks” are composed of clear and outlined by transitions that agree with different art periods. Moreover, shapes, while, in “painterly artworks,” the contours are subtle the observed evolution shows that these periods are marked by and smudged for merging image parts and passing the idea of distinct degrees of order and regularity in the pixel arrange- fluidity. Similarly, Riegl considers that “haptic artworks” depict ments of the corresponding artworks. We further show that these objects as tangible discrete entities, isolated and circumscribed, complexity measures partially encode fundamental concepts of whereas “optic artworks” represent objects as interrelated in art history that are frequently used by experts for a qualitative deep space by exploiting light, color, and shadow effects to description of artworks. In particular, the complexity measures create the idea of an open spatial continuum. The notions of distinguish different artistic styles according to their average order/simplicity versus disorder/complexity in the pixel arrange- order in the pixel arrangements, enable a hierarchical organiza- ments of images captured by the complexity–entropy plane tion of styles, and are also capable of automatically classifying partially encode these concepts. Images formed by distinct and artworks into artistic styles. outlined parts yield many repetitions of a few ordinal patterns, and, consequently, linear/haptic artworks are described by small Results values of H and large values of C . On the other hand, images Our results are based on a dataset comprising 137,364 visual art- composed of interrelated parts delimited by smudged edges pro- work images (mainly paintings), obtained from the online visual duce more random patterns, and, accordingly, painterly/optic arts WikiArt (https://www.wikiart.org). This web- artworks are expected to yield larger values of H and smaller page is among the most significant freely available sources for values of C . It is also worth mentioning that Wolfflin’s¨ and visual arts. It contains artworks from over 2,000 different artists, Riegl’s dual concepts are limiting forms of representation that covering more than a hundred styles, and spanning a period on demarcate the scale of all possibilities (51). In this regard, the the order of a millennium. Each one of these image files has been continuum of H and C values may help art historians to grade converted into a matrix representation whose dimensions corre- this scale. spond to the image width and height, and whose elements are In this context, we ask whether the scale defined by H and C the average values of the shades of , green, and (RGB) values is capable of unveiling any dynamical properties of art. To of the pixels in the RGB color space. For further details, we refer answer this question, we estimate the average values of H and C to Materials and Methods. after grouping the images by date. Because the artworks are not From this matrix representation of the artwork images, we uniformly distributed over time (see Materials and Methods), we calculate two complexity measures: the normalized permuta- have chosen time intervals containing nearly the same number of tion entropy H (44) and the statistical complexity C (45). As images in each time window. Fig. 1 shows the joint evolution of described in Materials and Methods, both measures are evalu- the average values of C and H over the years (i.e., the changes ated from the ordinal probability distribution P, which quantifies in the complexity–entropy plane), where a clear and robust the occurrence of the ordinal patterns among the image pixels (SI Appendix, Fig. S1) trend is observed. This trajectory of H and at a local scale. Here, we have estimated this distribution by C values shows that the artworks produced between the ninth considering sliding partitions of size dx = 2 by dy = 2 pixels (the and the 17th centuries are, on average, more regular/ordered embedding dimensions), leading to (dx dy )! = 24 possible ordi- than those created between the 19th and the mid-20th century. nal patterns. The value of H quantifies the degree of disorder Also, the artworks produced after 1950 are even more regu- in the pixel arrangement of an image: Values close to 1 indicate lar/ordered than those from the two earlier periods. We observe that pixels appear at random, while values close to zero indicate further that the pace of changes in the complexity–entropy plane that pixels appear almost always in the same order. More-regular intensifies after the , a period that coincides with images (such as those produced by ) are expected to the emergence of several artistic styles (such as have small entropy values, while images exhibiting less regular- and ), and also with the increase in the diversity of ity (such as Pollock’s drip paintings) are characterized by large color contrast observed by Lee et al. (40). values of entropy. The statistical complexity C , in turn, measures The three regions in Fig. 1 defined by the values of H and the “structural” complexity present in an image (46, 47): It is zero C correspond well with the main divisions of art history. The for both extremes of order and disorder in the pixel arrangement, first period (black rectangle) corresponds to , the and it is positive when an image presents more-complex spatial , Neoclassicism, and , which developed patterns. The joint use of the values of C and H as a discriminat- until the 1850s (52). The second period (red rectangle) corre- ing tool give rise to a “complexity–entropy plane” (45, 48), which sponds to , marked by the birth of Impressionism is a technique that has proven to be useful in several applications. in the 1870s, and by the development of several avant-garde Herein the complexity–entropy plane is our approach of choice artistic styles (such as , , and ) for quantifying the characteristics of different visual artworks. during the first decades of the 20th century. Finally, the latest period corresponds to the transition between Modern Art and Evolution of Art Contemporary/. The specific date marking the A careful comparison of different artworks is one of the main beginning of the Postmodern period is still an object of fierce methods used by art historians to understand whether and how debate among art experts (52). Nevertheless, there is some con- art has evolved over the years. Works by Heinrich Wolfflin¨ (49) sensus in that Postmodern Art begins with the development of and (50), for example, can be considered fundamen- in the 1960s (52).

2 of 10 | www.pnas.org/cgi/doi/10.1073/pnas.1800083115 Sigaki et al. Downloaded by guest on September 27, 2021 Downloaded by guest on September 27, 2021 iaie al. et Sigaki eldfie de sc si adEg anigadO Art Op and and large Painting of Edge use Hard 54)]. the (53, in artworks by as marked [such recognizable, and edges instantly well-defined objects, being ordinary as of art of made idea agree painterly/optic Postmodern to from the appears change fact with This rapid modes. and representation marked linear/haptic intense is to more Art even Postmodern to an and brushstrokes by Modern Intriguingly, smudged between 54). 53, and transition (49, looser the edges pronounced of of creation use the styles the avoid modern by Expressionism while marked and 54), are , 53, , (49, Impressionism, other as surfaces each such flat from by distinguished separated Romanticism art- rigidly and that and objects idea Neoclassicism, represent the Renaissance, usually with the the agrees from in thus works painterly/optic This Art to modes. Modern linear/haptic representation and from change before a represents produced art the W between of concepts the and periods them. art among different transitions identifies the observe and correctly We plane Art). complexity–entropy Contemporary/Postmodern the green: that regions Romanti- Art; highlighted and Modern Neoclassicism, The red: Renaissance, SEM. cism; (black: the periods represent art bars different Error show permutation of plot). the of values in values average (shown the average to the corresponds of evolution temporal entropy the is Shown 1. Fig. ie,sc saatto,ab-rdc ftebanscomplexity, brain’s These the 57). of by-product perspec- (56, a different adaptation, art as from such of art tives, consider complexity– theories theories the evolutionary proposed by recently is unveiled plane behavior dynamical entropy the of anisms change particular Gaiger, a to time. For linked over not history. and concepts W of art descriptive purely of work of categories when the dual development hold as the conceptions entire these such the of historians, neither analyzing that plane. art argue complexity–entropy of (51), the Gaiger persis- studies in the local recent changes the with the Indeed, with of consistent compatible behavior not more tent is conception seems cyclical of that this behavior transition dynamical and this overall single a of W How- through while 1. tion occurs (56), Fig. development representation, process in this observed the continuous of Riegl, transition from for first change the ever, painterly/optic with a agrees through the which develops to art linear/haptic that consider Riegl h ocpin fathsoypooe yW by proposed history art of conceptions The plane complexity–entropy the between analogy the carrying By nte osblt o nesadn h neligmech- underlying the understanding for possibility Another uniyn h vlto fatok hog h itr fart. of history the through artworks of evolution the Quantifying H n ttsia complexity statistical and lflnadRelfrad h transition the forward, Riegl and olfflin ¨ lflnadRelsol etetdas treated be should Riegl and olfflin ¨ C H cmlxt–nrp ln) ahdot Each plane). (complexity–entropy H lflnhsaccia concep- cyclical a has olfflin ¨ and and C C nteohrhand, other the On . o ie ieinterval time given a for lflnand olfflin ¨ 2sye aigmr hn10iae ah(orsodn to (corresponding the each to see images analysis data; 100 this of than limit of ∼90% more also values having We average styles style. 92 the by dataset. images calculate our the we in grouping capable so, styles is do artistic plane To different complexity–entropy among the discriminating whether of ask now We Styles Artistic Among Distinguishing follow 1. Fig. to in movements depicted artistic path and historical artists the driven has also could what and explain diversity, public and innovation, unprecedented the specialization, its of and toward degree art artists propelling prefer- for among responsible these be mechanism expand would and feedback attention his- This and engage preference art ences. to audience desire theories, between ’s interplay these the the by to social driven According is foster tory to creativity. and was attention, and contribution sharing cohesion evolutionary at art’s aimed that selection suggest natural and sexual or itneyed 4gop fsye niae ytedifferent the by indicated styles of 3. Fig. threshold groups in This styles. 14 of cohe- yields clusters the the maximizes distance among that separation distance and threshold in sion optimal described the is (as dataset. 0.03 (59) our Methods coefficient in and silhouette styles Materials the merging artistic maximizing for the By among criterion relationship the 3B intricate Fig. as clusters. that variance of procedure pairs within-cluster clustering distance hierarchical the the a of uses is representation method Ward dendrogram This by a matrix. proposed method construct variance to minimum (58) the use we cally, style of formation the observe groups. dis- qualitatively more we 3A where considered Fig. distances, are other. distances each from large similar by styles, pairs separated artistic whereas styles them, two between of similarity between the distance is significant in the more the styles closer have the of we between Thus, pair so, measure them. dissimilarity do a a as To between plane organization ordering. complexity–entropy distance the local hierarchical Euclidean this possible the to a considered respect for This with test pixels. styles image to of of us ordering enables local fact the regarding styles artistic of values The Styles Artistic the of avoid Hierarchical to colors 54). (53, blending edges by sharp and of also smudged creation of and use the brushstrokes, by diffuse characterized are Fauvism) and lism, values smallest abrupt the by displaying of Styles well-delimited 54). the are (53, by colors that of marked elements transitions all are simple which of Painting, use Field Color of and Painting, values of smallest values the highest the the rep- reflect and displaying of styles partially modes the Among painterly/optic values resentation. versus extreme linear/haptic into of whose styles values dichotomy different scale complexity the continuum maps larger Impressionism, a arrangement Expressionism, and This as entropy Fauvism). (such and styles smaller of modern of are values than region average a the in in trend general C the with ment values. average indistinguishable com- statistically pairwise with the styles the observe over of see spread majority (∼92%; the are aver- parisons for of styles the different values artistic significantly for average are the the values and that reliable plane, obtain shows complexity–entropy to 2 one) Fig. each ages. of images of oivsiaetecutrn ewe ritcsye systemati- styles artistic between clustering the investigate To ent ute httearneeto tlsi nagree- in is styles of arrangement the that further note We C vrtm nwihms otoensye r localized are styles Postmodern most which in time over n h ihs ausof values highest the and H and i.S5 Fig. Appendix, SI C .Hwvr ealso we However, ). S7 Fig. Appendix, SI and atr h ereo iiaiyamong similarity of degree the capture eit hsdnrga,uviigan unveiling dendrogram, this depicts H ,w n that find we S8 ), Fig. Appendix, SI efidMnmls,Hr Edge Hard Minimalism, find we , H sc sIpesoim Pointil- Impressionism, as (such hw h arxpo fthese of plot matrix the shows NSLts Articles Latest PNAS o aeadnumber and name for H and H C | H and f10 of 3 after and C C

APPLIED PHYSICAL SCIENCES Fig. 2. Distinguishing among different artistic styles with the complexity–entropy plane. The colored dots represent the average values of H and C for each one of the 92 styles with more than 100 images in our dataset. Error bars represent the SEM. For better visualization, only 41 artistic styles with more than 500 images each are labeled (see SI Appendix, Fig. S6 for all styles).

These groups partially reflect the temporal localization of dif- each artistic style. To do so, we have obtained the textual content ferent artistic styles and their evolution reported in Fig. 1. In of these webpages and extracted the top 100 keywords of each particular, several styles that emerged together or close in time one by applying the term frequency–inverse document frequency are similar regarding the local arrangement of pixels and thus approach (61). We consider the inverse of 1 plus the number belong to the same group. For instance, the first five groups of of shared keywords between two styles as a measure of simi- Fig. 3B contain mainly Postmodern styles. On the other hand, larity between them. Thus, styles having no common keywords these groups and their hierarchical structure organize the styles are at the maximum “distance” of 1, while styles sharing several regarding their mode of representation in the scale delimited by keywords are at a closer distance. the dichotomy of linear/haptic versus painterly/optic. This fact is By using a similar hierarchical clustering procedure to the more evident when examining groups in both extremes of order one used in Fig. 3, we obtain 24 clusters of artistic styles from and regularity in the complexity–entropy plane. The right-most the Wikipedia text analysis (SI Appendix, Fig. S9). This num- group of Fig. 3B, for example, contains styles that use relatively ber of clusters is much larger than the 14 clusters obtained small brush strokes and avoid the creation of sharp edges. This from the complexity–entropy plane. However, both clustering fact is particularly evident in artworks of Impressionism, Pointil- approaches share similarities, which can be quantified by using lism, and , but it is also evident in Neo- and the clustering evaluation metrics homogeneity h, completeness Neo-Romanticism, and in the works of muralists (such as David c, and v measure (62). Perfect homogeneity (h = 1) implies Siqueiros and Jos´e Orozco), as well as in the abstract paintings that all clusters obtained from the Wikipedia texts contain of P&D ( and Decoration). While devoted to patterning only styles belonging to the same clusters obtained from the paintings (such as printed fabrics), P&D is considered a “reac- complexity–entropy plane. On the other hand, perfect com- tion” to Minimalism and (which are located in pleteness (c = 1) implies that all styles belonging to the same the other extreme of the complexity–entropy plane) that avoids cluster obtained from the complexity–entropy plane are grouped restrained compositions by means of a subtle modulation of col- in the same cluster obtained from the Wikipedia texts. The v ors as in the works of Robert Zakanitch, who is considered one of measure is the harmonic mean between h and c, that is, v = the founders of P&D (60). As we move to groups characterized 2hc/(h + c). Our results yield h = 0.49, c = 0.40, and v = 0.44, by high complexity and low entropy, we observe the cluster- which are values significantly larger than those obtained from ing of styles marked by the presence of sharp edges and very a null where the number of shared keywords is ran- contrasting patterns, usually formed by distinct parts isolated domly chosen from a uniform distribution between 0 and 100 or combined with unrelated materials. That is the case for the (hrand = 0.42 ± 0.02, crand = 0.35 ± 0.01, and vrand = 0.38 ± 0.01; group containing , Pop Art, and , but also average values over 100 realizations). Therefore, the similari- for the group formed by , Hard Edge Painting, and ties between the two clustering approaches cannot be explained Concretism (53, 54). by chance. This result indicates that, despite the very local We can also verify the meaningfulness of these groups by com- character of our complexity measures, the values of H and paring the clustering of Fig. 3B with an approach based on the C reflect the meaning of some keywords used for describing similarities among the textual content of the Wikipedia pages of artistic styles.

4 of 10 | www.pnas.org/cgi/doi/10.1073/pnas.1800083115 Sigaki et al. Downloaded by guest on September 27, 2021 Downloaded by guest on September 27, 2021 niae yteclrdbace r bandb utn h edorma h hehl itne00.Ti au aiie h ihutecoefficient al. dendrogram. et silhouette the Sigaki the in maximizes used value as This same styles the 0.03. of is distance groups plot threshold 14 matrix the The the at (58). in dendrogram Ward by the and proposed rows cutting method by variance obtained minimum are (see the (59) branches applying colored by obtained the matrix by distance indicated the of representation Dendrogram (B) 3. Fig. B A arxpo fteEciendsac ntecmlxt–nrp ln ewe vr aro styles. of pair every between plane complexity–entropy the in distance Euclidean the of plot Matrix ( A) styles. artistic of organization Hierarchical aeil n Methods and Materials and IAppendix SI ,adi stu ntrl ubrfrdfiigtenme fcutr nordtst h re of order The dataset. our in clusters of number the defining for number “natural” a thus is it and S8), Fig. , NSLts Articles Latest PNAS | f10 of 5

APPLIED PHYSICAL SCIENCES Predicting Artistic Styles investigate this, we again use a stratified n-fold -validation Another possibility of quantifying the information encoded by strategy with n = 10 for estimating the learning curves. Fig. 4B the values of H and C is trying to predict the style of an image shows the training and cross-validation scores for the k-nearest based only on these two values. To do so, we have implemented neighbors, where we observe that both scores increase with train- four well-known machine learning algorithms (63, 64) (nearest ing size. However, this enhancement is very small when more neighbors, random forest, support vector machine, and neural than ∼50% of the data are used for training the model. SI network; see Materials and Methods for details) for the classi- Appendix, Fig. S10 shows results analogous to those presented fication task of predicting the style of images for all 20 styles in Fig. 4 as obtained with the other three machine learning that contain more than 1,500 artworks each. For each method, algorithms. we estimate the validation curves for a range of values of the By combining the previous analysis with a grid search algo- main parameters of the algorithms with a stratified n-fold cross- rithm, we determine the best combination of parameters enhanc- validation (63) strategy with n = 10. Fig. 4A shows the validation ing the of each statistical learning method. Fig. curves for the k-nearest neighbors as a function of the num- 4C shows that the four algorithms display similar , ber of neighbors. We note that this method underfits the data all exhibiting accuracies close to 18%. We have further com- if the number of neighbors is smaller than ∼250. Conversely, pared these accuracies with those obtained from two dummy the cross-validation score saturates at ∼0.18 if the neighbors classifiers. In the stratified classifier, style predictions are gen- are 300 or more, and there is no overfitting up to 500 neigh- erated by chance but respecting the distribution of styles, bors. Another relevant issue for statistical learning is related to while predictions are drawn uniformly at random when using the number of data necessary to properly train the model. To the uniform classifier. The results in Fig. 4C show that all machine learning algorithms have a significantly larger accu- racy than is obtained by chance. This result thus confirms that the values of H and C encode important information about AB the style of each artwork. Nevertheless, the achieved accu- 0.5 0.185 racy is quite modest for practical applications. Indeed, there are other approaches that are more accurate. For instance, 0.4 0.180 Training score Zujovic et al. (65) achieved accuracies of ∼70% in a classifi- Cross-validation score 0.3 0.175 cation task with 353 paintings from five styles, and Argarwal Scores Scores et al. (66) reported an accuracy of ∼60% in a classification task 0.2 0.170 Training score Cross-validation score with 3,000 paintings from 10 styles. However, our results can- 0.1 0.165 not be directly compared with those works, since they use a 0 100 200 300 400 500 0.2 0.4 0.6 0.8 much smaller dataset with fewer styles and several image fea- Number of neighbors Training size tures, while our predictions are based only on two features. Our C 0.20 approach represents a severe dimensionality reduction, since images with roughly 1 million pixels are represented by two num- 0.15 bers related to the local ordering of the image pixels. In this context, an accuracy of 18% in a classification with 20 styles 0.10 and more than 100,000 artworks is not negligible. Moreover,

Accuracy the local of H and C makes these complexity measures 0.05 very fast, easy to parallelize, and scalable from the compu- 0.00 tational point of view. Thus, in addition to showing that the

rm) complexity–entropy plane encodes important information about o the artistic styles, we believe that the values of H and C , com- bined with other image features, are likely to provide better

Random Forest Machine (RBF)Neural Network classification scores. or Dummy (Unif Nearest Neighbors Dummy (Stratified) Discussion and Conclusions We have presented a large-scale characterization of a dataset composed of almost 140,000 artwork images that span the lat- Fig. 4. Predicting artistic styles with statistical learning algorithms. (A) est millennium of art history. Our analysis is based on two Training and cross-validation scores of the nearest neighbors algorithm as a relatively simple complexity measures (permutation entropy H function of the number of neighbors. We note that the algorithm underfits and statistical complexity C ) that are directly related to the the data for values of the number of neighbors smaller than ∼250, but there ordinal patterns in the pixels of these images. These measures is no significant accuracy improvement for larger values nor for overfitting map the local degree of order of these artworks into a scale up to 500 neighbors. (B) Learning curve, that is, the training and cross- of order–disorder and simplicity–complexity that locally reflects validation scores, as a function of the training size (fraction of the whole data) for the nearest neighbors algorithm with the number of neighbors the qualitative description of artworks proposed by Wolfflin¨ and equal to 400. We observe no significant improvement in the cross-validation Riegl. The limits of this scale correspond to two extreme modes scores when more than 50% of the data are used to train the model. of representation proposed by these art historians, namely, to In both plots, the shaded regions are 95% confidence intervals obtained the dichotomy between linear/haptic (H ≈ 0 and C ≈ 0) and with a 10-fold cross-validation splitting strategy. (C) Comparison between painterly/optic (H ≈ 1 and C ≈ 0). four different statistical learning algorithms (nearest neighbors, random By investigating the dynamical behavior of the average val- forest, support vector machine, and neural network; see SI Appendix, Fig. ues of the complexity measures used, we have found a clear and S10 for details of the parameters for each algorithm) as well as the null robust trajectory of art over the years in the complexity–entropy accuracy obtained from two “dummy” classifiers (stratified: generates ran- plane. This trajectory is characterized by transitions that agree dom predictions respecting the style distributions; uniform: predictions are uniformly random). Error bars represent the SEM. The four classifiers have with the main periods of art history. These transitions can be clas- similar accuracy (∼18%), and they all significantly outperform the dummy sified as linear/haptic to painterly/optic (before and after Modern classifiers. These results are based on the 20 styles with more than 1,500 Art) and painterly/optic to linear/haptic (the transition between images each, although similar results are obtained when including others Modern and Postmodern Art), showing that each of these his- styles as well (SI Appendix, Fig. S11). torical periods has a distinct degree of entropy and complexity.

6 of 10 | www.pnas.org/cgi/doi/10.1073/pnas.1800083115 Sigaki et al. Downloaded by guest on September 27, 2021 Downloaded by guest on September 27, 2021 ln vr 7y lo oeta 0 fteeatok eeproduced were artworks these of 50% than more Also, y. 77 every equal bling time characteristic Fig. the with 2016. to growth and exponential observe 1031 an we by years where approximated the dataset, between our created in 5B were 5A year artworks Fig. per files). these images (33,724 that of specified all number not excluded the have were depicts we dates evolution, composition that temporal of whose the users images of style the analysis by artistic the maintained For and by collaboratively webpage. provided date, and labels generated artists), style are The different WikiArt Baroque). 2,391 and down- artwork, Surrealism, are we each Impressionism, (there 2016, (e.g., to of related painter August metadata as in and such WikiArt to images of available digitalized pages artworks 137,364 web visual loaded the the of crawling of datasets By available one date. freely is and which online (https://www.wikiart.org/), largest WikiArt encyclopedia arts Data. Methods and Materials consider move- may projections spiral possible we a the 234), of W only p. one of that 49, to as (ref. plane but return complexity–entropy facts.” stood, not the the once did meet it here would even which ment art at that point shows the soon certainly of context tion the In evolution. their information W and relevant style, more carry their and, artworks, do historians about measures art these by that proposed can importantly, concepts metrics to physics-inspired of connected nev- simple all be results our capture that However, cannot demonstrate art. course, ertheless of of complexity they, and uniqueness artwork, the an of scale local information meaningful turn carry style. in artwork measures about classifiers, two these dummy that outperforms showing 18% approximately of iaie al. et Sigaki predicted is image an a of of of style values means the the infor- on by solely which of based measures in amount complexity task these the classification in quantified encoded further mation have well-defined/sharp/abrupt We or transitions. soft/smudged/diffuse the regarding also of particular they in content and presence them, style, textual among each similarities the describing some well for reflect used reflect We pages groups plane. Wikipedia the these of on that position verified their to have orga- according hierarchically grouped be these can and using similar- styles nized different By a that concepts. find history represent we art distances, plane oppos- these complexity–entropy regarding these pairs measure between the of ity distances in use the styles the Also, of quantifying representation. of for be modes ruler can ing thus a reflects and as partially painterly/optic, plane considered versus complexity–entropy linear/haptic the location duality the in the that style emphasize each We of entropy–complexity artworks. of corresponding degree the average in the to according styles tic of in from II, of transition beginning World the the books. of history marks that end context usually envisage the that the to by event driven the difficult in in was not certainly and, Postmodern is to “lies 233), Modern it linear p. 1, (ref. the Fig. 49, art” to of of (ref. return history be the circumstances” same of would However, outward the problem law 17). in this central p. law determine the 49, “natural “to problem, and central a growth,” a by physical governed as is way painterly in depicted to evolution W linear global For the plane. with complexity–entropy consistent the histo- quite of art is contemporary values it other and rians, the (51) withstand Gaiger in not of scrutiny persistence does critical time painterly and local linear the between transition cal W While lflnsmtpo bu h vlto fat Acoe inspec- closer “A art: of evolution the about olfflin’s ic u w opeiymaue r ae nieyo the on entirely based are measures complexity two our Since nadto ouviigti yaia seto r,tevalues the art, of aspect dynamical this unveiling to addition In ¨ τ H hw httecmltv rcino rwrsi u aae swell is dataset our in artworks of fraction cumulative the that shows = lflnsspiral. olfflin’s ¨ and h iia mgsue nti td eeotie rmtevisual the from obtained were study this in used images digital The 111 ± lflnscneto fathsoyi em facycli- a of terms in history art of conception olfflin’s ¨ C .Cneunl,tecmltv ubro rwrsi dou- is artworks of number cumulative the Consequently, y. 1 r aal fdsigihn ewe ifrn artis- different between distinguishing of capable are H and C h bandscesrate success obtained The . lfln h rniinfrom transition the olfflin, ¨ H and C o the nor hl rsn hstcnqetruhasml xml framr formal matrix more the a Let articles). (for original example we the simple Here, to refer 47). a we (46, through general- description, images technique been as this has such present data it shall higher-dimensional recently, with only use and, to 48), ized (44, series time characterizing entropy permutation normalized complexity cal the calculate we images, Plane. Complexity–Entropy The value. average simple the of using values to the resorted for therefore 0.992 have and 0.989 is average ple the between correlation linear of different Pearson values against the robust sensibil- example, remarkably For are color results choices. the our weighting mimic but eye, to human chosen the of often ity are values weighting These tively. luminance and or 0.2125R G, calculating reflectance to grayscale corresponds the and most (67) defines the of values One values. weighting the different the by over common to weighted average similar usually the is is that channels approach color except three This images, file. of image transformation each grayscale pixel, each usual for of matrix ele- shades color simple the three analysis, a the our and of yielding For value intensity. channels, average color the color the calculated have represent the we 255) to to 0 from correspond (ranging as layers ments of the thought where be RGB dimensions can 256 height), the of file allows matrix image in total, an three-layer colors in purposes, a which, blue practical blue, For and 256 variations. and green, by color green, characterized red, is red, image for the of of each shades pixel each bits that (8 meaning space”), pixel “color per bits the 24 by Files. Image marked of is Representation Matrix that movements. period art a different of century, variety 20th large the a of of development decade first the after 50% than more 1912. particular, year in the century; after 20th painted the were of onset the after [f produced fraction This [f to scale. growth equal log-linear exponential an a by on mated curve) (blue years the mgsprya nordtst (B dataset. our in year per images 5. Fig. B B A h ubrof number The (A) years. the over images artwork of Distribution τ H tn o h hd neste frd re,adbu,respec- blue, and green, red, of intensities shade the for stand = acltdwt h sa rycl rnfrainadtesim- the and transformation grayscale usual the with calculated 111 C ± o ahoe hstcnqewsoiial rpsdfor proposed originally was technique This one. each for dse ie.W bev htms rwrswere artworks most that observe We line). (dashed y 1 yuigtemti ersnaino all of representation matrix the using By uuaiefato fatok over artworks of fraction Cumulative ) n (t x l mg lsaei PGfra with format JPEG in are files image All ) teiaewdh by width) image (the ∝ exp(t + NSLts Articles Latest PNAS 0.7154G /τ C ]wt hrceitctime characteristic with )] IAppendix (SI + where 0.0721B, 3 (t H n = ]i approxi- is )] .We S2). Fig. , y n statisti- and 6 7,216 777, 16, teimage (the | f10 of 7 R,

APPLIED PHYSICAL SCIENCES 6 0 2 We estimate the values of H(P) and C(P) for all 137,364 images by A = 4 5 2 considering the embedding dimensions dx = dy = 2. As we previously men- 6 7 4 tioned, these values are the only “tuning” parameters of the permutation complexity–entropy plane. However, the choice for the embedding dimen- represent a hypothetical image of size 3 × 3. The first step is to define slid- sions is not completely arbitrary, as the condition (dx dy )!  nx ny must hold ing submatrices of size dx by dy , where these values are called embedding to obtain a reliable estimate of the ordinal probability distribution P (46). In dimensions (the only parameters of the method). By choosing d = d = 2, x y our dataset, the average values of image length nx and height ny are both we have the following four partitions: close to 900 pixels (SI Appendix, Fig. S3), thus practically limiting our choice to dx = dy = 2. 6 0 0 2 4 5 5 2 A1 = , A2 = , A3 = , and A4 = . 4 5 5 2 6 7 7 4 Independence of H and C in Relation to Image Dimensions. The image files obtained from WikiArt do not have the same dimensions. SI Appendix, Fig. The next step is to investigate the ordinal patterns of occurrence of the S3 shows that image width and height have a similar distribution, with aver- elements of these submatrices. By letting a0, a1, a2, and a3 represent the age values equal to 895 pixels for width and 913 pixels for height. Also, elements of these matrices line by line, we have that A1 is described by 95% of the images have width between 313 and 2,491 pixels, and height the ordinal pattern Π1 = (1, 2, 3, 0), since a1 < a2 < a3 < a0, where Π1 rep- between 323 and 2,702 pixels. Because of these different dimensions, we resents the permutation that sorts the elements of A1 in ascending order. have tested whether the values of H(P) and C(P) display any bias as a result. Similarly, A3 is described by Π3 = (0, 1, 2, 3) (since a0 < a1 < a2 < a3), and A4 This is an important issue, since we expect the values of H(P) and C(P) to is described by Π4 = (1, 3, 0, 2) (since a1 < a3 < a0 < a2). In case of draws, reflect the characteristics of an image, not its dimensions. SI Appendix, Fig. we keep the occurrence order of the elements a , a , a , and a . Thus, A S4 shows scatter plots of the values of H(P) and C(P) versus the square root 0 1 2 3 2 √ is described by Π2 = (0, 1, 3, 2), since a0 < a1 ≤ a3 < a2 and because a1 pre- of image areas (that is, nx ny ) at several different scales, where any visual cedes a3. From all ordinal patterns associated with A for a given dx and relationship is observed. The Pearson linear correlation is very low (∼0.05) dy , we estimate the probability distribution of finding each permutation for both relationships. We also estimate the maximal information coeffi- from its relative frequency of occurrence. In our example, from all 24 possi- cient (MIC) (70), a nonparametric coefficient that measures the association ble permutations [that is, (dx dy )!], four have appeared just once, and thus between two variables, even if they are correlated in a nonlinear . their probabilities are 1/4, while all other permutations have zero probabil- The MIC value is very small (∼0.07) for both relationships. Thus, we con- ity. Therefore, the probability distribution of the ordinal patterns associated clude that the values of H(P) and C(P) are not affected or biased by image with A is P = {1/4, 1/4, 1/4, 1/4, 0, ... , 0}. We have omitted the specifica- dimensions. tion of each permutation in P because the order of its elements is irrelevant for the described procedure. Finding the Number of Clusters with the Silhouette Coefficient. The hier- Having the probability distribution P = {pi; i = 1, ... , n}, we calculate archical organization of artistic styles presented in Fig. 3B enables the the normalized Shannon entropy determination of clusters of styles. To do so, we must choose a threshold distance for which styles belong to different clusters. The number of clus- n ters naturally depends on this choice. A way of determining an optimal 1 X H(P) = pi ln(1/pi), [1] threshold distance is by calculating the silhouette coefficient (59). This coef- ln(n) i=0 ficient evaluates both the cohesion and the separation of data grouped into clusters. The silhouette coefficient is defined by the average value of where n = (dx dy )! is the number of possible permutations and ln(n) corresponds to the maximum value of the Shannon entropy S(P) = bi − ai Pn s = , [5] pi ln(1/pi), that is, when all permutations are equally likely to occur i i=0 max(ai, bi) (pi = 1/n). The value of H quantifies the degree of “disorder” in the occur- rence of the pixels of an image represented by the matrix A. We have H ≈ 1 where ai is the average intracluster distance and bi is the average nearest- if the pixels appear in random order, and H ≈ 0 if they always appear in the cluster distance for the ith datum. By definition, the silhouette coefficient same order. ranges from −1 to 1, and the higher its value the better the clustering con- Despite the value of H being a good measure of randomness, it cannot figuration. SI Appendix, Fig. S8 shows the silhouette coefficient as a function adequately capture the degree of structural complexity present in A (47). of the threshold distance used for determining the clusters in Fig. 3B. We Because of that, we further calculate the so-called statistical complexity (45, observe that the coefficient displays a maximum (of 0.57) when the thresh- 68, 69) old distance is 0.03. Thus, this threshold distance is the one that maximizes D(P, U)H(P) C(P) = , [2] the cohesion and separation among the artistic styles. By using this value, D* we find 14 different groups of artistic styles shown in Fig. 3. Also, a similar where D(P, U) is a relative entropic measure (the Jensen–Shannon diver- approach yields the 24 different clusters that are associated with the sim- ilarities among the Wikipedia pages of the styles reported in SI Appendix, gence) between P = {pi; i = 1, ... , n} and the uniform distribution U = Fig. S9. {ui = 1/n; i = 1, ... , n} defined as

 P + U  S(P) S(U) Implementation of Machine Learning Algorithms. All machine learning algo- D(P, U) = S − − , [3] 2 2 2 rithms used for predicting the artistic styles from an image are implemented by using functions of the Python scikit-learn library (71). For instance, the

+ / function sklearn.neighbors.KNeighborsClassifier implements the k-nearest P+U = { pi 1 n = ... } where 2 2 , i 1, , n and neighbors. In statistical learning, a classification task involves inferring the category of an object by using a set of explanatory variables or features   1 n + 1 associated with this object, and the knowledge of other observations (the D* = max D(P, U) = − ln(n + 1) + ln(n) − 2 ln(2n) , [4] P 2 n training set) in which the categories of the objects are known. For the results presented in the main text, we have included 20 different styles that is a normalization constant (obtained by calculating D(P, U) when just each have more than 1,500 images in our analysis (see SI Appendix, Fig. S5 one component of P is equal to 1 and all others are zero). The quan- for names). However, similar results are obtained by considering a larger tity D(P, U) is zero when all permutations are equally likely to happen, number of styles. For example, the overall accuracy of different learning and it is larger than zero if there are privileged permutations. Thus, C(P) algorithms is approximately 13% if we consider all of the styles with more is zero in both extremes of order (P = {pi = δ1,i; i = 1, ... , n}) and disor- than 100 images each (SI Appendix, Fig. S11) (compared with ∼18% if only der (P = {pi = 1/n; i = 1, ... , n}). However, C(P) is not a trivial function 20 styles are considered). of H(P). Namely, for a given H(P), there exists a range of possible values Thus, our classification task involves identifying the artistic style (the for C(P) (47, 48, 69), from which C(P) quantifies the existence of structural categories) of an image from its entropy H and complexity C (the set of complexity and provides additional information that is not carried by the features). To perform the classification, data are randomly partitioned into value of H(P). Mainly because of this fact, Rosso et al. (48) proposed to n equally sized samples that preserve the total fraction of occurrences in use a diagram of C(P) versus H(P), a representation space that is called the each category. One of the samples is used for validating the algorithm, complexity–entropy plane (45). and the remaining n − 1 samples are used for training the algorithm. The

8 of 10 | www.pnas.org/cgi/doi/10.1073/pnas.1800083115 Sigaki et al. Downloaded by guest on September 27, 2021 Downloaded by guest on September 27, 2021 0 alrR,Mclc P oa 19)Fatlaayi fPloksdi paintings. drip Pollock’s of analysis Fractal (1999) D Jonas art AP, the Micolich in RP, competition Taylor and 20. review (1933) Peer GD Birkhoff (2016) D 19. Helbing RL, Goldstone S, Balietti 18. 1 oe-mt ,Mtu 20)Fatlaayi:Rvstn olc’ rppaintings. drip Pollock’s Revisiting analysis: Fractal (2006) H Mathur K, Jones-Smith 21. Barab C, P, Deville D, Wang R, Sinatra 17. 0 uhsJ,Gaa J okoeD 21)Qatfiaino ritcsyethrough style artistic of Quantification (2010) DN investigations. Rockmore scientific DJ, Recent Graham JM, nature: Hughes the 30. and Mondrian Pollock, (2004) space. R color in Taylor fractality and 29. view Jackson stochastic The of Lisa: Mona analysis (2008) GR Fractal Jafari P, (2016) Pedram E 28. Rodriguez C, Ibarra-Valdez J, Alvarez-Ramirez 27. analysis. fractal and paintings . Drip (2009) fractal LM using Krauss H, paintings drip Mathur Pollock K, Pollock’s Jones-Smith Authenticating Revisiting 24. (2007) analysis: al. Fractal et (2006) RP, D Taylor Jonas 23. AP, Micolich RP, Taylor 22. Barab D, Wang M, Szell P, Deville R, Sinatra reveal 16. networks in patterns Inheritance of (2014) patterns D Quantitative Helbing (2012) M, Perc DN T, Rockmore Kuhn DC, 14. Krakauer NJ, Foti JM, Hughes 13. communication mobile in Saram strengths 12. tie and Structure (2007) al. et JP, Onnela 10. 1 hmrL(02 optraayi eel iiaiisbtenteatsi tlsof styles artistic the between similarities reveals analysis (2012) L Shamir 31. complexity. and forms Artistic (2011) RP Taylor J, Casti abstract JP, in Boon transitions Order-fractal (2016) 26. J Calleja la De F, Cervantes EM, Calleja la De 25. physics. of century the across progress of Self-organization (2013) M Perc 15. dynamics. communication human in patterns Calling (2013) al. et ZQ, Jiang 11. h it fterda ai ucinkre n h eat aaee for with parameter penalty associated the parameter and the kernel function the method, basis of forest radial the case random of the the trees width of the of parameters in depth case neighbors The maximum the study. the of and our in number number the in the algorithm, use are neighbors we nearest studied that have the methods we in learning that noise the four learning modeling the the for for of even enough complexity (overfitting). high the set is, when training that high, happen too usually is hand, model the errors, other Variance about (underfitting). the information data are relevant on the methods the describe learning of that all the variables account explanatory when into occur between taking errors -off properly Bias best not validation errors. the (the variance estimating parameters for and main practice bias their common is of This function curves). a as algorithm their learning and the as scores known the is of approach values (63). This strategy average cross-validation calculated. the are repeated which intervals is from confidence process validation ensemble This the from an score. obtained ing cross-validation one the the the from and is score, obtained set training identifications) the correct is of set fraction training the is, (that accuracy iaie al. et Sigaki .DvleP ta.(06 cln dniycnet ua oiiyadsocial and mobility human connects identity Scaling (2016) human al. individual et Understanding P, (2008) AL Deville Barabasi 9. CA, history. Hidalgo cultural of MC, framework network Gonzalez A (2014) bias. al. 8. positivity et M, universal Schich a digitized reveals 7. of Human millions (2015) using al. culture et PS, of Dodds analysis Quantitative 6. (2011) al. et JB, Michel 5. (1971) cooperation. human HE of physics Statistical Stanley (2017) vaccination. al. 4. of et physics M, Statistical Perc (2016) al. 3. et Z, Wang (1999) 2. HE Stanley RN, Mantegna 1. eetmt h riigadtecosvldto crsfrec machine each for scores cross-validation the and training the estimate We Nature game. impact. scientific individual Nature Phys precdn nlssi h rwnso itrbuglteelder. the bruegel pieter of USA the in analysis coding sparse Lett Complexity Chaos C Phys Mod J Int evolution. painting Pollock’s Sci Life E Rev Lett Recognit Pattern (reply). paintings 3:1720. memes. scientific literature. of evolution the in influence stylistic USA Sci Acad Natl Proc USA Sci Acad networks. interactions. patterns. mobility USA Sci Acad books. Oxford). Finance in Complexity a ohadPollock. and Gogh Van paintings. 107:1279–1283. 11:791–796. 79:046111. k ,e l 21)Pritneo oilsgaue nhmncommunication. human in signatures social of Persistence (2014) al. et J, aki ¨ Science 399:422. 444:E9–E10. 15:265. rcNt cdSiUSA Sci Acad Natl Proc n Phys Ann rcNt cdSiUSA Sci Acad Natl Proc 110:1600–1605. 112:2389–2394. 331:176–182. 19:855–866. Nature rcNt cdSiUSA Sci Acad Natl Proc hsRvX Rev Phys Nature etei Measure Aesthetic 371:313–322. hs rniin n rtclPhenomena Critical and Transitions Phase 28:695–702. CmrdeUi rs,Cmrde UK). Cambridge, Press, Univ (Cambridge 1:29. 111:942–947. Leonardo 444:E10–E11. 453:779–782. IAppendix SI Science ho oiosFractals Solitons Chaos 4:041036. 45:149–154. 104:7332–7336. 354:aaf5239. nrdcint cnpyis orltosand Correlations Econophysics: to Introduction 113:7047–7052. HradUi rs,Cmrde A,Vl38. Vol MA), Cambridge, Press, Univ (Harvard i.S10 Fig. , 113:8414–8419. s L(06 uniyn h vlto of evolution the Quantifying (2016) AL asi ´ s L(05 etr fphysics. of century A (2015) AL asi ´ rcNt cdSiUSA Sci Acad Natl Proc 83:97–104. hw h aiaincurves validation the shows hsRep Phys hsRep Phys olna Dyn-Psychol Nonlinear 664:1–113. Science n CaednPress, (Clarendon rcNt cdSci Acad Natl Proc ie,produc- times, 109:7682–7686. 687:1–51. 345:558–562. rcNatl Proc rcNatl Proc c Rep Sci n-fold Phys Nat k - 8 adJ r(93 irrhclgopn ootmz nojciefunction. objective an optimize to grouping Hierarchical (1963) Jr JH Ward 58. arts. painting of analysis quantitative Large-scale (2014) H Jeong SW, Son D, Kim 39. (2011) eds A, Bentkowska-Kafel J, Coddington DG, Stork 38. 7 aa ,G M, art. Nadal of 57. theories Evolutionary (2005) B Boyd 56. (1984) ES Blatt SJ, Blatt 55. (2013) AN Hodge 54. (2013) FS Kleiner 53. (1997) L Goehr style. AC, pictorial of Danto analysis The 52. (2002) J Gaiger 51. (2004) A Riegl 50. Distinguishing (2007) MA Fuentes A, W Plastino 49. MT, multiscale Martin the HA, Larrondo with OA, textures Rosso image 48. Discriminating (2016) HV Ribeiro L, Complexity-entropy Zunino (2012) RS 47. Mendes PA, Santoro EK, Lenzi complexity. L, of Zunino measure HV, statistical Ribeiro A (1995) 46. X Calbet user- HL, Mancini of R, Lopez-Ruiz development time for measure 45. the complexity natural A Quantifying entropy: Permutation (2002) (2017) B Pompe L C, Bandt Manovich 44. J, Chow history. art M, digital large and Yazdani science of Data 43. (2015) analytics L Cultural Manovich (2012) 42. J Douglass T, Margolis of emergence L, historic Manovich the Understanding D, (2017) Ushizima J Park 41. S, Sun H, Jeong D, Kim B, Lee 40. (2010) eds A, Bentkowska-Kafel J, Coddington DG, Stork (2008) 37. eds J, Coddington DG, colors Stork of Statistics 36. (2016) SMC Nascimento M, Vilarigues JMM, Linhares C, Montagner 35. 1/f (2010) C Redies J, Denzler M, Koch 34. in complexity quantify to Castrej used JR, be Castrejon-Pita can invariants 33. Topological (2017) R Zenit M, Elsa 32. a nrdc os otemdl h eut rsne in presented results data The unnecessary model. adding the S10 while when Fig. to model, common noise the also introduce fitting is may for are practice sets enough training This small not set. very usually since training algorithms, cross-validation learning the the statistical of with and dealing size training the the of on dependence scores the is, that curves, ing are values tun- These best obtained. 4C the Fig. are applying in which method for from learning guide in (71), each a reported algorithm of as search parameters results grid these ing exhaustive use more We model. a so-called the network and neural classification, the machine vector support the n lvna eerhAec rnsJ-09adP5-0027. and J1-7009 Grants Agency Research Slovenian and 9 N de Pessoal de mento Cient Desenvolvimento de ACKNOWLEDGMENTS. the if significant data. not the of practically 50% is exceeds growth set training this However, algorithms. all for nadto otevldto uvs ehv loetmtdtelearn- the estimated also have we curves, validation the to addition In d ii PPL, Tinio Assoc eds Arts, 167–194. pp the UK), and Cambridge, Press, Univ of (Cambridge JK Psychology Smith the of Handbook Cambridge Rep 7869. II Art of Analysis 7531. Art of Analysis isnD Nrhetr nvPes vntn L,p 147–176. pp IL), Evanston, Press, Univ (Northwestern DS Wilson Representation London). II. Vol Ed, 14th ), (Wadsworth, History Art Later in chaos. from noise plane. 679–688. causality complexity-entropy patterns. two-dimensional two-dimensional for measure complexity a 7:e40689. as plane causality A Lett series. 2001–2010. during art generated Media Social and Weblogs flickr. from datasets arXiv:1701.07164. contrast. color via painting in diversity 6810. Vol WA), Bellingham, Photonics, Opt Soc (Int SPIE of Proceedings scenes. natural of and paintings categories in different and , comics, , art, . visual of spectra power enigma. an in wrapped mystery 13:836–838. A lines: Nasca paintings. abstract lflnH(1950) H olfflin ¨ 4:7370. 58:236–244. hwta h rs-aiainsoeicesswt h riigsize training the with increases score cross-validation the that show 209:321–326. hsRvLett Rev Phys . PictnUi rs,Pictn,Vl 197. Vol. Princeton), Press, Univ (Princeton IAppendix SI mzPet 21)Eouinr prahst r n aesthetics. and art to approaches Evolutionary (2014) G omez-Puerto ´ Dvr iel,NY). Mineola, (Dover, LSOne PLoS itrclGamro h iulArts Visual the of Historical rceig fSI ItScOtPoois elnhm A,Vol WA), Bellingham, Photonics, Opt Soc (Int SPIE of Proceedings , Rulde e York). New (Routledge, rceig fSI ItScOtPoois elnhm A,Vol WA), Bellingham, Photonics, Opt Soc (Int SPIE of Proceedings , itr fAt anigfo itot h rsn Day Present the to from Painting Art: of History A rnilso r itr:TePolmo h eeomn fStyle of Development the of Problem The History: Art of Principles hsRvLett Rev Phys nw-ae Syst Knowl-Based ade’ r hog h gs h etr Perspective Western The Ages: the through Art Gardner’s 88:174102. h rceig fteSxhItrainlAA ofrneon Conference AAAI International Sixth the of Proceedings The vlSpro rns405/043ad303642/2014- and 440650/2014-3 Grants Superior ´ ıvel otniyadCag nAt h eeomn fMdsof Modes of Development The Art: in Change and Continuity 5:e12268. hsrsac a upre yCneh Nacional Conselho by supported was research This i.S10 Fig. , nPt A Sarmiento-Gal AA, on-Pita ´ AA rs,Pl lo A,p 30–34. pp CA), Alto, Palo Press, (AAAI fe h n fAt otmoayAtadtePl of Pale the and Art : of End the After fioeTecnol e ´ ıfico 99:154102. OAA JOSA LSOne PLoS n sdfrotiigterslsshown results the obtaining for used and 126:48–55. optrIaeAayi nteSuyo Art, of Study the in Analysis Image Computer 33:A170–A177. 2 hrceitc n stoyi h fourier the in isotropy and characteristics gc,Coordenac¸ogico, ´ 12:e0175350. rJAesthetics J Br h ieayAnimal Literary The ho nedsi olna Sci Nonlinear J Interdiscip Chaos NSLts Articles Latest PNAS nA Castrej A, an Zn ok,NwYork). New Books, (Zone ´ n iia r Hist Art Digital J Int ho oiosFractals Solitons Chaos optrVso n Image and Vision Computer optrVso n Image and Vision Computer 42:20–36. od Aperfeic¸oa- de ao ˜ nGrı (2003) R on-Garcıa ´ d otcalJ, Gottschall eds , eat for penalty L2 IAppendix SI | (Arcturus, 1:21631. mStat Am J lSOne PloS f10 of 9 Phys The 91: Sci ,

APPLIED PHYSICAL SCIENCES 59. Rousseeuw PJ (1987) Silhouettes: A graphical aid to the interpretation and validation 65. Zujovic J, Gandy L, Friedman S, Pardo B, Pappas TN (2009) Classifying paintings by of cluster analysis. J Comput Appl Math 20:53–65. artistic : An analysis of features & classifiers. IEEE Workshop on 60. Swartz A (2007) Pattern and Decoration: An Vision in American Art, 1975–1985 Signal Processing, 2009 (Inst Electr Electron Eng, ), pp 1–5. (Hudson River , New York). 66. Agarwal S, Karnick H, Pant N, Patel U (2015) Genre and style based painting classifica- 61. Chowdhury GG (2010) Introduction to Modern Information Retrieval (Facet, tion. Availabe at https://ieeexplore.ieee.org/document/7045938/. Accessed August 1, London). 2016. 62. Rosenberg A, Hirschberg J (2007) V-measure: A conditional entropy-based external 67. van der Walt S, et al. (2014) scikit-image: Image processing in Python. PeerJ 2:e453. cluster evaluation measure. Proceedings of the 2007 Joint Conference on Empiri- 68. Lamberti PW, Martin MT, Plastino A, Rosso OA (2004) Intensive entropic non-triviality cal Methods in Natural Language Processing and Computational Natural Language measure. Physica A Stat Mech its Appl 334:119–131. Learning (Assoc Comput , Stroudsburg, PA), Vol 7, pp 410–420. 69. Martin M, Plastino A, Rosso OA (2006) Generalized statistical complexity measures: 63. Hastie T, Tibshirani R, Friedman J (2013) The Elements of Statistical Learning: Geometrical and analytical properties. Physica A 369:439–462. Data Mining, Inference, and Prediction, Springer Series in Statistics (Springer, 70. Reshef DN, et al. (2011) Detecting novel associations in large data sets. science New York). 334:1518–1524. 64. Muller¨ A, Guido S (2016) Introduction to Machine Learning with Python: A Guide for 71. Pedregosa F, et al. (2011) Scikit-learn: Machine learning in Python. J Machine Learn Data Scientists (O’Reilly Media, Sebastopol, CA). Res 12:2825–2830.

10 of 10 | www.pnas.org/cgi/doi/10.1073/pnas.1800083115 Sigaki et al. Downloaded by guest on September 27, 2021