SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
S A N K H A R A
sound sculpture
1 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
2 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
Written part and contextual research documentation for the diploma artwork – the sound sculpture titled:
“SANKHARA – Sonic Formations”
by Nikita Zhukovskiy
im Diplomstudium Medienkunst / Studienzweig DIGITALE KUNST an der Universität für Angewandte Kunst Wien im Jänner 2019
Angestrebter akademischer Grad: Mag.ª art. (Magistra artium)
Betreuerin: Univ.-Prof.Mag.ª art. Ruth Schnell
Mitbetreuung: DI Arch. Nicolaj Kirisits, Mag. art. Martin Kusch, Ph.D. Patricia J. Reis
3 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
4 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
TABLE OF CONTENT
INTRODUCTION 7 DESCRIPTION OF THE WORK 9
THEORY AND METHOD DEFINITION OF SOUND SCULPTURE 11 LANGUAGE AS SOURCE – LANGUAGE AS INSTRUMENT 13 GRAPHEME AND ABSTRACT STROKE 19 ENCODING ABSTRACT 21 SHAPES: INFLUENCES AND ORIGINS OF “FORMATIONS” AESTHETIC 23 ENCODING NOISE 27 THE UNPRONOUNCEABLE ABSTRACT – UNPRONOUNCEABLE LETTERS 31 CONNOTATIONS OF PHONEMES – MEANINGS OF NOISE 33 SOUND & SHAPE, GESTURE & GRAPHEME – GENESIS OF ELEMENTS 33 SYNESTHESIA 36 AGAINST GENERATIVE 40 PATTERN RECOGNITION VS HIGHER STATE OF CONSCIOUSNESS 41 FREE GRID SOUND SYNTHESIS 43 PHYSICALITY OF SOUND 47
CONCLUSION 48 DECLARATION OF AUTHORSHIP 49 LIST OF REFERENCES 50 TABLE OF IMAGES 53 ADDITIONAL RESEARCH MATERIALS 54 ______
If not stated otherwise images are the property of Nikita Zhukovskiy
5 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
6 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
INTRODUCTION
This essay is a contextual research text written as a part of the diploma artwork titled „Sankhara - Sonic Formations”.
The following text provides a formal and technical description of the work.
It introduces the artistic method and the role of linguistic and calligraphy studies and stresses the neurological phenomena of synesthesia as the key inspiration inducing the creative process. It outlines the influence of Futurism as a native concept for digital arts, which guides the creative process aesthetically justifies deconstruction as a relevant method.
Furthermore, the essay articulates the challenges and possibilities of inter- medial transformations and subjective versus algorithmic approach.
In conclusion, it provides a reflection on artistic strategy and choices, outlining the significance and the value of an auditory experience in presence of the work and subjectivity of any interpretation.
7 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
8 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
DESCRIPTION OF THE WORK
„Sankhara - Sonic Formations" is a metallic sound sculpture of human scale.
It is a development of the series of works made of irregular asymmetrically cut metal plates, which has numerous connection slots embedded into them, allowing the work to be assembled, dissembled and re-assembled in multiple combinations. The final outcome size is variable, with one of the combinations to be approx. 1450 x 1750 x 2200 mm.
Connected to a digital sound setup, the assembled structure radiates a resonant ambient music piece, written specifically on and for the sculpture installation, using an experimental customized sound synthesis setup.
The experimental use of transduction technology with a modified transduction device allows the sound wave signal to enter and flow through the material of the sculpture in a form of a micro vibration. That alters the soundscape. Transformed and amplified by the resonant body of work the sculpture fills up the installation space with sound of unusual physical quality bringing certain elements of work into motion.
9 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
François and Bernard Bashet “French Monument Born on 57th St.” Sound sculpture, steel, aluminium, 1955 1
1 Grayson, John “Sound Sculpture”, A.R.C. The Aesthetic Research Centre of Canada, 1975.
10 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
THEORY AND METHOD
DEFINITION OF SOUND SCULPTURE
This method exercises the approach of creating a sculptural artwork, that is in addition to variability and mobility due to its modular organization, addresses the medium of sound and its physicality, as one of the dimensions of an expanded field [1].
According to the 1979 essay ‘Sculpture in the Expanded Field' by Rosalind Krauss:
"The logic of sculpture, it would seem, is inseparable from the logic of the monument. By virtue of this logic, a sculpture is a
commemorative representation. It sits in a particular place and speaks in a symbolical tongue about the meaning or use of that place” [1].
A sound sculpture differs from a sound installation, as installation manifests itself through elements or fragments, as a kind of 3-dimensional collage of objects and sounds in space of the exhibition, while a piece of sound art may exist as a record, and may or may not necessarily have a physical body.
11 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
A sound sculpture, however, is an integral body of work composed of shape and sound that manifest itself in the exhibition space as a whole [2].
In the first-ever publication on the notion of Sound Sculpture, an openly gay American composer and artist Lou Harrison defines it as "the integration of visual form and beauty with magical, musical sounds through participatory experience [3]"; and calls for the act of learning through action by writing:
"Making an instrument is one music's greatest joy. Indeed, to make
an instrument is in some strong sense to summon the future […] Nor all the instruments invented and over, so to speak. The world is rich
with models – but innumerable forms, tones, and powers await their summons from the mind and hand. Make an instrument – you will
learn more in this way than you can imagine.”[3].
Formation series exercises the approach of creating a sculpture-instrument- body of work. It's an embodied extension of a subjective synasthetic experience of perceiving sound as shape, and at the same time, vice-versa – an instrument to exercise improvisations inspired and defined by shapes, as well as by the gesture of writing in relation to the tradition of futurists, fluxus and graphical notations.
12 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
In the introductory essay to the 2012 Sound Art exhibition at ZKM, Peter
Weibel declares:
"From the mid-1960s, media artists and conceptual artists were in the vanguard when it came to establishing a new foundation for sound
art and pushing its development. […] All objects that make noise can become sound instruments. The sound is liberated from the prison of
music. All kinds of sounds can be art. For the first time, the musical alphabet is coextensive with the universe.” [4]
LANGUAGE AS SOURCE – LANGUAGE AS INSTRUMENT
"Die Grenzen meiner Sprache bedeuten die Grenzen meiner Welt." "The limits of my language are the limits of my world." - Ludwig Wittgenstein [5]
The name of the artwork “Sankhra - Sonic Formations" refers to ‘Sanhkara’
–– the buddhist concept of mental formations which are understood as the subjective form-creating faculty of mind. [6]
The word itself is native to Pali language and it may be translated in English as ‘Formations. It literally means: ‘that which had been put together’. This implies that everything we perceive as the ultimate reality is, in fact, an illusion created by overlaying. [7]
13 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY
’Xing’ as a typeface. 2
‘Xing’ handwritten in different calligraphy styles. 3
2 ‘Xing’ character as a digital typeface. 3 Examplse of Calligraphy writing of the Combinations of our own state, our consciousness and physical body, our experience, and environmental conditions — everything is dependent on pre-set, on our knowledge and conditioned mind. The word ‘Sankhara’ could be written as a symbol, which is identical both in Chinese and Japanese Kanji, in 6 strokes as . In Japanese it simply means ‘to go' or ‘moving/traveling' or just a ‘line’. [8] In relation to the “Formations” creative method, Kanji symbols are significant, as it is a system of writing that consists of staking and overlaying strokes. A way to create a sign which has a potential to condense a complex meaning that is at times abstract. For example in the case of ‘Sankhara’ Kanji symbol – 6 strokes describe a complex and abstract philosophical concept, which has no direct materiality and no physical body, that could be reproduced in a pictorial way. As all Kanji characters in Japanese, it is appropriated from old Chinese and has a much more complex meaning as a radical 144 sign ‘Xing’, which is explained by the “Analysis and Research on the Concept of Three Connotations” as a notion of impermanence and migration, or in broader meanings — all kinds of impermanence [9]. 15 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Digitalized Radical sign #144 ‘Xing' written in the ‘Oracle’ script – an earliest form of Chinese writing a kind of an ancient protolanguage 4 4 ShuoWenJieZi 說⽂解字 The earliest complete 987 copy by XuXuan 徐鉉. Collection of Richard Sears 16 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY On the relationship between geometry and language Bertrand Russel, in his introduction to the side-by-side-by-side edition of Wittgenstein "Logisch- philosophische Abhandlung", containing a comparison of two different version of English translations of philosophical text next to the German original, writes that: “He compares a linguistic expression to projection in geometry. A geometrical figure may be projected in many ways: each of these ways corresponds to a different language.”[5]. Visual stimulation draft#5 - digital render sketch. 17 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Typeface - overlays - graphemes - overlays - shapes. 18 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY GRAPHEME AND ABSTRACT STROKE With the intention to create a purely abstract work various manipulations with language and system of writing are used as a method. The outcome does not have any figurative image, even subconsciously encoded into it yet remains open to interpretation. Particularly – a letter undergoes the process of deconstruction through a so- called aesthetical distillation of signs, down to lines constituting one letter. Furthermore translating a digital output of a printed letter, inherited from analog typefaces tools, back into its analogue ancestor – a handwritten letter. Focusing on the smallest unit of any writing system – a grapheme [10], its intensity, force and the spatial gesture behind it. In order to deconstruct it all further down to a simple mark making gesture – an abstract stroke. This notion of a stroke is significant, as it is a crucial element of the authenticity of any artistic work and it is also related to the signature which is a proof of authorship. In the Russian language, the same verb ‘to write' describes a process of writing a text and the process of creating a painting. The notion of artistic creation is highly related to the nature of a line as a stroke, and to the nature of language as a tool of creating meaning. 19 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Sciene and Klor of 123KLAN “TEENAGE KICKS RENNES FRANCE”, An illegal wall piece stating the graffiti writers artist names on a public wall, 2013 5 5 Sciene and Klor, https://www.123klan.com/blogs/news/93981569-teenage-kicks-rennes-france accessed on 24.12.18 20 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY ENCODING ABSTRACT I am interested in the process of rendering any symbol, including the author’s signature, abstract. For this matter I focus on graffiti aesthetics as one of the exciting and living form of contemporary calligraphy. It is one of the crucial touchstones of inspiration for the process of transforming the graphemes of letters and its fragments into abstract shapes under the influence of sound. The phenomena of New York ‘Wild Styles’ emerged in the 80’s as a celebration of authenticity of handwriting, that incorporates the notion of transforming any letter or symbol into abstract shapes. Rendering the message encoded and unreadable to one audience yet including the other key audience that is able to decode it [11], ‘Wild Styles’ aesthetics is distributed through photographs of illegal pieces and sketches via underground networks of artists. Internet grows and it spreads online, induced by the means of digital conversion it evolves from a local phenomena to a global one. Political aspects of graffiti and use of abstraction to execute an inclusion-exclusion mechanism are highly important, as well as the fact that the graffiti pieces were a kind of aesthetical extensions, that symbolised complex underground cultural processes, which were taking place in urban spaces marked by the writers. Juxtaposed against standardised state authorised labelling and lettering of urban spaces – indeed the notion of handwriting authenticity is most significant. [11] 21 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Davide Quayola “PTA” multi-ch HD projection, 2-ch sound, 40:00 fragments of a live Audio-Video Performance from the ‘FORM-SOUND-ABSTRACTION’ series inspired by futurist and synesthesia. A neo-classic example of digital constructivism. 2007 6 “Sankhara – sonic formations” initial stage maquette variations. 6 https://www.quayola.com/pta/ accessed on 21.12.2018 22 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY SHAPES: INFLUENCES AND ORIGINS OF “FORMATIONS” AESTHETIC At the end of the 20th century and early 00s, the recognition of graffiti aesthetics in general and Wild Styles in particular, reaches such a level that it starts to influence artists that are working primarily with digital media. At the same time, digital artists such as Quayola and Alex Rutterford are noticing certain cultural parallels with the utopian spirit and philosophy of revolutionary art movements of the beginning of 20th Century. The implementation of creative methods of Futurists and Constructivists artists, and practices of ‘De Stijl’ movement, results in the emergence of ‘Digital Constructivism' aesthetics. Which in its own way was fused by artists such as David Quayola, mentioned above, and the winner of the Prix Ars Electronica 2001 – Joshua Davis and members of ‘123Klan’. The ‘Wild Styles’ calligraphy aesthetics, which was always influenced by urban spaces and city scapes, with its industrial noises of factories and traffic as a background soundtrack, undergoes the process of cultural digitalization. A complex cultural shift driven aesthetically by the change in music production methods and composition styles. Many of the artist of the scene of that time were contributing their works to to electronic musicians, creating videos and graphic for EPs and LPs record covers, focusing on experimental sound aesthetics and synthetic soundscapes. It is not a coincidence that all of the instruments that were used by those electronic musicians were originating in Japan. 23 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Scien “Winter warm up!”, pen sketch on paper, 2013 7 Joshua Davis “Pray Station” internet-artwork - online sketchbook, this fragment – “Graffiti Architecture”, screenshot of the entry from 30 of August 2001, generative flash code. 8 7 Scien photo archive - KingScien @ Flickr - https://www.flickr.com/photos/kingscien/ 8 Joshua Davis Archive - praystation.com 24 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY ’AKAI’, ‘YAMAHA’, ‘ROLAND’ Japanese music instruments of late 90’s early 00’s. 9 Various visual styles’ aesthetic transformations are taking place, as a result of a feedback loop between digital and analog. What kind of aesthetics emerges as an outcome of conversion loops? What kind of artistic outcome results become possible in the sense of transferring the aesthetical input between analogue and digital, on the scale of one particular work, but also in the sense of a cultural process as a whole, on a more complex level, on larger scale and broader continuity? Analogue undergoes digitalization, conversion transforms the content, and nature of a physical medium that carries out a digital projection of the content back into an analogue manifestation of the body of work transforms it again. 9 http://www.akaipro.com/, https://www.roland.com, http://yamaha.com/ accessed on 21.12.2018 25 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Al Gordon, Jerry Ordway, John Workman “Wildstar”, 1993 Wildstar synthesizes a beam of energy producing the sound ‘TZZZZAT!’ 10 Katsuhiro Otomo “GIIIIIiiii”, “Shrak”, “kaSLOOSH”, Sound images from the ‘Akira’ comic book, 1988 11 10 “Wildstar: Sky Zero”, #1 (Mar 1993, Image), Comic Book fragments - private collection scans. 11 Katsuhiro Otomo, ‘Akira’, Titan Books 1982 - 1990 - private collection scans. 26 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY ENCODING NOISE I am highly interested in the notion of a stroke and its relation to sound and phonetics. In deconstructing phonetics in relation to music, sound, and noise. In liberation of noise and its artificial phonetics. Wildstar launches timemachine producing the sound ‘NNNNEEEEE!!!!’12 I am especially interested in how a simple gesture may signify a certain sound effect and how this can be translated into visual aesthetics. For example, to imitate a melody one may produce phonetical sounds which serve as abstract dummies of absent words, replacing forgotten, or at times non-existent lyrics of a song. However, if the sound or noise has nothing organic to it, yet has industrial genesis and properties, or, which is even more interesting, is purely a product of pure electric synthesis – what kind of phonetics may be attributed to it? Which letters? What kind of symbols or strokes? There are many examples of such translations of sounds in strokes in various comic books classics. Noises alien to phonetics are communicated through the letters that are signifying sounds not only by the choice and the sequence of letters but by visual design of their shapes as well. 12 “Wildstar: Sky Zero”, #1 (Mar 1993, Image), Comic Book fragments - private collection scans. 27 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Example of graphical notation music scores from the collection of John Cage, 1969 13 13 Cage, John “Notations”, Something Else Press, Inc., 1969 28 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Yet the most inspiring examples are found in works of futurists of the early 20th century in graphic music scores, which introduced Idiosyncratic notation systems, which I’m going to come back to later on describing the process of sound sculpture composition. A new way to subjectively record music scores to provide performers with visual guide for improvisation, where in addition to tones, chords and temporal intervals, more information such as intensity velocity and expression of gesture of sound is encoded as well [12]. Udo Kasemets “Timepiece for a Solo Performer”, 1964, One page, 21.5cm X 28 cm Pencil on graph paper Two pages instructions, Two sketches of piece.14 14 Cage, John “Notations”, Something Else Press, Inc., 1969 29 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY 30 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY THE UNPRONOUNCEABLE ABSTRACT – UNPRONOUNCEABLE LETTERS Exploring the relationships between gesture and sound, for this project I am focusing on the sounds that in the German language are described with 3 letters, namely ‘SCH’. But it is written down with only one letter in the Russian language, which is most often ‘Ш’ [SH], yet may also be a more intense version of it - ‘Щ’ [SHCH]. However a sharp equivalent of both letters above is ‘Ж’ [ZH]. That is also the first letter of my family name and the main starting grapheme of this very “Sankhara - Sonic Formation” sound sculpture. Therefore, within this methods’ framework, I focus on my signature, in order to render the letters and the very notion of an artist signature, abstract. Only this time I deconstruct it, therefore symbolising the loss of ego and freeing oneself. The shapes are created in the process of handwriting deconstruction. Each handwriting is different and it communicates the relationship between the subjectivity of expression, the process of adaptation to be understood, and the ability to read into what may appear to be abstract. Art curator Hans Ulrich Obrist in relation to his years’ long project of collecting handwritten notes says that: "It's the opposite of when, through globalization, differences can disappear and there is homogenization. Handwriting is the opposite because two people never have the same handwriting. It is an incredible celebration of difference [13!].” 31 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Grapheme example – Gesture#1 Voronoi diagram interactve code with a grapheme curve input. 15 15 http://blog.ivank.net/voronoi-diagram-in-javascript.html accessed on 17.11.2018 32 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY CONNOTATIONS OF PHONEMES – MEANINGS OF NOISE The sound Ш [SH] itself may attribute a meaning such as – to stay quiet, in an imperative form. It can also mean a slow movement or wind. Щ [SHCH] is an aggressive sound, it can mean the same request becoming intense, and it can also symbolize a sound of a snake. Yet the sharp version of both – Ж [ZH] may communicate a sound of work or the noise of a working machine. SOUND & SHAPE, GESTURE & GRAPHEME – GENESIS OF ELEMENTS Below is the elaboration of the process of deconstruction of the grapheme and the process of shaping the elements of the sculpture in relation to the transformation of the graphemes into sound. The drawing on the left shows that the grapheme of a letter Ш consists of 3 gestures. I start with the gesture #1 and deconstruct it into micro-gestures that are define the shape of one element, meaning that the velocity of a gesture and the intensity of pressure are translated into the shape of the element. The element is then altered further on inspired by sound aesthetics and subjective synesthetic experiences of frozen explosions of sound. On its initial stages, this process is inspired, by the practices of futurist artists [14]. 33 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Elements cutting paths layout, draft #2. 34 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Simplifying lines and focussing on a subjective perception of geometry and shape of sound. Emphasizing that the state of synesthesia is at the core of the process Advanced stages of the sculpture development addresses various feedback loop relationships between shape and sound. The elements’ geometry is inspired by a subjective synesthetic experience. Then the shapes are distorted and altered further on, in order to make the elements more resonant to a broader frequency range in response to customized sound input. My artistic research experiments with a smaller scale work of the same material proved that different shapes are responsive to certain frequencies more than others: Low frequencies are wide and round as the sound wave has a wider period and it needs more surface to resonate. High frequencies become narrow, long and sharp, as they need extensions to charge into the space and resonate, acting as a kind of tuning forks. 35 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Therefore, the grapheme is like an invisible bone of the shape, while its flesh is expanding and filling up with vibrating frequencies of sound. Additionally, the Voronoi diagram is used to generate more shapes by taking the vector and the grapheme curve points data as an input. Using in addition to that the image to sound sonification method. The drawing is then digitalized and connection slots are placed respectively to the weight and the thickness of the material. The final composition is defined by overlaying, gravity and material resistance, sound quality and resonance. SYNESTHESIA Speaking of the state of synesthesia as perception of sound as a shape I would like to stress, that being a subjective experience does not make it less real. Just one of many evidences of this subjective reality is the research of Danko Nikolic at the Max Planck Institute for Brain Research in Frankfurt, Germany. Subjects with grapheme-to-color synesthesia condition were tested in comparison with a control group with a variety of color eliciting letters and numbers with congruent, incongruent and opponent incongruent groups — the same as its common association, different but not completely opposite of the association, or on the opposite end of the spectrum from the associated color. Temporal aspect was measured during the experiment to 36 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY find out how long it took the participants to name the color of the grapheme. The result suggest that synesthetic colors are perceived in a realistic way, just as synesthetes report [15]. In visual arts the phenomena of synesthesia is known to be an initial moment of inspiration through sound. In opposition to algorithmic approach to visualise music and sound, synesthets are approaching the process of visualising sound subjectively – as a kind of reconstruction of an imaginary visual experiences. One might also say, according to a dream or a hallucination, induced by an auditory experience of experimental synthetic music. In relation to that I would like to quote Alex Rutterford, a British digital artist and film director who created the ‘Gantz Graf' video and several music record cover graphics for Autechre. He also speaks about subjective visions of 3-dimensional objects and shapes embodying sound – a genuine synesthetic experience: "The inspiration of almost all my work is based around music or sound, and so most of the visual inspiration that I draw. Normally I have a fantastically accurate view in my head of precise edit points and exactly what I want the thing to look like. But then it's an ongoing struggle, a nightmare to get that image out of your head. [16]" 37 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY Alex Rutterford “Gantz Graph”, music video screenshots, music record cover and inner label Art for the Autechre CD single, Warp Records, 2002 16 16 http://alexrutterford.com/graphic/autechre-gantz_graf/ 38 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY An experience that I highly relate to, as the same kind of inspiration – something that might be called a ‘frozen explosions of sound’, which may be understood as a kind of a metaphor for a cognitive sensory phenomena, that on a neurological level works as a kind of explosions of electrical impulses in neurones in response to stimuli [18,19,20], that induces a mutating feedback loop dynamic, which is taking place during the aesthetical exchange between the medium of sound and shape. This is what fuels the creative method of the ‘Formations’ sound sculptures series. The article written by Siri Carpenter for American Psychological Association describes synesthesia in the following words: "For scientists, synesthesia presents an intriguing problem. Studies have confirmed that the phenomenon is biological, automatic and apparently unlearned, distinct from both hallucination and metaphor. … The most common form of synesthesia, researchers believe, is colored hearing: sounds, music or voices seen as colors. Most synesthetets report that they see such sounds internally, in "the mind's eye." Only a minority […] see visions as if projected outside the body, usually within arm's reach.[17]” Therefore, during the “Sonic Formations” creative process I transform graphemes into an abstract shapes inspiring mutations with live sound performance, and relying on a subjective image in the mind’s eye. 39 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY AGAINST GENERATIVE – SUBJECTIVITY VS AUTOMATISATION OF ARTISTIC CHOICES As an argument against algorithmic approach to visualisation of sound I would like to bring up the example of the ‘Max for Live' sound visualisation patch named ‘Ganz Graf’ in reference to the video described above – ‘Gantz_Graf' that does feature a customisable dynamic sound reactive 3- dimensional structure as an output – yet stands far aside from the original video artwork that inspired its name. Here is how Alex Rutterford puts it: "Everyone says ‘how long did it take you?' How did you do it, they always want to ask me technical questions. I'd really love to be able to say to them, ‘I just wrote a computer algorithm, and the computer did it all. I wrote a program and it all just intelligently works it out,' but it doesn't exist, it's fools gold thinking that someone can sit there writing a piece of software that can make intelligent decisions [16]”. Or as Edgar Varese writes in ‘Liberation of Sound' in 1966: "no machine is a wizard, as we are beginning to think, and we must not expect our electronic devices to compose for us. [21]" 40 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY PATTERN RECOGNITION VS HIGHER STATE OF CONSCIOUSNESS One reason for this is that I believe that human cognitive abilities and vision, in particular, is quite good at figuring out algorithmic systems in action. Hence any kind of automation, no matter how complex it is, would only remain interesting until the core principle of system organization reveals itself through repetition allowing thus pattern recognition. This is why I avoid visualizing sound algorithmically, focusing instead on subjective synesthetic experience of how certain sounds appear to me as imaginary 3 dimensional objects and take those moments of inspiration as starting points for a process of mutating handwritten graphemes of letters into abstract shapes, deconstructing them and then altering those shapes on advanced stages of artwork development amplifying resonant qualities of material adopting topological qualities of tuning forks. Therefore it is not a visualization of sound, but rather a deconstructed grapheme defining a synthetic, abstract and artificial phonetic sound, an abstract phoneme. Charged and inspired by futurist aesthetics, transferred into a physical body to becomes a kind of resonant sound sculpture music instrument. With an intention to share and also cause a state of altered perception, and cause the state of synesthesia or a higher state of consciousness. A homage to experimental sound practices of futurists of early 20th century and an exercise in Ultra Futurism [14, 22]. 41 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY TC-11 Sound Synthesis process screenshot 17 17 http://www.bitshapesoftware.com/instruments/tc-11 accessed on 25.12.2018 42 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY FREE GRID SOUND SYNTHESIS For the sound synthesis part, I focus on working with a programmable multitouch synthesizer environment that allows to program the touch-points positions distances, the velocity of motion and sometimes pressure to alter optional variables in a sound synthesis formula, which makes it an extremely flexible instrument in synthesizing abstract sounds and noises. As Lougi Russolo writes in his 1913 ‘The Art of Noise' manifesto: "The need for and the search for the simultaneous union of different sounds (that is to say of its complex, the chord), came gradually: the assonant common chord was followed by chords enriched with some random dissonances, to end up with the persistent and complicated dissonances of contemporary music [14]’’. Synthesizers of the 20th century were still relying on the horizontal chord organization and fixed tone intervals. Drum machines and pattern based sequencers revolutionized the way music was produced which resulted in an explosion of an ever-evolving landscape of new electronic music genres. In the 21st century new interfaces appear with the implementation of multitouch screen technology. Many examples of multitouch software application are still relying on skeuomorphic control principles and tend to simply emulate the rhythm sequencers of the late 20 century models. 43 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY TC-11 Multi-Patch Sound Synthesis process screenshot 18 18 http://www.bitshapesoftware.com/instruments/tc-11 accessed on 25.12.2018 44 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY TC-11 however is a new kind of music interface software, as it takes advantage of the unique touch screen interfaces of high-end consumer devices allowing full programmability and customization [23] Developed in 2011 by Kevin Schlei, it provides new possibilities to simultaneously control different variables of a sound synthesis patch by 11 points of touch input forming a so-called ‘Painterly Interface' [24]. In his paper on TC-11 Schiel elaborates: "The transition from the sensor to "painterly interface" means accessing the intrinsic qualities of the multi-point data […] No virtual objects or widgets are incorporated for performance. Instead, the user's multi-touch performance is directly connected to the synthesis engine." [25] This introduces a method of composing and performing synthesized sound, that opens up a new dimension of gestures-to-input relation including micro- gestures. This also opens up possibilities for new, less subjective, yet customizable, readings and performances of Idiosyncratic notation systems and graphical music scores [24, 26]. Allowing therefore synesthetic cross-modal input transition, such as sonification of a stroke, hence sonification of a grapheme as well. 45 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY As Thor Magnusson puts it: "Live scoring, live notation, live coding, visual notation, dynamic notation, and animated notation are just a few names given to the possible cross-modal strands of exploration afforded by the computer [27]. In the process of sonic improvisation a sound inducing gesture is performed live, which is related to the gesture behind the grapheme that is at the core of the artwork’s shape. Therefore making a full cycle – coming in a turn of a spiral to the notion of language as a source, and language as a creative instrument once more. Paul Dourish emphasizes on the notions of gesture, writing and language: “Computation is a medium like any other, and a computational representation is like any symbol in that it gains meaning from its combination in use with each other and with symbols in other more traditional media such as speech, gesture, writing, architecture and so forth. Just like the meaning of the word, the meaning of the system is its use in the language” [28]. Therefore I aim to bring the expression of the writing gesture, deconstructed and detached from the grapheme, transformed and translated into the body of the sculpture through its form and shape and the medium of sound. 46 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY PHYSICALITY OF SOUND Sound remains to be the central modality of human communication in direct immediate presence while one way and even two-way channels of tele- communication are more dominated by visual content. In the post-factual world [29] accelerated by circulationism [30], as Hito Steyerl describes it: "Data, sounds, and images are now routinely transitioning beyond screens into a different state of matter [31]”. Almost any kind of non-time-based artwork may be translated into a 2D digital image and distributed via networks - compressing and downgrading, yet carrying the illusion of being seen and experienced [32,33]. However, the very nature of sound art in general, and sound sculpture in particular, underlines the importance of spectatorship, as an act of a physical presence, in order to experience the physical qualities of sound artwork in space. 47 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY CONCLUSION According to the “Sound Art” essay written by Julia Gerlach for 2012 ZKM sound art exhibition: “Sound art assigns the loudspeaker a twofold function. On the one hand, it is indispensable for creating a complex immersive sonic space; on the other, its sculptural qualities are also emphasized [34].” However, in the case of the ‘Sankhara - Sonic Formations' of ‘Formations’ series, not only sculptural qualities of a sound producing body are celebrated – but the sculpture itself is a speaker. A resonant body of work in which sound, form, and shape are manifested in its body in synergy. If synesthesia is what inspires the artwork, then the method of linguistic deconstruction influenced by futurists artists and digital aesthetics are the keys that secure the absence of any figurative image encoded in the work, leaving it open for interpretation. 48 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY DECLARATION OF AUTHORSHIP I hereby declare that the thesis submitted is my own unaided work. All direct or indirect sources used are acknowledged as references. I am aware that the thesis in digital form can be examined for the use of unauthorized aid and in order to determine whether the thesis as a whole or parts incorporated in it may be deemed as plagiarism. For the comparison of my work with exist- ing sources I agree that it shall be entered in a database where it shall also remain after examination, to enable comparison with future theses submitted. Further rights of reproduction and usage, however, are not granted here. This paper was not previously presented to another examination board and has not been published. ______27.12.2018 49 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY LIST OF REFERENCES [1] Krauss, Rosalind “Sculpture in the expanded Field” The MIT Press, 1979 [2] Rocha Iturbide, Manuel “The Expansion of Sound Sculpture and Sound Installation on Art” UNAM Mexico, 2014 online: [3] Grayson, John “Sound Sculpture”, A.R.C. The Aesthetic Research Centre of Canda, 1975. online: [4] Weibel, Peter “Sound Art, Sound as a Medium of Art”, ZKM, 2012 online: [5] Wittgenstein, Ludwig “Tractatus Logico-Philosophicus Logisch-philosophische Abhandlung” By First published by Kegan Paul, 1922. SIDE-BY-SIDE-BY-SIDE EDITION, VERSION 0.53 (FEBRUARY 5, 2018), containing the original German, alongside both the Ogden/Ramsey, and Pears/McGuinness English translations. online: [6] Bhikkhu, Bodhi (trans.) (“The Connected Discourses of the Buddha: A Translation of the Samyutta Nikaya”, Boston: Wisdom Publications, 2000 [7] William, Thomas; Davids, Rhys; Stede, William. “Pali-English Dictionary.”, pp. 664– 665., Motilal Banarsidass, 1921 [8] https://www.nihongomaster.com/dictionary/kanji/898/%E8%A1%8C [9] Ruiliang, Zhang “Analysis and Research on the Concept of Three Connotations” “The Review of Philosophy of Taiwan University” Eight issues, pp. 107-121. [10] Coulmas, F. “The Blackwell's Encyclopedia of Writing Systems”, Oxford: Blackwells, p.174, 1996 [11] Silver, Tony “Style Wars”, Director's Cut: 111 mins, Public Art Films, Plexfilm 1983 [12] Cage, John “Notations”, Something Else Press, Inc., 1969 50 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY [13] Silver, Leigh “Interview: Hans Ulrich Obrist Talks His Instagram Project "The Art of Handwriting”, Complex, 2014, online: [14] Russolo, Luigi “The Art of Noise” (futurist manifesto, 1913) translated by Robert Filliou, originally published as “A Great Bear Pamphlet”, Something Else Press, 1967 [15] Association for Psychological Science "Hearing Colors And Seeing Sounds: How Real Is Synesthesia?." ScienceDaily. ScienceDaily, 26 July 2007 online: [16] Kilroy, Nick “ on creating of Gantz Graph video’, Warp, 2008 online: [17] Carpenter, Siri “Everyday fantasia: The world of synesthesia” American Psychological Association, 2001 online: [18] Harrison, J.E.; Baron-Cohen, S “Synaesthesia: Classic and contemporary readings”, Cambridge, 1996 [19] Grossenbacher, P.G., & Lovelace, C.T. (). “Mechanisms of synesthesia: Cognitive and physiological constraints. Trends in Cognitive Sciences”, 5 (1), 36-41., 2001 [20] Cytowic, R.E. “The man who tasted shapes”, MIT Press, 1993 [21] Varese Edgard; Wen-Chung, Chou “The Liberation of Sound Author(s): and Source: Perspectives of New Music, Vol. 5, No. 1, pp. 11-19, Perspectives of New Music Stable, 1966 online: [22] Parreira da Silva, Manuela “Ultra-Futurism, Occultism and Queer Politics: Concerning an (almost unpublished) Letter of Raul Leal to F. T. Marinetti” International Yearbook of Futurism Studies, Volume 3, Issue 1, Pages 394–415, 2013 [23] Schlei, Kevin “TC-11 Version 3.0” about page, Bitshape, 2011 online: 51 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY [24] Levin, Golan “Painterly Interfaces for Audiovisual Performance”, B.S. Art and Design Massachusetts Institute of Technology, 1994 [25] Schlei, Kevin “TC-11: A Programmable Multi-Touch Synthesizer for the iPad”, University of Wisconsin-Milwaukee, 2012 online: [26] Enström, Warren; Dennis, Joshua; Lynch, Brian “Musical Notation for Multi-Touch Interfaces”, University of Wisconsin-Milwaukee, 2015 online: [27] Thor Magnusson, “Scoring with Code: Composing with algorithmic notation.” Organised Sound, 19, pp 268-275, 2014 [28] Dourish, Paul “Where the Action Is: The Foundations of Embodied Interaction.”, University of California, Irvine, 2001 [29] Alloa, Emmanuel “Who’s afraid of postfactual” online: [30] Steyerl, Hito “Too Much World: Is the Internet Dead?”, e-flux journal #49, 2013 [31] Steyerl, Hito “In Defence of the poor image”, e-flux journal #10, 2009 [32] McIntyre, Lee “Post Truth” The MIT Press, 2018 [33] Benjamin, Walter “The Work of Art in the Age of Its Technological Reproducibility, and Other Writings on Media” second version, THE BELKNAP PRESS OF HARVARD UNIVERSITY PRESS 2008 - Originally published: 1936 online: [34] Gerlach, Julia “The Aesthetic Potential of Sound”, ZKM, 2012 online: 52 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY TABLE OF IMAGES 1 Grayson, John “Sound Sculpture”, A.R.C. The Aesthetic Research Centre of Canada, 1975. 2 ‘Xing’ character as a digital typeface. 3 Examplse of Calligraphy writing of the 4 ShuoWenJieZi, The earliest complete 987 copy by XuXuan. Collection of Richard Sears 5 https://www.123klan.com/blogs/news/93981569-teenage-kicks-rennes-france accessed on 24.12.18 6 Davide Quayola “PTA”, fragments, https://www.quayola.com/pta/ accessed on 21.12.2018 7 Scien “Winter warm up!”, Scien archive - KingScien @ Flickr - https://www.flickr.com/photos/kingscien/ 8 Joshua Davis Archive - praystation.com 9 http://www.akaipro.com/, https://www.roland.com, http://yamaha.com/ accessed on 21.12.2018 10 Gordon, Al “Wildstar: Sky Zero”, #1 (Mar 1993, Image), Comic Book fragments - private collection scans. 11 Katsuhiro Otomo, ‘Akira’, Titan Books 1982 - 1990 - private collection scans. 12 Gordon, Al “Wildstar: Sky Zero”, #1 (Mar 1993, Image), Comic Book fragments - private collection scans. 13 Cage, John “Notations”, Something Else Press, Inc., 1969 14 Cage, John “Notations”, Something Else Press, Inc., 1969 15 http://blog.ivank.net/voronoi-diagram-in-javascript.html accesed on 17.11.2018 16 http://alexrutterford.com/graphic/autechre-gantz_graf/ 17 http://www.bitshapesoftware.com/instruments/tc-11 accessed on 25.12.2018 18 http://www.bitshapesoftware.com/instruments/tc-11 accessed on 25.12.2018 53 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY ADDITIONAL RESEARCH MATERIALS Schlei, Kevin “Bit Shape”, multitouch interface software Tsutsui, Masato; vh; synnack; “GANZ GRAF”, Maxforlive patch Coward, Harold “Derrida and Indian Philosophy.”, pp. 161–162., State University of New York Press, 1990 Barthes, Roland “Elements of Semiology”, Hill and Wang, 1968 Chessa, Luciano “Luigi Russolo, Futurist, Noise, Visual Arts, and the Occult”, University of California Press, 2012 54 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY THANK YOU To my professors: Ruth Schnell, Martin Kusch, Nicolaj Kirisits and Patricia J. Reis for challenging and teaching. To my partner Nicolaj Bischoff for support and inspiration. And to my friend Bettina Sofie Zottler. Nothing would have been possible without you. 55 SANKHARA - SONIC FORMATIONS NIKITA ZHUKOVSKIY 56