On Search for Emotion in Hindusthani Vocal music 1 2 2 Alicja Wieczorkowska , Ashoke Kumar Datta , Ranjan Sengupta , Nityananda Dey2 and Bhaswati Mukherjee3 1Multimedia Department, Polish-Japanese Institute of Information Technology, Koszykowa 86, 02-008 Warsaw, Poland 2Scientific Research Department, ITC Sangeet Research Academy, 1, N S C Bose Road, Tollygunge, Kolkata 700 040, India 3Center for Development of Advanced Computing, Kolkata, India Abstract Indian music is primarily melodic. Appreciation of it is supposed to be mainly based on the melodic content and rhythm, and not so much on the timbral content. In north Indian classical vocal music (also known as Hindusthani vocal music) raga forms the base bone. The most common style of performance is khayal. The first part in khayal is known as alap, wherein one tries to establish the image and emotional distinctiveness in slow tempo. The present study concentrates on the alap part of professional khayal performances. For each raga there are some distinctive sequences known as chalans and pakads. A raga is said to elicit some specific emotions from 8 categories referred to in Indian treatises as Rasas. The database for the experiments consists of short segments extracted from khayal performances from eleven most emotive ragas selected from the archives of ITCSRA. The first part of the study consists of identifying significant emotions evoked by the aforesaid segments in western listeners as well as native Indian listeners and the cross-cultural differences, if any. This also includes an examination of to what extent the raga–rasa relationship indicated in treatises matches with the observed data. The second part consists of the extraction of the distinctive note-sequences in these segments. The signal processing part performed here includes extraction of pitch contour, and segmentation of it into sequences of notes. The result of the examination of the relation of these sequences with the perceived emotion is also presented. Thirty seconds long music segments were used in our experiments, to check what specific emotion(s) they elicit. The listening tests showed that different segments from the same raga do not generally correspond to the emotions prescribed in Indian treaties. The responses from the two cultural groups are found to be similar. Melodic sequences vaguely relate with emotional response. INTRODUCTION Emotions give meaning to our lives. No aspect of our mental life is more important to the quality and meaning of our existence than emotions. They make life worth living, or sometimes ending. The English word 'emotion' is derived from the French word émouvoir which means ‘move’. Great classical philosophers—Plato, Aristotle, Spinoza, Descartes conceived emotion as responses to certain sorts of events triggering bodily changes and typically motivating characteristic behavior. It is difficult to find a consensus on the definition of emotion [4]. Most researchers would probably agree that emotions are relatively brief and intense reactions to goal-relevant changes in the environment that consist of many subcomponents: cognitive appraisal, subjective feeling, physiological arousal, expression, action tendency, and regulation. It therefore suggests that some part of the brain would be selectively activated [10]. Origin of emotion may be traced back to 200,000 years ago to semi-nomadic hunter–gatherer [11]. It is argued that their way of living, which involved cooperating in such activities as hunting, avoiding predators, finding food, rearing children, and also competing for resources, could be related to the origin of emotion. Most emotions are presumably adapted to living this way. Several of the activities are associated with basic survival problems that most organisms have in common. These problems, in turn, require specific types of adaptive reactions. A number of authors have suggested that such adaptive reactions are the prototypes of emotions as seen in humans [13], [14]. If various emotions are cognitively differentiable, as is again likely to be publicly agreed upon, there should be differences in sites of brain being excited for different emotions. Bower further suggested [1] that every emotion is associated with autonomic reactions and expressive behaviors. These expressive behaviors or responses to the same stimuli can vary depending on many factors external to the stimuli, like the mood of a person [4], [8], memory association of the person to the applied stimuli [4], [5], etc. Objects or a sequence of objects elicit feeling through a sequence of psychophysical processes. The sensory organs convert the signals from the objects to neural pulses. These pulse trains are processed in sub-cortical neural structures, which are primarily inherited. These processed signals produce perception in brain. Through the process of learning and experience we cognize the objects or sequence of objects from these perceived signals. Again through the process of learning we learn to associate these with some emotive environment in the past. This ultimately evokes emotion or feeling. The evaluation of emotional appraisals of stimuli may be done by having the person report the emotions they perceive as reaction to the stimuli. This can be done in several different ways such as verbal descriptions, choosing emotional terms from a list, or rating how well several different emotional terms describe the appraisal [4], [7]. The emotional terms used should be limited in number and as unambiguous as possible. It is also possible to represent these terms in vector forms. The splitting of emotion into dimensions is consistent with Bower’s network theory of emotion [3], [6]. However, the number of components and the type of components vary between studies [2], [6], [7], [9]. While sound stimuli may cause general physiological changes ("arousal"), these changes must be interpreted cognitively in order for a specific emotion to emerge. The listener does not come to the listening experience as a blank slate. He or she already has existing musically pertinent knowledge. Even for a musically untrained listener the general exposure to listening to music since childhood is also learning, though not formal. Thus any emotional behavior, even habitual and seemingly automatic and natural, is actually learned. In case of music this behavior serves as a means of communication, since often emotional behavior is differentiable and intelligible. One major problem that arises in the study of the emotional power of music is that the emotional content of music is very subjective. A piece of music may be undeniably emotionally powerful, and at the same time be experienced in very different ways by each person who hears it. The emotion created by a piece of music may be affected by memories associated with the piece, by the environment it is being played in, by the mood of the person listening and their personality, by the culture they were brought up in; by any number of factors both impossible to control and impossible to quantify. Under such circumstances, it is extremely difficult to deduce what intrinsic quality of the music, if any, created a specific emotional response in the listener. It seems that the listeners experience gross emotion through the unfolding of successive events. If the successive events are always predictable, the emotion is boredom, unless an association of past evokes a specific emotion. Again, extensive uncertainties are likely to lead to apprehension and anxiety. These two emotional experiences may therefore be the robust ones. As soon as the unexpected is experienced, the listener attempts to fit it into the general system of beliefs relevant to the theme. This may happen in one of the following three ways: (1) The mind may suspend judgment, expecting that the subsequent event will bring in clarity. (2) If no clarification takes place, irritation will set in. (3) The expected consequent may be seen as a purposeful blunder. When we expect music to convey emotion, we accept it to be a language, as emotion is built upon the meaning. In order to extract the right emotion, it is essential that the listener is acquainted with the grammar, and thus the extra-musical world of concepts, actions, emotional states, and character. Even if he does not have the appropriate grammar in his mind, he can still extract some emotional meaning out of it, with some of his own stock grammar. In this sense emotion in music may be called referential. A piece of music can be universally pleasant or universally irritating. On the other hand, a musical stimulus or series of stimuli can be considered to indicate and point to other musical events, which are about to happen, rather than extra-musical concepts and objects. That is, one musical event (be it a tone, a phrase, or a whole section) has meaning because it points to and makes us expect another musical event. Even then the affective experience is still dependent on cognition (involving a process of intellection, conscious or unconscious) that cannot be restricted to the musical concepts alone. The musical expectations and experience grow out of the innate processes of grouping, closure, and good continuation in Gestalt psychology. Hevner [12] studied as early as in 1936 grouping of emotions described by listeners using adjectives through listening experiments. The experiments substantiated a hypothesis that music inherently carries emotional meaning. Hevner discovered the existence of clusters of descriptive adjectives and laid them out in a circle. These are: a) cheerful, gay, happy, b) fanciful, light, c) delicate, graceful, d) dreamy, leisurely, e) longing, pathetic, f) dark, depressing, g) sacred, spiritual, h) dramatic, emphatic, i) agitated, exciting, j) frustrated, k) mysterious, spooky, l) passionate, m) bluesy. Actually, emotion detection in musical information is better considered as a ‘Multi-label Classification problem’, where the music sounds are classified into multiple classes simultaneously. That means that a single music sound may be characterized by more than one label, e.g. both “dreamy” and “cheerful.” Figure 1: Possible descriptions of emotion using valence and arousal [22].