THE QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY iFirst,1–28 The integration of stimulus dimensions in the perception of music Jon B. Prince School of Psychology, Murdoch University, Murdoch, WA, Australia A central aim of cognitive psychology is to explain how we integrate stimulus dimensions into a unified percept, but how the dimensions of pitch and time combine in the perception of music remains a largely unresolved issue. The goal of this study was to test the effect of varying the degree of conformity to dimensional structure in pitch and time (specifically, tonality and metre) on goodness ratings and classifications of melodies. The pitches and durations of melodies were either presented in their original order, as a reordered sequence, or replaced with random elements. Musically trained and untrained participants (24 each) rated melodic goodness, attending selectively to the dimensions of pitch, time, or both. Also, 24 trained participants classified whether or not the melodies were tonal, metric, or both. Pitch and temporal manipulations always influenced responses, but participants successfully emphasized either dimension in accordance with instructions. Effects of pitch and time were mostly independent for selective attention conditions, but more interactive when evaluating both dimensions. When interactions occurred, the effect of either dimension increased as the other dimension conformed more to its original structure. Relative main effect sizes (| pitch h2 – time h2 |) predicted the strength of pitch–time interactions (pitch × time h2); interactions were stronger when main effect sizes were more evenly matched. These results have implications for dimen- sional integration in several domains. Relative main effect size could serve as an indicator of dimen- sional salience, such that interactions are more likely when dimensions are equally salient. Keywords: Pitch; Time; Music; Dimensions; Integration. The cognitive mechanisms responsible for the per- dimensions combine in order to form a unified Downloaded By: [Murdoch University] At: 03:47 19 May 2011 ceptual integration of stimulus dimensions are and coherent mental percept is a primary goal complex and multifaceted. Nonetheless, percep- of cognitive psychology (Treisman & Gelade, tion of multidimensional objects in the natural 1980). Therefore much research has concentrated environment proceeds automatically and without on the central issue of how stimulus dimensions conscious effort. Understanding how these combine in perceptual processing. Correspondence should be addressed to Jon B. Prince, School of Psychology, Murdoch University, Murdoch, WA 6150, Australia. E-mail: [email protected] Grants from the Natural Sciences and Engineering Council of Canada to Mark A. Schmuckler and William F. Thompson sup- ported part of this research, in addition to grants from the National Science Foundation of the USA awarded to Peter Q. Pfordresher. Portions of this work were presented at the Auditory Perception, Cognition, and Action meeting, Boston, MA, November 2009. The author would like to thank Peter Q. Pfordresher for helpful comments and discussions. # 2011 The Experimental Psychology Society 1 http://www.psypress.com/qjep DOI:10.1080/17470218.2011.573080 PRINCE Seminal work by Garner (1974) introduced the Despite considerable advances in the field of concept of integral and separable dimensions, spe- multidimensional visual object perception, less is cifying that integral dimensions cannot be isolated known about integration of multidimensional in perceptual processing, whereas separable properties in patterned auditory stimuli such as dimensions can be processed independently. speech and music. Rather than representing a When attending a stimulus comprised of integral static object, auditory sequences unfold over dimensions, changes along one dimension will time. Accordingly, integrating the dimensions of affect judgements along the other dimension; for such sequences has an additional layer of complex- separable dimensions no such interfering effect ity by virtue of their dynamic nature. In the case of arises. However, the integrality or separability of music, the main stimulus dimensions of pitch and dimensions is influenced by their relative discri- time can be manipulated independently, and there minability, which corresponds to the baseline dif- is a rich background on these two dimensions from ficulty of perceiving a given dimension (Garner & the perspectives of music theory (Aldwell & Felfoldy, 1970) and can be measured using reac- Schacter, 2002; Schenker, 1935/1979) and cogni- ′ tion times, accuracy, or d . A more discriminable tive science (Jones & Boltz, 1989; Krumhansl, dimension will interfere with a less discriminable 1990). Nevertheless, there is still little agreement dimension, even if they have demonstrated separ- on how exactly pitch and time contribute to, and ability under conditions of equal discriminability. combine in, our internal representation of music Yet many dimensions defy a neat classification (this process is referred to hereafter as pitch– into integral and separable dimensions. For time integration). Responses to any stimulus will example, asymmetric interactions may occur even depend on the internal representation of its fea- when dimensions have been matched for discri- tures, and music is no exception to this principle. minability, such that one dimension interferes Thus, understanding how pitch and time integrate with another, but not vice versa (Garner, 1976). is vital to research on musical behaviour (listening, Further research refined the original framework performing, and composing) and can contribute to of integral and separable dimensions into a more the larger literature on dimensional integration in detailed and nuanced view of dimensional inte- other sensory modalities. gration. One such development was the idea of Much research on pitch–time integration has decisional and perceptual separability (Ashby & taken a position supporting either the idea that Townsend, 1986), which proposes that a dimen- pitch and time are independent, or that they sion may interfere with another at either of these are interactive (as reviewed in Ellis & Jones, processing levels when responding to a multi- 2009; Krumhansl, 2000; and Prince, Thompson, Downloaded By: [Murdoch University] At: 03:47 19 May 2011 dimensional stimulus. Other authors have pro- et al., 2009). Palmer and Krumhansl (1987a) pro- posed the concepts of dimensional imbalance and vided one of the landmark demonstrations of inde- dimensional uncertainty (Melara & Algom, pendence, where an additive combination of pitch 2003), theorizing that the amount of information and temporal information predicted judgements of present in a given dimension influences how the melodic phrase goodness, suggesting that pitch perceiver attends to the stimulus. Additionally, and time were processed separately. Other research some stimulus dimensions appear to be privileged proposed instead that pitch and temporal infor- and can exert dominance (i.e., asymmetric mation is processed jointly, because variations in interference) over other dimensions despite rhythmic structure influenced detection of pertur- careful matching of other factors (Atkinson, bations to pitch patterns (Jones, Boltz, & Kidd, Tipples, Burt, & Young, 2005; Haxby, Hoffman, 1982). Since then, the numerous reports of inde- & Gobbini, 2000; Mullennix & Pisoni, 1990; pendence and interaction of these dimensions Prince, Thompson, & Schmuckler, 2009; have led to much confusion over the nature of Schweinberger, Burton, & Kelly, 1999; Tong, pitch–time integration. It is more likely, therefore, Francis, & Gandour, 2008). that pitch and time function neither as purely 2 THE QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY, 0000, 00 (0) STIMULUS DIMENSIONS independent nor as interactive dimensions, but of individual (local) stimulus elements foster that pitch–time integration varies on the basis of observations of independence; conversely, inter- a number of stimulus, task, and cultural factors. actions are more common when attending to In real-life situations, the characteristics of the characteristics of a stimulus on a larger time- music that a listener hears (i.e., the stimulus) scale (global tasks). Because attending to larger, may highlight one dimension, such as emphasizing more global structural relations will require com- the pitch dimension by using a repeating pattern, bining a number of preceding events, such tasks or the time dimension by accelerating the tempo. are likely to require processing of the stimulus in Similarly, listeners may choose to focus on differ- an integrated form. This integrated internal rep- ent aspects of a musical sequence, such as listening resentation would include both pitch and temporal to a particular instrument, a rhythmic figure, or the information; thus these stimulus dimensions are timbral features of the music. Based on what more likely to interact in perceptual processing. aspect is of interest, the task of the listener will Conversely, attending locally to an isolated event involve isolating different dimensions of the can proceed without reference to more global music. Finally, listeners encultured in the properties of a stimulus, not depending on a Western musical style will probably hear music joint (integrated) representation, and therefore of another culture differently from a native listener fostering independent relations. Although typical (e.g., Castellano, Krumhansl, & Bharucha, 1984).