Improvisation Planning and Jam Session Design using concepts of Sequence Variation and Flow Experience Shlomo Dubnov, Gérard Assayag To cite this version: Shlomo Dubnov, Gérard Assayag. Improvisation Planning and Jam Session Design using concepts of Sequence Variation and Flow Experience. Sound and Music Computing 2005, Nov 2005, Salerno, Italy. pp.1-1. hal-01161335 HAL Id: hal-01161335 https://hal.archives-ouvertes.fr/hal-01161335 Submitted on 28 Jan 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Improvisation Planning and Jam Session Design using concepts of Sequence Variation and Flow Experience S. Dubnov G. Assayag CRCA/Music, UCSD IRCAM [email protected] [email protected] ABSTRACT choices of the different participants in the We describe a model for improvisation design based on improvisation. Factor Oracle automation, which is extended to perform learning and analysis of incoming sequences in terms of As a model for machine improvisation method we sequence variation parameters, namely replication, choose Factor Oracle (FO), an automaton that recombination and innovation. These parameters effectively captures all sub-phrases (factors) in a describe the improvisation plan and allow designing new sequence. This automation is extended so as to improvisations or analysis and modification of plans of allow production of variations on a template existing improvisations. We further introduce an idea of sequence with control over the amount of flow experience that represents the various improvisation randomness or innovation and analysis of new situations in a mental space that allows defining interactions between improvisers in terms of mental material so as to recognize which FO and which states and behavioural scripts. segment within FO was used to produce a variation, to what extent the variation differs from the 1. INTRODUCTION reference template and the rate of innovation versus replication and recombination of the different In the field of music improvisation with computers materials represented by FO. there has been recently a great advance in music Equipped with these improvisational and listening modeling that allows capturing stylistic musical tools, we proceed to construct an interface for surface rules in a manner that allows musically communication between the improvisers in terms of meaningful interaction between humans and higher order notions of emotional or related computers. One of the main challenges in cognitive descriptive characteristics. One of the producing a larger form or improvisation of problems in designating such a mapping between significant span is in creating “handles” or means system or data parameters and cognitive categories of control of musical generation, so that the result is the lack of clear definition and agreement on becomes more then accidental play of imitation and emotional terminology and representation. Various response. In this paper we try to identify the schemes have been proposed, such as basic principles and possible methods for creating a emotional categories, emotional dimensions or meaningful play between computers and human spaces, grouping according to cognitive eliciting improvisers. The main tasks at hand are the conditions etc. following: In this work we use one such model, which relates emotions only indirectly to mental states using a 1. Defining meaningful controls for music notion of experience flow. The concept of Flow material generated by computer. Experience has been introduced in psychology to 2. Allowing machine analysis and recognition describe an optimal experience of humans when of these parameters. dealing with tasks that involve certain balance of 3. Characterization of the overall musical task skills and task complexity. This concept has experience that is created as a result of been recently applied to design of media specific improvisation choices. presentation, such as choice of levels in computer 4. Defining rules of interaction between games. The idea of flow describes the overall players (human and machine) that enhances engagement of a player in dynamic experience, such or inhibits (supports or contradicts) the as learning or playing computer games [4]. We shall explain in more detail the flow model later on in the two factor links, one suffix link, and one factor link, paper. It should be noted that our model of flow one generates the sequence ABBAB. Same differs from the original flow idea in terms of the sequence can be generated also using other parameters and axes of the flow space. The purpose sequences of links (such as three factor links, one of using flow in our work is to allow identifying suffix link and then two factor links). Musical FO’s and labeling improvisation states for designing have been implemented in real-time environment musical interactions. Before proceeding to discuss called OMAx (OpenMusic+Max) by one of the these aspects of our model we introduce the authors and Marc Chemillier, and successfully methods for sequence modeling and the experimented in real-life, live situations with Jazz improvisation model that are used in this work. performers. B 2. THE MODEL A B 2.1 Sequence Modeling, Improvisation and A B A B A B A B A A B B Analysis 0 1 2 3 4 Machine improvisation and closely related style learning problems usually consider building representations of time-based media data, such as music or dance, either by explicit coding of rules or Figure 1: Factor Oracle of a sequence. Black arcs represent the forward transitions (factor links). Grey arcs are the suffix links applying machine learning methods. Stylistic machines for dance emulation try to capture movement by training statistical models from a 2.2 The Improvisation Model sequence of motion-capture sequences [3]. Stylistic Our model consists of one or more players, set of learning of musical style use statistical models of template sequences, which may preexist, or may be melodies or polyphonies to recreate variants of learned on the fly by an FO listening to a player, musical examples [5]. These and additional and a “composition design” that represents the researches indicate that emulation of particular improvisation parameters and / or sequence of behaviors is possible and that credible behavior mental states and interactions according to these could be produced by a computer for a specific states, possibly changing in time. An individual domain. player consists of an automation (FO) that Deterministic model for sequence modeling and efficiently captures the improvisational and analysis improvisation was introduced in [2]. The model possibilities with respect to the set of template employs an automation called Factor Oracle (FO) sequences. As will be explained below, one of the [1] that is computed incrementally and efficiently goals that we try to accomplish in this work is to represents all factors in a sequence with least find a mapping between improvisation parameters number of states and linear number of transition and states of the improviser, which we shall call called factor links. Beyond its compactness, FO “mental states”. This mapping shall be done using a also provides, via its construction, a set of pointers, flow diagram, which is a modification of the flow called suffix links, from every point in the sequence experience model, as will be explained in the next to its last repeated factor. The transition and suffix paragraph. links allow “traveling” from any point in the In the case of a single improviser, the player sequence to future and past locations where similar operates according to a predetermined sequence that factors reside. Following factor links replicates specifies explicitly the improvisation parameters or factors from the sequence. Following suffix links their mapping through mental states. In the case of creates a recombined sequence in which a suffix is two improvisers, a communication exists between known to belong to the original sequence, a process the players by exchanging musical sequences and close to variable memory Markov models. Figure 1 inferring the mental states of each player. shows an example of FO of the sequence Interaction rules specify the logic of the player ABABABABAABB. Traversing this FO can reactions to each other’s mental states. generate new sequences. For instance, by following 2.3 Analysis Module close to 1, recombination probability Prec = 1-Prep The analysis of an incoming sequence is done and no innovation (Pinn = 0) during x phrases, then relatively to the available templates (modeled by increase Prec during y phrases, keep increasing it FO’s), by estimating some sort of similarity with until climax after z phrases and then introduce new these templates. The FO’s may have captured material by assigning high probability to Pinn, and previous sequences played by the performers so on. It should be noted that using the probabilistic during the same session, or they may relate to interpretation, same plan results in different formerly archived sessions, or even to pieces in the improvisations (different instances of same plan). repertoire. A particular case that must be considered is The general situation is: player B receives a new “learning on the fly”, described as follows: Suppose sequence S from player A. B tries to relate this we do not have any template at hand and we start sequence to an FO F which models a template from an empty FO. A performer plays an sequence T. So B runs S along F trying to devise to improvisation, which is incrementally learned into which extent S is build by repetition of T, by this FO.