Tscore: Making Computers and Humans Talk About Time
Total Page:16
File Type:pdf, Size:1020Kb
tScore: Making Computers and Humans Talk about Time Markus Lepper1 and Baltasar Trancon´ y Widemann1;2 1<semantics/> GmbH, Berlin, Germany 2Ilmenau University of Technology, Ilmenau, Germany Keywords: Knowledge Acquisition, Temporal Data, Man–Machine Interface, Domain-specific Languages. Abstract: Textual denotation of temporal data is a challenge. In different domains very different notation systems have been developed for pen and paper during history, reflecting domain specific theory and practice. They work with symbolic representation of activities and combined expressional and spatial representation of time. In contrast, existing computer-readable formats are either simple lists or complicated expression languages. The formalism and software presented here, called tScore, try to bridge the gap between these two worlds by studying the most advanced example of the former, the Conventional Western Notation of music. By ab- stracting its basic principles, a generic notation framework is defined, suited for reading and writing by both humans and computers. The syntactic framework of the front-end representation and a mathematical formu- lation of the underlying semantics are given, which both are parametrisable and allow to plug in application specific parsers and data models. The current state of library implementation is shortly sketched, together with a practical example of moderately complex music notation. 1 MOTIVATION AND DESIGN GOALS OF tScore In the first place, tScore is the name of a new text for- mat for denotating arbitrary time-related information structures. Furthermore, it is the software framework T for processing this information. The development of tScore starts from the obser- (c) vation that a temporal description language is highly (b) desirable which . (a) • can be read and written equally well by humans and computers, • is neither restricted to one fixed model of “time” nor to prescribed parameter values, Figure 1: Information Flow Between User, Computer and Stage. • and can be handled with paper and pencil, or with chalk and blackboard, as the minimally required • computer aided analysis, information processing hardware. • documentation and type setting into multiple for- There are many potential application areas for mats. such a language in the fields of handling technical sys- tems and of æsthetic production. In general it enables The most complex use cases arise in the context the communication between users and computer sys- of æsthetic production and live performance. Figure 1 tems for the purposes of shows different flows of information related to some live stage activity between a human user and a com- • automated performance (sequencing), puter system: (a) The user writes a score which is • automated transformation/generation of temporal “sequenced” or “performed” by the IT system, (b) the data, user writes a protocol of what he/she perceives, which 176 Lepper M. and Trancón y Widemann B.. tScore: Makes Computers and Humans Talk About Time. DOI: 10.5220/0004540701760183 In Proceedings of the International Conference on Knowledge Engineering and Ontology Development (KEOD-2013), pages 176-183 ISBN: 978-989-8565-81-5 Copyright c 2013 SCITEPRESS (Science and Technology Publications, Lda.) tScore:MakesComputersandHumansTalkAboutTime is rendered nicely by the IT system, (c) the computer distances on the writing surface, in one distin- writes a protocol (using motion/gesture/pitch detec- guished direction. tion technologies) and delivers a readable score to The traditional German word is “Streckennota- the human. In all theses cases (and many more one tion”, and the technique is employed in avant- can think of) a tScore score object can be the cen- garde music, mixed with classical notation ele- tral means for information exchange. The motivation ments or even exclusively. The different value as- of the authors came out of a concrete composition pects which change in time are denotated either project, where it is necessary to unify “Conventional by traditional pitch indication, using five line mu- Western Music Notation” (CWN) and technical con- sical note systems, or by some “y-axis” which in- trol parameters for electronic sound processing in one terpretes some curved lines. single human- and computer-readable score. • On the opposite extreme are the “symbolic” rep- But tScore as such shall not be restricted to these resentations. In the field of music these are two domains. In fact, it is one of its central design cri- the pure expression languages, like “musixT X” teria that the “time axis” as well as the “value range” E (Taupin et al., 2002), “Guido”, “lilypond” (lily- are in no way a priori restricted, but can be mapped pond, 2011), “musicXML” (musicxml, 2011) and to almost arbitrary user-defined domains. many more. Here no relation between spatial lo- The main problem w.r.t. the time axis is that even cation and time is defined. Only the mere sequen- notion the basic of time, the assumptions about its be- tial order of sub-terms carries any semantics, in haviour and structure, the permitted denotations and many cases denotating the mere sequential order model of algebraic operations, the whole so-called in time. Duration, synchronicity, temporal dis- time , shall not be defined in advance, but definable tances and parallelism are expressed by expres- by the user, fitting the application context. The value sion terms only, built from lexical atoms and com- axis is in most cases less critical, but, e.g. in case of bining operators. CWN, subject of controversial discussions and cannot be pre-defined either. • Somewhere in the middle, more to the symbolic Of course, all these necessary definitions need not side, is the classical music notation CWN. It is be reiterated from scratch, but the user is given a col- a term language because the exact values, e.g. lection of pre-defined library components, which can of durations, are encoded by the chosen lexi- be parameterised and plugged together. cal atoms and combining operators (e.g. beams). So tScore is intended to serve as a denotation of Nevertheless, a certain flavour of analog represen- temporal structures in very diverse contexts, like tation is also present: The distance of the “terms”, the note symbols, and their horizontal alignment • light control across different, simultaneously played voices, is • video cue lists expected to “redundantly” represent the temporal • kinetic sculptures / robots positioning as spatial layout. The balance of ana- log and symbolic aspects has been carefully anal- • dramaturgy / radio play ysed by DAHLHAUS (Dahlhaus, 1965). • web animation But all of these existing approaches lack the desir- • stage performance able flexibility. The above-mentioned problem how to • music integrate CWN and technical control parameters is in • (every other conceivable time-related structure) practice usually solved by “abusing” some formalism, e.g. by a brute-force mapping of “Midi” control val- ues to some semantic user domain without any type 2 EXISTING APPROACHES TO check or declaration. This ubiquitous practice is a “hack” and neither DENOTATE TIME adequate to professional software engineering nor to informatics as the scientific discipline which analyses The existing approaches for denotating temporal culturally determined information structures. structures can be arranged on a scale from “analog” to “symbolic”, following the categories of GOODMAN (Goodman, 1976): • Modern variants of pure graphical music nota- tion are “analog” encodings. This means that dis- tances in time are proportionally represented by 177 KEOD2013-InternationalConferenceonKnowledgeEngineeringandOntologyDevelopment 1 2 3 7 6 8 PARS prima 4 T 19 ! ! 20 5 VOX sop a b c (d e)f (g a h) 9 P dyn f f > pp 10 P art - - > > > T 2011 21 11 VOX sop a b (e f) P nota [ ! clef-vl ] P art ( ).- PARS seconda ... 11 12 EOF Figure 2: Example of the tScore Input Format. 3 tScore AS A CONSEQUENCE • But even when applying this restriction, there is a FROM CWN multitude of semantic interpretations, with partly contradictory assumptions. The CWN has been developed over hundreds of years, • It is hardly readable by computers. and is the result of both research results of dedicated The design of tScore is the consequence of gen- specialists, and of every-day practical experience of eralizing the “orthogonal” aspects of CWN: the musical community. Consequently, it has a lot of advantages: • Time flows from left to right, top to bottom where lines must be broken. • It is highly ergonomic, economic and compact, due to the requirements and experiences of mu- • Temporal distribution is indicated by dividing sical practice. space. • • It is flexible, parametrisable and adaptable: With Synchronicity is established by super-position and some “plugged-in” additional definitions it can horizontal alignment. cover a large historic era from PEROTIN to • Multiple voices run in parallel, which again con- STOCKHAUSEN. sist of multiple parameter tracks. • As mentioned above, it can be “abused” for notat- On the other hand tScore avoids the above- ing temporal structures of value ranges outside the mentioned restrictions and deficiencies of CWN and traditional realm of music, e.g. physical motion or ad-hoc computer languages: film rhythms. • There is no pre-defined model of time, but only Naturally, the principle of historic development minimally necessary basic definitions. led to detours and idiosyncrasies: • There are no pre-definitions on the value axes, but • There are multitudes of notational elements for the