Sheet Music Unbound

http://researchcommons.waikato.ac.nz/ Research Commons at the University of Waikato Copyright Statement: The digital copy of this thesis is protected by the Copyright Act 1994 (New Zealand). The thesis may be consulted by you, provided you comply with the provisions of the Act and the following conditions of use: Any use you make of these documents or images must be for research or private study purposes only, and you may not make them available to any other person. Authors control the copyright of their thesis. You will recognise the author’s right to be identified as the author of the thesis, and due acknowledgement will be made to the author where appropriate. You will obtain the author’s permission before publishing any material from the thesis. Sheet Music Unbound A fluid approach to sheet music display and annotation on a multi-touch screen Beverley Alice Laundry This thesis is submitted in partial fulfillment of the requirements for the Degree of Master of Science at the University of Waikato. July 2011 © 2011 Beverley Laundry Abstract In this thesis we present the design and prototype implementation of a Digital Music Stand that focuses on fluid music layout management and free-form digital ink annotation. An analysis of user constraints and available technology lead us to select a 21.5‖ multi-touch monitor as the preferred input and display device. This comfortably displays two A4 pages of music side by side with space for a control panel. The analysis also identified single handed input as a viable choice for musicians. Finger input was chosen to avoid the need for any additional input equipment. To support layout reflow and zooming we develop a vector based music representation, based around the bar structure. This representation supports animation of transitions, in such a way as to give responsive dynamic interaction with multi-touch gesture input. In developing the prototype, particular attention was paid to the problem of drawing small, intricate annotation accurately located on the music using a fingertip. The zoomable nature of the music structure was leveraged to accomplish this, and an evaluation carried out to establish the best level of magnification. The thesis demonstrates, in the context of music, that annotation and layout management (typically treated as two distinct tasks) can be integrated into a single task yielding fluid and natural interaction. i ii Dedication To my friends and family who convinced me that this was possible, and to Oscar who stayed put long enough to prove them right. iii iv Table of Contents Abstract ....................................................................................................................... i Dedication ................................................................................................................. iii Table of Contents ..................................................................................................... v List of Figures ........................................................................................................... ix Chapter 1 - Introduction ........................................................................................ 1 Chapter 2 - Related Work ...................................................................................... 3 2.1 Digital Music Stand Development ................................................................... 3 What is a Digital Music Stand?................................................................................. 3 Muse ............................................................................................................................... 3 Page Turning ................................................................................................................ 4 Commercial Products ................................................................................................. 6 Research Systems ...................................................................................................... 10 2.2 Annotation systems ........................................................................................... 13 Digital Ink ..................................................................................................................... 13 Sketching Music ......................................................................................................... 17 Annotating Sheet Music .......................................................................................... 18 2.3 Summary .............................................................................................................. 21 Chapter 3 - Design Considerations.................................................................... 23 3.1 Physical restraints on Musicians ..................................................................... 23 Introduction ................................................................................................................ 23 Physical Restraints by Instrument .......................................................................... 23 Table 1 – Physical restraints on musicians (Part 1 of 2) .................................. 26 Table 1 – Physical restraints on musicians (Part 2 of 2) .................................. 28 v Observations .............................................................................................................. 30 3.2 Hardware Considerations ................................................................................ 33 3.3 Device Options ................................................................................................... 39 3.4 Software Environment – WPF with .NET ..................................................... 42 Chapter 4 - Sheet Music Unbound .................................................................... 45 4.1 Breaking the boundary of the page ............................................................. 45 4.2 How to store/represent a bar......................................................................... 47 PDF ................................................................................................................................ 48 XAML ............................................................................................................................ 49 4.3 Breaking music into bars ................................................................................. 52 Where does a bar begin? ....................................................................................... 52 Where does a bar end? ........................................................................................... 54 Aligning bars vertically ............................................................................................. 57 Handling multiple parts ........................................................................................... 61 The XAML bar creation process ............................................................................ 68 Automation of the XAML bar creation process ................................................ 79 4.4 Fluid Layout ......................................................................................................... 80 4.5 Score Personalization ....................................................................................... 86 Choosing display size ............................................................................................... 86 Moving and hiding sections of music .................................................................. 90 Adding cues or display complete score .............................................................. 92 4.6 Some ideas on page turning .......................................................................... 94 Visualisation options ................................................................................................. 94 Initiating a page turn ................................................................................................ 99 vi Animation speed ....................................................................................................... 99 Chapter 5 - Supporting Annotation ................................................................101 5.1 Digital Ink ...........................................................................................................101 5.2 Storing annotations across bars...................................................................101 WPF Ink Canvas .......................................................................................................103 Custom Ink Control .................................................................................................107 5.3 Creating space for annotations ....................................................................110 Corner Drag ..............................................................................................................113 Tab Style ....................................................................................................................115 Roller Blinds ..............................................................................................................117 5.4 Annotation Input Scale ...................................................................................121 Zoom to Annotate ..................................................................................................122 Zoom in Place ..........................................................................................................122

Load more