A Unified Toolkit for Accessing Human Interface Devices in Pure Data And
Total Page：16
File Type：pdf, Size：1020Kb
Proceedings of the 2007 Conference on New Interfaces for Musical Expression (NIME07), New York, NY, USA A Unified Toolkit for Accessing Human Interface Devices in Pure Data and Max/MSP Hans-Christoph Steiner David Merrill Olaf Matthes IDMI/Polytechnic University MIT Media Lab nullmedium Brooklyn, NY, USA Cambridge, MA, USA Greifswald, Germany [email protected] [email protected] [email protected] ABSTRACT For an electronic musical instrument designer, easy ac- In this paper we discuss our progress on the HID toolkit, cess to gestural data (motion, pressure, buttonpresses, etc.) a collection of software modules for the Pure Data and and output capabilities (lights, force feedback) enables rapid Max/MSP programming environments that provide unified, prototyping of musical affordances and mapping strategies. user-friendly and cross-platform access to human interface Many HIDs are temporally and gesturally sensitive enough devices (HIDs) such as joysticks, digitizer tablets, and for musical performance, including gaming mice, certain joy- stomp-pads. These HIDs are ubiquitous, inexpensive and sticks, and most graphics tablets. Another factor that makes capable of sensing a wide range of human gesture, making many existing HIDs appealing for electronic music perfor- them appealing interfaces for interactive media control. mances is that they are relatively familiar objects (as com- However, it is difficult to utilize many of these devices for pared to custom electronic hardware), which can allow an custom-made applications, particularly for novices. The audience to more easily understand the connection between modules we discuss in this paper are [hidio] 1, which a performer’s actions and the resulting sonic output. A num- handles incoming and outgoing data between a patch ber of contemporary musicians have used standard HIDs as and a HID, and [input noticer], which monitors HID musical controllers. Leon Gruenbaum’s Samchillian Tip Tip plug/unplug events. The goal in creating these modules Tip Cheeepeeeee [10] is built upon a standard ergonomic is to preserve maximal flexibility in accessing the input keyboard; Luke Dubois plays the Wacom tablet with The and output capabilities of HIDs, in a manner that is ap- Freight Elevator Quartet [8]; August Black started by hack- proachable for both sophisticated and beginning designers. ing existing devices to make his El Lechero [6]; Loic Kessous This paper documents our design notes and implementa- has built his instrument using a Wacom tablet and a joystick tion considerations, current progress, and ideas for future [11]. extensions to the HID toolkit. In addition to the ranks of experienced professionals lever- aging HIDs, there is a growing number of students and other novices around the world interested in ‘physical computing’ 1. INTRODUCTION AND MOTIVATION and its application to media art. Several universities now Human Interface Devices (HIDs2) such as joysticks, digi- offer courses specifically in physical computing and/or the tizer tablets, keyboards, mice, gamepads and ‘stomp-pads’ design of electronic music controllers [16] [7] [3] [2]. For stu- have become widely available and inexpensive. Most ex- dents, the utilization of HIDs for expressive control of sound, isting HIDs are built robustly, and due to their prevalence video, or other media can be an appealing alternative to the and low cost, there is growing interest in utilizing them for construction of custom hardware, an endeavor that requires musical control and other performance applications. a much greater level of technical sophistication. A number of prototyping platforms have become available lately that 1a word in square brackets denotes a Max/Pd object also support HID standards, such as CUI [14] and Phidgets 2Human Interface Device (HID) has become the standard [9]. term to describe devices designed to sense physical human- As well as input capabilities, some HIDs provide audi- computer input, and to send feedback from computers to tory, visual or haptic feedback. Our goal is to provide easy- users. The term ‘HID’ most commonly refers to the USB- to-use input and output access to a wide range of HIDs, HID specification, which is a USB device class that describes human interface devices that utilize USB communication. through a unified, cross-platform, standardized and coher- We use ‘HID’ more generally, to mean the larger set of hu- ent approach. man interface devices that may or may not use USB. [hidio] is a software object for Pure Data and Max/MSP that provides unified and standardized access to HIDs. Pure Data and Max/MSP are popular environments for imple- menting physical interfaces for musical performance or other Permission to make digital or hard copies of all or part of this work for media-rich interactive applications. Our goal in creating personal or classroom use is granted without fee provided that copies are [hidio] is that the end user of these environments will not not made or distributed for profit or commercial advantage and that copies have to learn how to use a separate object for each differ- bear this notice and the full citation on the first page. To copy otherwise, or ent HID, and that a patch written on one operating system republish, to post on servers or to redistribute to lists, requires prior specific platform should work in the same way on another. We have permission and/or a fee. NIME07, June 7-9, 2007 New York, NY, USA tried to preserve flexibility while simplifying the complicated Copyright 2007. Copyright remains with the authors. 375 Proceedings of the 2007 Conference on New Interfaces for Musical Expression (NIME07), New York, NY, USA HID APIs as much as possible. 3.2 Managing Devices In this paper we present our current progress implement- A common programmer’s frustration in handling user in- ing [hidio]. We also discuss [input noticer] external that put is how to specify the device to open for communication. monitors device plug/unplug events, that works in concert A number of schemes exist for identifying devices, each with with [hidio] to allow a patch to be responsive to these its own advantages and disadvantages. The HID Toolkit events. We will discuss the design decisions that have led supports several methods for selecting devices: device num- to our current implementation, and will point out our plans ber, device type (gamepad, joystick, keyboard, etc.), and for future work on the HID toolkit. vendor ID/product ID. The device number is a unique nu- meric identifier assigned by the operating system. It is of- 2. PREVIOUS WORK ten useful behind the scenes in a selection menu, to select We are aware of a range of existing work, including some between multiple devices of the same type. However, of- HID access tools outside the Pd and Max/MSP environ- ten the specific model of the device is not as important as ments. However, Max’s [hi] and Pd’s [hid] are the most the general type. For example, a patch designed around a related to what the HID Toolkit is trying to accomplish, so mouse and joystick may work acceptably with any mouse we investigated these existing tools in more depth. and joystick. In these cases, the device type would be an appropriate way to select the first ‘mouse’ and a ‘joystick’ 2.1 Max’s [hi] found by the system. Vendor and product IDs and provide A number of objects exist within Max/MSP for the most detailed device information, but typically in a less getting data from HIDs, such as [hi], [hidin] [13], human-friendly format. These values can be used to select a [MouseState], [Insprock], [Wacom], [forcefeedback], particular device, even when there might be multiple devices and [MTCcentroid]. Each has a distinct programming of the same type plugged in. See figure 1 for more details. interface. [hi] is a good example for coherent integration [hidio] does not monitor HID plug/unplug events, because it provides a single interface for getting input data instead it builds a device list just before it tries to open a from many different kinds of HIDs. However, [hi] has device. The ability to monitor device insertion and removal limitations that impact its usefulness for creating and per- can be useful though, to avoid having to restart a patch forming with new instruments. For instance, it only checks if a device is removed and reconnected, or to mute sound for new devices on startup, and blocks other instances of if a control device is unplugged. The [input noticer] [hi] (and other programs) from accessing a connected object was designed to communicate HID plug/unplug device. Also, device elements like buttons and axes are events. It is currently implemented on GNU/Linux, and identified with numeric tags rather than semantically [input noticer] objects are created with a human-readable meaningful categorical identifiers. Finally, [hi] does not argument such as ‘Microsoft Sidewinder Dual Strike’ caus- support any HID output, making it incapable of controlling ing them to filter incoming messages for devices of that on-device feedback. model. In the future, [input noticer] will recognize iden- tical ways of specifying devices as [hidio] so that the same 2.2 Pd’s [hid] message can be used to specify devices in both [hidio] and Pd has a number of objects and patches for using [input noticer]. HIDs such as [MouseState], [linuxmouse], [linuxevent], [joystick], the Gem HID objects, and [hid] [15]. But, 3.3 Device Polling like the Max/MSP objects, these objects all have different Most USB mice have a poll interval of 10ms [12]. While interfaces, requiring a user to learn each object separately it is possible to get input reports more frequently, we be- in order to use the device it supports. We feel that since lieve that it would not be perceptible nor would it affect a large range of HIDs all share the same basic capabilities performance in a noticeable way.