Audacity* Gets a Touch UI for Tablets and 2 in 1 PCs

Introduction This article previews some in-development features of Audacity, which is a free, simple, multi-track audio editor and recorder for Windows*, Mac* OS X*, GNU/*, and other operating systems. The prototype adds a touch user interface (UI), providing the operator a simplified screen with the most frequently used commands and tools. In addition to providing a useful UI for running Audacity on a tablet, this enhanced version also detects the transition between tablet and laptop mode in Ultrabook™ 2 in 1s, then adjusts the display appropriately. It also provides a manual method for transitioning the user interface between these two modes.

Figure 1. Video demo of Audacity 2-in-1 transition

Audacity in a 2 in 1 Audacity is an Open Source sound editing program with production-quality, audio engineering-level tools. It was created in 1999 by Carnegie-Mellon grad student Dominic Mazzoni and his advisor Roger Dannenberg and has been updated continuously ever since. Audacity has won numerous awards for its ease of use and powerful functionality including a BOSSIE (Best of Open Source Software) for sound editing and SourceForge Community Choice Awards for Best Project for Multimedia. An established PC program, Audacity’s user interface was designed around mouse and keyboard. With the popularity of touch-enabled PCs and Ultrabook 2 in 1s, Audacity’s UI was modified to take advantage of touch input and the 2 in 1s’ special capabilities. This article details the challenges and solutions discovered by software engineers in adding these capabilities to Audacity.

Figure 2. Audacity* Laptop User Interface (UI)

Many hobbyist-level musicians have learned the ropes of audio recording and editing using Audacity while sitting at their home computers. Navigating traditional keyboard and mouse controls on a laptop or desktop computer is a challenge while playing a guitar.

Figure 3. Guitarist using Audacity* laptop UI Running Audacity using a touch interface on an Ultrabook 2 in 1 would transform the experience.

The Audacity / Intel Connection Intel presented the idea of 2 in 1 awareness to a member of the Audacity Technical Leadership Council, Vaughan Johnson. Vaughan was so receptive to the idea that he used his 12 years of experience with Audacity to create the prototype used in the demos and images you see in this paper.

One of the most unique ways Audacity is being used today is as a recorder for ultrasonic frequencies; “sounds” that go well beyond the human ability to hear (20-20,000 Hz). A team studying porpoises is recording the sea mammal’s sound emissions accurately well above the human hearing spectrum. Another novel usage is a pyrotechnics engineer’s modification of the source code so he can control firework launches as his soundtrack plays through Audacity. Recordings made with Audacity produce a visual graph for human analysis—like hearing the sounds visually.

Vaughan believes over 200 current Audacity uses will need to be supported by the touch UI. Many of these uses make better sense when a tabletop is not available; which created a need for a more versatile touch-based interface. The power and performance of Intel®-based 2 in 1 computers helped Vaughan decide his first target platform for touch development. Tablets with less powerful processors don’t have the horsepower to run Audacity at a level that can support all those uses. The 2 in 1 technology provides Audacity with high quality audio on mobile devices without having to do a complete code rewrite, as would be required for other mobile OS platforms. Adding Touch The development team’s primary goal in modifying the user interface is simplification. Any screen element, be it a button, control, or status indicator, must pass scrutiny. Does the element support a simpler user experience? When a grandchild records her grandmother’s family stories, she is not an audio engineer, and the recording is best done at her grandmother’s convenience. So, what controls does a novice need on the screen to complete the capture successfully? In Vaughan’s own words, “Simplification—provide a basic recording device with ability to label important points / comments / crescendos / etc. by touch.”

Figure 4. Audacity’s* simplified touch UI

Audacity needs a lot of performance that many tablets today can’t provide. On 64-bit Windows 8, the minimum hardware requirements to run Audacity are 2GB of RAM and a 1 GHz processor. Audacity recommends much higher if the application is to be used for recording lengthy multi-track projects.

Figure 5. Musician using Audacity’s* 2 in 1 touch UI Remove the Unnecessary

First thing is to remove the familiar Windows title bar. It has elements that aren’t necessary in the touch mode, such as application pane resizing. The menu bar is also not needed in the touch UI. The usual File, Edit, View, and Help buttons on the top of nearly every Windows traditional program are not present. The Audacity-specific control buttons Transport, Tracks, Generate, Effect, and Analyze are missing as well. Below these top two lines the Audacity laptop UI is very specialized. The area just above and below the audio spectrum display has no fewer than 43 visual control elements by default with room for adding more via customization.

Figure 6. Simplified UI in tablet mode Figure 7. Full Control UI in laptop mode

Audio engineers love controls. Have you ever tried recording your 4-piece-band through a mixing board like the one below? Sound engineers prefer having an adjustment knob for every aspect of the soundscape.

Figure 8. Professional audio mixing board For the person wanting to capture that special story from grandma, they prefer controls closer to the ones below.

Figure 9. Mechanical recorder user interface

The tablet UI has removed all but three controls from the top of the screen (microphone and speaker levels plus the Settings wrench) and placed only 20 remaining controls on the touch screen. Gone are numerical displays such as the scale of the spectrum display and the length of the selection on the bottom. They removed the information for file/encoding type, for audio hardware currently active, and the sampling rate of the project. These controls are still very relevant to the work Audacity does, however, they were just deemed less necessary when designing a simplified UI. These elements can still be accessed in the desktop UI.

Keep Only the Must-Haves

Audacity has grouped actions together that make sense, with a focus on ease-of-use. As you can see in the prototype, Audacity kept the audio spectrum graph largely unchanged, while altering the anchor/left side of each graph so it was touch friendly. A smaller number of channels are displayed in the simplified tablet interface. To add a track, the user only has to touch the + sign icon under current tracks, replacing the menu-bar nesting or hot-key combination Ctrl-Shift-N.

Figure 10. Adding a track - touch UI

Figure 11. Adding a track - laptop UI

Icons, like a speaker instead of the mute button and headphone instead of the solo button, replace less efficient elements.

Figure 12. Touch UI (left) ←Mute & Solo Controls → Laptop UI (right)

The most drastic changes in the touch UI are seen on the edges of the display. Where the laptop UI has a top to bottom flow, the touch UI has moved all the controls to the edge. In the prototype, transport controls (, Play, Pause, etc.) have been moved to the left and made larger for easy thumb access. Added to these are the Undo and Redo buttons. The last portion of the screen to be populated is the bottom. The remainder of the touch-enabled functions are at the bottom of the screen instead of the right-thumb favored right side to leave more screen real estate for left to right waveform display. If more detailed control is required, users can manually transform to the laptop interface.

Figure 13. Touch and Laptop UIs side by side What’s behind the curtain?

To create the new UI, Vaughan Johnson primarily uses Microsoft Visual 2008 on Windows. He also relies heavily on the body of development work the Audacity team built over the last 14 years including many cross-platform libraries. Vaughan is currently planning on porting the easy internal Windows standard controls as implemented by wxWidgets for touch UI. While the first of these descendant classes has been written, each subsequent layer is more complex and therefore needs to branch to simpler versions. Summary The task of adding a second UI to a popular, 14-year-old application like Audacity and its established user base will take time and effort. These features aren’t available for download today, but the work continues. If you would like to help or encourage the task, let your voice be heard at audacity.org. Better yet, join Vaughan, Roger, and many other developers at the Audacity Unconference in Preston England July 11th and 12th 2014!

Links: Audacity Home Page Audacity Wiki Page Audacity Awards Porpoise recording project Firework control project Audacity cross-platform libraries Audacity Unconference 2 in 1 information page on Intel.com Audacity 2-in-1 transition video demo

Additional information: Detecting Slate/Clamshell Mode & Screen Orientation in a 2 in 1 Device by Seung-Woo Kim Gameplay: Touch Controls and 2-in-1 Awareness for Your Favorite Games by Erik Niemeyer Mixing Stylus and Touch Input on Windows* 8 by Meghana Rao How to Write 2 in 1 Aware Applications by Stevan Rogers Krita Gemini 2 in 1 UI Change demonstration Video on IDZ or YouTube*

About the Author Tim Duncan is an Intel Engineer and is described by friends as “Mr. Gidget-Gadget.” Currently helping developers integrate technology into solutions, Tim has decades of industry experience, from chip manufacturing to systems integration. Find him on the Intel® Developer Zone as Tim Duncan (Intel).

Notices

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

UNLESS OTHERWISE AGREED IN WRITING BY INTEL, THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR ANY APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR DEATH MAY OCCUR.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.

The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.

Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.

Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or go to: http://www.intel.com/design/literature.htm

Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark* and MobileMark*, are measured using specific computer systems, components, software, operations, and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.

Any software source code reprinted in this document is furnished under a and may only be used or copied in accordance with the terms of that license.

Intel, the Intel logo, and Ultrabook are trademarks of Intel Corporation in the U.S. and/or other countries.

Copyright © 2014 Intel Corporation. All rights reserved.

*Other names and brands may be claimed as the property of others.