J. lnf. Commun. Converg. Eng. 18(2): 115-122, Jun. 2020 Regular paper Low-Delay, Low-Power, and Real-Time Audio Remote Transmission System over Wi-Fi Jinwoo Hong1* , Jeongju Yoo2, and Jeongkyu Hong3, Member, KIICE 1Media Research Division, Electronics and Telecommunications Research Institute, Daejeon 34129, Republic of Korea 2Radio & Satellite Division, Electronics and Telecommunications Research Institute, Daejeon 34129, Republic of Korea 3Department of Computer Engineering, Yeungnam University, Gyeongsan 38541, Republic of Korea Abstract Audiovisual (AV) facilities such as TVs and signage are installed in various public places. However, audio cannot be used to prevent noise and interference from individuals, which results in a loss of concentration and understanding of AV content. To address this problem, a total technique for remotely listening to audio from audiovisual facilities with clean sound quality while maintaining video and lip-syncing through personal smart mobile devices is proposed in this paper. Through the experimental results, the proposed scheme has been verified to reduce system power consumption by 8% to 16% and provide real-time processing with a low latency of 120 ms. The system described in this paper will contribute to the activation of audio tele- hearing services as it is possible to provide audio remote services in various places, such as express buses, trains, wide-area and intercity buses, public waiting rooms, and various application services. Index Terms: Wi-Fi audio, Audio remote transmission, Low power consumption, Low-delay audio I. INTRODUCTION hearing is always open and recognizes sound information regardless of intention. Of course, it is possible to listen to a Many types of public transportation and places such as specific sound selectively among various sounds, but this has high-speed buses, trains, and waiting rooms have audiovisual several drawbacks, such as different sounds being heard (AV) facilities that play videos to relieve the boredom of together and requiring intense concentration which can tire users caused by long waiting and/or traveling times. How- the listener [1]. ever, their audio is very low or muted to allow for individual To provide a comfortable audio environment for users preference and restriction of noise, which interferes with nat- without creating excessive noise in public spaces, this paper ural listening. As a result, users cannot enjoy AV content proposes a low-power and low-delay remote audio transmis- properly because their concentration and understanding of sion system. The proposed technique converts the received the content is reduced. This situation is apparent especially analog audio (or digital audio) output from the AV facilities in hospital rooms with two or more patients, families with to digital audio, compresses it, and then transmits it with low students, crowded tourist spots, digital signs in public places, latency via Wi-Fi, thereby achieving real-time transmission. restaurants, and cafes. This allows individuals to remotely listen to the audio of the This is a result of hearing openness, one of the characteris- target video using their smart mobile devices such as smart- tics of the human hearing mechanism. Specifically, human phones, tablets, laptops, and smartwatches. Received 07 May 2020, Revised 22 June 2020, Accepted 22 June 2020 *Corresponding Author Jinwoo Hong (E-mail: [email protected], Tel: +82-42-860-6260) Media Research Division, Electronics and Telecommunications Research Institute, Daejeon 34129, Republic of Korea. https://doi.org/10.6109/jicce.2020.18.2.115 print ISSN: 2234-8255 online ISSN: 2234-8883 This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by- nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. Copyright ⓒ The Korea Institute of Information and Communication Engineering 115 J. lnf. Commun. Converg. Eng. 18(2): 115-122, Jun. 2020 The proposed Wi-Fi based audio remote transmission sys- tem provides real-time playback within a 120 ms latency to maintain synchronization between the video and audio played on the AV facilities. In addition, our system adopts data reliability technology to reduce memory power con- sumption and ensure high-performance computing. We tested our proposed system on 304 people by implementing it in a real environment and achieved an average score of 4.37 out of 5.0, which verifies the quality and convenience of the pro- Fig. 1. Audio remote transmission system diagram. posed system. Similar products, such as Bluetooth which uses radiofre- quency and wireless headphones, are available; however, hours per day, technical considerations to minimize they necessitate dedicated transmission and reception their power consumption should be investigated [7]. devices and do not allow multiple users to access the target 7) Up to 40 users should be able to send and receive AV facilities simultaneously [2]. Conversely, our system remote audio simultaneously. offers a service at a low price by using a smartphone as the 8) The system must provide a real-time protocol (RTP) receiving device, allows up to 40 users to access a single AV including a Wi-Fi access point (AP) function and for- facility concurrently, enables a specific facility device from ward error correction (FEC) function in the unicast and multiple devices to be selected, and can provide various broadcast mode [8-10]. additional services through the smartphone screen while Fig. 1 shows the proposed audio remote transmission sys- using our audio transmission service. tem, which satisfies all eight requirements described above, In this paper, Section 2 describes the system design and consisting of two modules. The first is an audio remote implementation issues of the proposed remote audio trans- transmission module (ARTM) which transmits the AV facil- mission system and Section 3 explains the various experi- ity’s audio output via Wi-Fi and the other is an audio play- mental results and verification details. Finally, conclusions back module (APM) which renders the received audio on are included in Section 4. users’ smart devices. Using a user’s existing smart device as an APM does not require dedicated hardware (HW) and can provide a remote audio transmission service using only the II. DESIGN AND IMPLEMENTATION OF AUDIO audio playback application and the device’s Wi-Fi capability. REMOTE TRANSMISSION SYSTEM B. ARTM Design and Implementation A. System Requirements and Concept An ARTM consists of a remote transmission controller to Several technical factors affect the way in which audio can support a transmission function as shown in Fig. 2. ARTM be listened to remotely from AV facilities. However, to apply converts analog audio signals from AV facilities into digital the most convenient and efficient method for the user and signals. The digital signals are then compressed in real time system requirements, we first derive the following character- and transmitted over Wi-Fi via an RTP-FEC stream. istics. Especially, the audio remote TX service controller consists 1) The audio quality at a remote position must be main- of five parts, as shown in Fig. 3. The functions of each part tained at 64 kbps or higher to satisfy the MP3 audio are shown in Table 1. In the ARTM, audio input is received standard [3]. through a sound device driver and the received data are com- 2) The maximum delay time should be less than 150 ms to pressed in a format suitable for transmission, which are then maintain lip-syncing between the video playback and remote audio of the AV facilities [4, 5]. 3) The audio remote service must be processed in real time without interruption of transmission and reception processing [6]. 4) By using the users’ smart devices, the system prevents users from incurring extra costs due to purchasing addi- tional hardware or audio playback software (SW). 5) The system provides a free audio playback application that uses a high-quality open audio codec. 6) Since the target facilities are usually operated for 24 Fig. 2. ARTM configuration diagram. https://doi.org/10.6109/jicce.2020.18.2.115 116 Low-Delay, Low-Power, and Real-Time Audio Remote Transmission System over Wi-Fi receiving audio from smart devices. C. APM Design and Implementation The APM functionality consists of checking for errors with RTP-FEC, restoring the signals received over Wi-Fi as Fig. 3. Audio remote transmission service controller. shown in Fig. 5, and then converting the signals from digital to analog and printing via SW operation. Table 1. Parts functions of the audio TX service controller The audio remote RX service controller in Fig. 5 includes Parts Major Function the functions of the four parts shown in Fig. 6. Table 2 lists Initialization/start/stop/restart control of audio Service Manager the functions of each part. remote transmission service SW, which operates on the APM, is implemented as apps Management of service target facility list using TCP that could be installed on both the iOS and Android plat- Session Manager socket connection/release over smart device in uni- forms and provides the ability to search for all APs of AV cast mode facilities installed in the vicinity, and to automatically set up Encoder type setting and control for compressing of Encoder I/F networks using a dynamic host configuration protocol audio data (DHCP). RTP Streamer setting and control, valid FEC setting Streamer I/F and packet generation according to Wi-Fi status in multicast mode D. Implementation of Audio Compression Technology Storage/management/transmission control for added Added Info. I/F information data To compress and restore audio to ARTM and APM, the Internet engineering task force (IETF) standardized (RFC 6716) and developed Opus codecs were used. Opus is an provided in the form of an appropriate library that can be open audio codec that is accessible to anyone at no extra used in the receiving application of the smart devices.