NHK Launched 4K and 8K Channels!

Home , NHK

NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 1 Science & Technology Research Laboratories NHK TECHNOLOGY

No.76 SPRING 2019 ISSN 1345-4099

NHK launched 4K and 8K channels!

NHK finally launched the satellite broadcasting of 4K and 8K Super Hi-Vision on December 1, 2018. The channels are called “NHK BS4K” and “NHK BS8K”. The launch of 8K channel is the world’s first, for which NHK Science & Technol- ogy Research Laboratories (STRL) has been leading the R&D and standardization for more than 20 years. After the count- down, a test color-bar pattern on the 8K TV screen was replaced at exactly 10 am with brand-new ultra-high definition 8K images along with immersive 22.2 multichannel sound. To promote the two new channels, a special event “Broadcasting Starts on December 1st! 4K/8K Super Hi-Vision Park" was held at an event hall in Shibuya, Tokyo from November 30 to December 4. NHK STRL brought “8K Living Room” and “8K×22.2ch Acoustic Home Theater Experience” exhibitions to the event —which provided a viewing experience of 8K broadcasts in a home setting. Also, on December 1st, NHK STRL held an event at the entrance hall of STRL building to give a demonstration of receiving the BS4K and BS8K channels. Many viewers gathered and enjoyed a special opening program, which includes live feed from Roman Forum and

Pantheon in Rome, Italy, and a remastered 8K version of the classic 4K/8K satellite broadcast reception demo at NHK movie “2001: A Space Odyssey.” STRL (December 1, 2018) NHK BS8K is now being broadcast for about 12 hours per day. FROM THE EDITORS A wide range of programs, such as entertainment, art, documentary and sports, are covered, which are exclusively produced for the 8K channel in the world’s highest quality, providing faithful represen- tations of the scenes and artifacts and highly immersive experienc- es to the viewers. NHK BS4K, which NHK defines as a gateway channel for ultra-high definition visual images, basically provides familiar programs to HD channels in true 4K quality for 18 hours per day. Other major TV networks in Japan have also launched 4K channels, and we expect 4K and 8K Super Hi-Vision viewing will become widespread around 2020 in Japan. 4K/8K Super Hi-Vision Park at Shibuya, Tokyo

We provide various information about our research activities, events, annual reports, etc. on our website. Please have a look and send us your opinions. Contents www.nhk.or.jp/strl/index-e.html Topic -NHK launched 4K and 8K channels! 1 R&D -3D Imaging Technology: 22 -NHK’s latest technology was exhibited at Inter BEE 2018 2 Achieving High Resolution with a Camera Array

Feature -Research and Development for Advanced 3 Terrestrial Broadcasting -Transmission System for Advanced Digital 10 NHK Science & Technology Research Laboratories Bulletin Terrestrial Television Broadcasting Journals 23 Kohji MITANI, publisher, Masaru MIYAZAKI, editor -Hybridcast Connect Library: 20 R & D Utilizing New Image Expression Technology © NHK Science & Technology Research Laboratories Director of STRL Yoshiki NAKAJIMA, editor For Easy Integration of TVs and Smartphones NHK Technology 24 for Sports Broadcasts Address: 1-10-11, Kinuta, Setagaya-ku, Tokyo, 157-8510, Japan Editors Keiji ISHII, editor-in-chief Ikuko SAWAYA, editor -Toward the realization of a Bendable Display: 21 Kimihiro TOMIYAMA, editor Yuki OGAWA, editor Phone: +81(0)3-3465-1111 Fax : +81(0)3-5494-3125 Extending the Lifetime Hirokazu KAMODA, editor From the Editors 24 https://www.nhk.or.jp/strl/index-e.html using an Inverted-Structure Organic EL Device 2 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

TOPIC FTOPIC

NHK’s latest technology was exhibited at Inter BEE 2018

At the broadcasting-equipment exhibition “Inter BEE 2018” held at Makuhari Messe (Chiba City, Japan) from November 14 to 16, 2018, NHK exhibited new viewing equipment and program-production technology for 4K/8K satellite broadcasting, as well as Integrated Broadcast-Broadband technology and technology for supporting program production by making use of AI. At our booth, shared with the Japan Electronics and Information Technology Industries Association (JEITA), we introduced equipment such as a television and external tuner for receiving new 4K/8K satellite broadcasts (which started on December 1, 2018) while exhibiting an “8K Living-room Theater”—presenting 8K images for viewing at home—that enabled visitors to the booth to experience ultrahigh-definition, realistic images with 22.2 multichannel three-dimensional acoustics. In addition, we introduced the content-creation technology of the new era; demonstrating the integrated production of 2K and 4K broadcasts with 4K OB van, and “transmission technology dividing 8K flow into multiple 2K flows,” for implementing equipment that can be used for the production of 2K, 4K, and 8K broadcasts. In the area called “INTER BEE CONNECTED”, where the latest trends related to the convergence of broadcasting and broadband were introduced, we exhibited a basic technology called “Hybridcast Connect Library” for linking, for example, broadcasting and smartphone applications. We also introduced examples of new services generated with these technologies. In the area called “INTER BEE IGNITION”, where the possibilities of new media utilizing the latest video technology were introduced, we exhibited a “automatic video summarization system”, which uses social-media information and video-analysis technology, and “automatic colorization technology for monochrome video”, which uses AI to colorize images of black-and-white film. Here, visitors could see for themselves new technologies for supporting program production utilizing the latest video-processing technology and AI technology. From now onwards, we will continue to further pursue such efforts described above to introduce NHK’s latest broadcasting technologies to as many people as possible.

A scene at the NHK/JEITA booth Exhibition of Hybridcast Connect Library NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 3

FEATURETOPIC FEATUREFTOPIC

Research and Development for Advanced Terrestrial Broadcasting

Kenichi Tsuchida

About 15 years have passed since the start of digital tion terrestrial broadcasting, our research and development terrestrial television broadcasting in Japan in December towards next-generation advanced terrestrial broadcasting 2003. Research and development on advanced terrestrial is explained. broadcasting technology for realizing next-generation terrestrial broadcasting, such as terrestrial “Super Hi-Vision” 1. Trends in next-generation terrestrial broadcasting (4K/8K) broadcasting, is progressing. About 15 and 20 years have respectively passed since In our development of a next-generation terrestrial the start of digital terrestrial television broadcasting in Ja- broadcasting system, while we are inheriting the features pan and in Europe and the United States, and at this point of ISDB-T (Integrated Services Digital Broadcasting-Ter- in time, global widespread adoption of next-generation restrial), which is the current transmission system for terrestrial broadcasting is being considered. The digital digital terrestrial television broadcasting, we aim to create terrestrial television broadcasting systems1)2) widely used a system that adopts the latest technology, has higher fre- throughout the world are ISDB-T, DVB-T (Digital Video quency-utilization efficiency, is capable of large-capacity Broadcasting – Terrestrial), ATSC (Advanced Television transmission, and has excellent transmission robustness. Systems Committee), and DTMB (Digital Terrestrial We have been advancing research to increase the capacity Multimedia Broadcast), which were developed in Japan, of terrestrial broadcasting, developed a transmission meth- Europe, the USA, and China, respectively. In recent years, od that can simultaneously provide Super Hi-Vision broad- technical specifications for next-generation systems have casting and mobile broadcasting services on one channel, been studied, and some countries have already put them to and produced experimental equipment on a trial basis. In practical use. this report, considering the global trends in next-genera- In Europe, the standardization of DVB-T2, which was

Table 1: Comparison of next-generation terrestrial digital television broadcasting systems

DVB-T2 ATSC3.0 advanced ISDB-T ISDB-T (for reference) (Europe) (USA) (Japan) (Japan)

Basic signal structure TDM TDM/LDM FDM/LDM FDM

Error correction code LDPC + BCH LDPC + BCH LDPC + BCH Convolution + RS

Carrier modulation QPSK - 256QAM QPSK - 4096QAM QPSK - 4096QAM QPSK - 64QAM scheme NUC*1 compatible NUC compatible

Multiplexing method TS-based IP-based IP-based Compatible with TS only (Baseband interface)

Control signal Preamble*2 Preamble TMCC*3 TMCC

Hierarchical transmis- · Although methods · For terrestrial 4K · Hierarchical transmis- · Results available sion with mobile recep- exist, no examples of broadcasting in South sion using partial reception practical use including Korea, operation starting tion is implemented. DVB-T exist. with 16k FFT is used for · Characteristics can mobile service and 32k be improved by partial FFT for fixed service. power increase.

Stabilized broad- · Since signal processing for moving and fixed · Inherit the past results · Equalization devices cast-wave relay symbols differs, in the equalizer, interpretation of the of the equalizer using an using FDM structure control signal is required, so device cost increases. FDM structure. have a proven track record.

*1 Non-uniform constellation *2 A signal transmitted at the beginning of a frame. It is used for frame synchronization and transmission of control signals. *3 Transmission and multiplexing configuration control 4 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

FEATURE FEATURE

Based on LTE Based on LTE Advanced Pro standard 5G broadcast mode LTE broadcast mode LTE broadcast mode Release 16 or later Release 9 (2009) Release 14 (2017) (as from 2019)

Communication use 1 frame (10 ms) 1 frame (10 ms) Broadcasting Broadcasting use use

Effective symbol Effective symbol (800 µs) GI (16.7µsec) (66.7µs) GI (200µsec)

• 60% signal area can be allocated for • The entire signal can be used for Assumed specifications broadcasting broadcasting • Compatibility for major areas • Maximum GI length: • GI length up to 200 µs Suitable • GI length: up to 200 µs (including control signal) 16.7 µs only for small cells for large cells • Error correction code: LDPC *2 • Receiver requires a communication (However, GI length of the control • Increase in capacity by MIMO in addition to *1 contract (SIM card) signal is 16.7 µs) SISO • Receiver does not require a • Frame structure based on NR*3 communication contract (SIM card) (scheduled for Release 17 or later)

*1 Single input and single output *2 Multiple input and multiple output *3 New Radio: new wireless communication standard for 5G. Figure 1: Transition of broadcast mode of wireless communication standards

developed from DVB-T, was completed in April 2009, and tion to 4K broadcasts using DVB-T2 and HEVC (high-ef- regular broadcasting began in the UK in December 2009. ficiency video coding) - targeting the Paris Olympic Like DVB-T, DVB-T2 uses OFDM (orthogonal frequen- Games in 2024 - have started. cy-division multiplexing) as a digital modulation scheme. In the United States, standardization work of the Regarding the error correction code, by using concatenat- next-generation terrestrial broadcasting system ATSC3.0 ed codes, namely, the LDPC (low-density parity check) started in 2012. Basic standardization was completed by code and BCH (Bose–Chaudhuri–Hocquenghem) code, June 2017, and the first edition of the technical specifi- and using 256QAM (quadrature amplitude modulation) for cation was released in January 2018. The multiplexing the carrier modulation scheme, it is possible to improve scheme used is IP (Internet Protocol) taken from the con- transmission capacity by about 45% compared with that ventional MPEG-2 (Moving Picture Experts Group-2) possible with DVB-T (compared with transmission param- TS (Transport Stream) standard, and the error correction eters used in current broadcasting). codes are concatenated codes of LDPC and BCH codes. In the UK and Sweden, broadcasting uses DVB-T and Moreover, the carrier modulation (OFDM) schemes used DVB-T2 in combination, and multiple SD (standard-defi- cover QPSK to 4096QAM. nition) programs are broadcast on the DVB-T channel, Broadcasting using ATSC3.0 started in Seoul, South while multiple HD (high-definition) programs are broad- Korea, in May 2017, expanded to some major cities of cast on the DVB-T2 channel. In the UK, although demon- Korea in December 2017, and will be expanded to cover stration experiments on terrestrial 4K broadcasts were the whole country by 2021. 4K programs are included in conducted around 2014, since then such broadcasts have the broadcasts, and the composition ratio of 4K programs not been considered in a concrete manner. is scheduled to be gradually increased from 5% in 2017 to In Germany, since March 2017, efforts to complete the 10% in 2018 and 15% in 2019. transition from DVB-T to DVB-T2 have been underway, In Japan, an advanced version of ISDB-T (hereafter and broadcasting of HD programs by using DVB-T2 has referred to as “advanced ISDB-T”) is being studied. The started in some areas. In Italy, the transition from DVB-T study on this system is explained in detail in Section 3. to DVB-T2 will start from January 2020. In France, pro- The above-mentioned, next-generation digital terrestrial grams with SD/HD picture quality are currently being television broadcasting systems are compared in Table 1. broadcast by DVB-T; however, discussions on the transi- As the signal structure, TDM (time-division multiplexing) NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 5

FEATURE FEATURE

is used for DVB-T2 and ATSC3.0, and FDM (frequen- 3GPP meeting in June 2018, the 5G broadcast mode han- cy-division multiplexing) is used for advanced ISDB-T. dling the downlink only (under the assumption of a fixed In addition, for both ATSC3.0 and advanced ISDB-T, the service area and bit rate) was agreed as a study item, and signal structure can handle LDM (layered division mul- study to formulate the technical specification of Release tiplexing), which multiplexes two layers with different 16 will start at the end of 2019. powers. Regarding error correction codes, in all systems, transmission efficiency is increased by using concatenat- 3. Development of next-generation terrestri- ed codes of LDPC and BCH codes. With regard to the al-broadcasting systems carrier modulation scheme, DVB-T2 can handle up to We have also promoted R&D aimed at advanced ter- 256QAM, ATSC3.0 and advanced ISDB-T can handle up restrial broadcasting. In 2007, we began R&D aimed at to 4096QAM, those multi-level modulations enable the increasing the capacity of terrestrial broadcasting, and increase in the transmission capacity. we have been developing high-order modulation OFDM and polarization MIMO (multiple-input multiple-output) 2. Approach to broadcast mode taken by 3GPP technology3). In 2013, we undertook a research project (3rd-Generation Partnership Project)*1 outsourced by the Ministry of Internal Affairs and Com- As well as approaches based on broadcasting systems, munications titled “Research and Development of Basic approaches based on communication systems can be con- Technology Encouraging Effective Utilization of Frequen- sidered with next-generation terrestrial broadcasting sys- cy for the Next Generation Broadcasting System.” and in tems. In particular, in Europe, sometimes the infrastructure January 2014, we achieved 8K long-distance transmission for broadcasting and communication is operated by the (with a transmission capacity of 91.8 Mbps over a trans- same company, and the “Broadcast mode”*2 based on mission distance of 27 km) in the Hitoyoshi district of wireless communication standards for 4G and 5G is being Kumamoto Prefecture4). From 2014 to 2016, we were en- studied. The transition of the broadcast mode of wireless trusted with a project called “Research and Development communication standards is shown in Fig. 1. of Technology Encouraging Effective Utilization of Fre- For LTE (Long-Term Evolution) broadcast mode Re- quency for Ultra High Definition Satellite and Terrestrial lease 9 launched in 2009, although the MBMS (Multimedia Broadcasting.” During that period, we conducted exper- Broadcast and Multicast Service) specification (for simul- iments on technology for effective frequency utilization; taneous broadcasts based on LTE) was standardized, the for example, we performed experiments on a coded SFN time rate of the downlink that can be used for broadcasting (single-frequency network), and we studied transmission was restricted. Furthermore, other restrictions, namely, characteristics of high-order modulation OFDM5). short guard-interval (GI) length and small cells (reception Considering the results of these investigations, we have areas) in a similar manner to mobile-phone base stations, been studying provisional specifications aimed at future are imposed, and no examples actually linked to services terrestrial broadcasting systems. In particular, we aim to have been reported. boost capacity by incorporating ISDB-T-based technolo- In Release 14 launched in 2017, some functions of gies in these specifications6). MBMS were revised on the basis of the completed tech- Since 2016, NHK, receiver manufacturers, a university, nical specification LTE Advanced Pro. The whole signal and another organization have been entrusted with the re- was made available for broadcasting and the maximum GI search project “Research and Development for Advanced length became 200 μs, so the idea of incorporating a large Digital Terrestrial TV Broadcasting System”7), and we are cell for broadcasting was considered; however, problems working in cooperation with them on developing advanced such as the small GI length of the control signal (called the technologies for terrestrial broadcasting. “cell acquisition subframe”) remained to be solved. At the The research items (“technologies” hereafter) stated in “Research and Development for Advanced Digital Terres- *1 An organization in which seven worldwide communication standard- trial TV Broadcasting System” are shown in Fig. 2. The ization bodies gather and discuss next-generation communication goal of this R&D is to further promote the utilization of standards. From Japan, the ARIB (Association of Radio Industries and Businesses) and the TTC (Telecommunication Technology Com- radio waves and establish technologies that enable services mittee) are participating members. including ultrahigh-definition terrestrial broadcasting such *2 Based on wireless communication standards such as LTE, 4G, and 5G, a mode specialized for one-way unidirectional transmission as 4K and 8K (Super Hi-Vision broadcasting) by improv- (broadcasting). ing transmission efficiency while inheriting the features 6 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

FEATURE FEATURE

Achieving more effective use of radio waves and establishing technologies that enable services such as ultrahigh-definition terrestrial broad- Purpose casting by inheriting the features of the current terrestrial television broadcasting system and by improving transmission efficiency.

Implementation period Fiscal year 2016 - 2018 (three years)

NHK (Japan Broadcasting Corporation), Sony Corporation, Panasonic Research institutes Corporation, Tokyo University of Science, NHK Integrated Technology Inc.

Technology A: Advanced technology Technology B: Advanced technology for terrestrial broadcasting for mobile services

• Improve transmission efficiency, develop a transmission scheme*1 • Evaluate the mobile reception characteristics of the transmission and a video-coding scheme*2 that can simultaneously provide method developed via Technology A and develop technology for 4K/8K and mobile services on one channel, and prototype the improving reception. developed equipment. *1 High-order modulation, error correction code, etc. *2 Noise elimination, band-limited HEVC, etc.

Technology C: Transmission technology Technology D: Relay technology based on SFNs for large-scale stations compliant with advanced ISDB-T

• Establish a large-scale station and evaluate the transmission • Develop technologies for synchronizing transmission waveforms characteristics of the schemes developed via Technology A by from multiple transmission stations so that SFN can be created field experiments. with IP signals. • Establish SFN experimental stations and evaluatetransmission Implemented in Tokyo area characteristics by field experiments.

Implemented in Nagoya area

Figure 2: Research and Development for Advanced Digital Terrestrial television Broadcasting System

of the current digital terrestrial television broadcasting. studying—that can simultaneously provide Super Hi-Vi- The research is split into four technologies: “A: advanced sion and mobile services on one channel and (ii) a vid- technology for terrestrial broadcasting,” “B: advanced eo-coding scheme that improves coding efficiency, and technology for mobile services,” “C: transmission tech- we have prototyped equipment utilizing these schemes. To nology for large-scale stations,” and “D: relay technology improve transmission efficiency, we are using high-order based on SFNs compliant with advanced ISDB-T”. NHK modulation technology, extending the signal bandwidth is focused on addressing technologies A, C, and D. and FFT (fast Fourier transform) size*3, and introducing For technology A (advanced technology for terrestrial

broadcasting), we developed (i) a transmission scheme— *3 Number of FFT samples used for OFDM signal modulation/demod- based on the provisional specifications we have been ulation processing. NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 7

FEATURE FEATURE

new error correction codes8). In addition, we are studying pliant with advanced ISDB-T), we are developing relay tech- technologies for eliminating coding noise and limiting the nology based on SFNs (which broadcast radio waves with video band to improve video-coding efficiency, which are the same frequency from multiple transmitting stations). important tasks in regard to advanced ISDB-T (which has To create SFNs using IP signals, we have started to de- smaller transmission capacity than satellite broadcasting)9). velop a technology for synchronizing waveforms transmit- For technology C (transmission technology for large- ted from multiple transmitting stations, develop SFN ex- scale stations), we plan to evaluate the transmission speci- perimental testing stations, and evaluate their transmission fications of large-scale experimental test stations, improve characteristics in field experiments in the autumn of 2018. these facilities, and evaluate the transmission characteris- In this way, regarding the development of the advanced tics in field experiments10). Moreover, while extending the ISDB-T, we are targeting a system that adopts the latest signal bandwidth, we are also investigating sharing condi- technology while inheriting the features of ISDB-T; as tions with ISDB-T11). a result, it will provide superior transmission robustness For technology D (relay technology based on SFNs com- while enabling higher-capacity transmission and higher

Table 2: Specifications of transmission by experimental stations

Tokyo Area Nagoya area

Nabeta (relay station) Shiba Higashiyama (main station) Transmission location (NHK Nabeta Radio Transmit- (Tokyo Tower) (Chukyo TV Tower) ting Station)

563.143 MHz, 563.210 MHz Transmission frequency 605.143 MHz, 605.210 MHz (UHF 35ch) (UHF 28ch)

Polarization Horizontal, vertical (Dual-polarized MIMO)

Transmission power 1 kW 1 kW 10 W (horizontal/vertical)

Effective radiated power (ERP) 2.1 kW 980 W 74 W (horizontal/vertical)

Transmission height above sea 280 m 203 m 42.5 m level

Higashiyama Shiba

Nabeta

Higashiyama Nabeta antenna1 Nabeta antenna2

Tokyo area Nagoya area (area with electric field strength of 60 dBµV/m or more) (area with electric field strength of 60 dBµV/m or more) Figure 3: Experimental stations and area guidelines 8 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

FEATURE FEATURE

frequency-utilization efficiency. Moreover, toward the ISDB-T being developed in Japan by NHK and other or- development of a next-generation terrestrial broadcasting ganizations, and large-scale experimental testing stations system in Japan, in addition to developing the above-de- were outlined. scribed advanced ISDB-T, we are also investigating meth- The author hopes readers will get an understanding of ods of multiplexing new signals with those of the current the various initiatives aiming to create a terrestrial-broad- digital terrestrial television broadcasts and aiming to cre- cast advancement system that can simultaneously provide ate terrestrial 4K broadcasting12). Super Hi-Vision broadcasting and mobile broadcasting services. 4. Experimental testing stations to evaluate advanced ISDB-T Acknowledgments: The experimental testing stations are The large-scale experimental testing stations for carrying being developed with the cooperation of Nippon Televi- out these experiments are outlined as follows. sion Tower Co., Ltd., and Chukyo Television Broadcasting To evaluate the transmission scheme in various receiv- Corporation. In addition, a part of this research is being ing environments, the radio waves of the experimental performed under the auspices of the Ministry of Internal testing stations must reach a wide range, so we established Affairs and Communications, Japan as part of its program transmission specifications targeting transmission power titled “Research and Development for Advanced Digi- as the current key station. The transmission specifications tal Terrestrial TV Broadcasting System”, and the author of the experimental testing stations in the Tokyo and Na- thanks everyone concerned. goya areas are listed in Table 2. In the Tokyo area, the key-station-scale transmission technology was evaluated, References and transmission tests were executed in an urban area. In the Nagoya area, a transmission network was established 1) Rec. ITU-R BT.1306-7, “Error Correction, Data Framing, by setting up a key station and a relay station, the synchro- Modulation and Emission Methods for Digital Terrestrial nization technology was verified by using the IP between Television Broadcasting” (2015) transmitting stations, and transmission experiments were 2) Rec. ITU-R BT.1877-1, “Errorcorrection, Data Framing, executed under an SFN environment. Modulation and Emission Methods for Second Generation of The transmission center frequency can be selected from Digital Terrestrial Television Broadcasting Systems” (2012) the following two types: a frequency with an offset of 0.143 3) K. Murayama, M. Taguchi, T. Shitomi, H. Hamazumi, and K. MHz (similar to that of the current digital terrestrial tele- Shibuya: “Technology for the next generation of digital ter- vision broadcasting) or a frequency with an offset of 0.21 restrial broadcasting -Toward digital terrestrial broadcasting MHz so that the balance between upper adjacent channel of Super Hi-Vision-,” ITE Technical Report, BCT 2010-67, interference and lower adjacent channel interference is pp. 37-40 (2010) (in Japanese) satisfactory (according to the results of interference exper- 4) S. Saito, T. Shitomi, S. Asakura, A. Satou, M. Okano, K. Mu- iments in which the signal bandwidth was extended). rayama and K.Tsuchida: “8K Terrestrial Transmission Field Rough estimates of the experimental test areas (i.e., the Tests Using Dual-Polarized MIMO and Higher-Order Mod- range in which the field strength is 60 dBμV/m or more) ulation OFDM,” IEEE Transactions on Broadcasting,Vol.62, calculated on the basis of the transmission specifications No.1, Pt.2, pp.306-315 (2016) listed in Table 2 are shown in Fig. 3. In these test areas, 5) S. Saito, T. Shitomi, S. Asakura, A. Sato, M. Okano, and K. transmission characteristics in various receiving environ- Tsuchida: “Advanced SFN field experiment for 8K transmis- ments, such as urban areas, suburbs, and highways, can sion at Hitoyoshi City, ”33 D-2 (2015) (in Japanese) be evaluated. Transmission specifications of the advanced 6) T. Takeuchi, A. Sato, H. Miyasaka, S. Asakura, T. Shitomi, S. ISDB-T are described in the subsequent article “Transmis- Saito, Y. Narikiyo, M. Nakamura, K. Murayama, M. Okano, K. sion system for advanced digital TV broadcasting” and the Tsuchida, and K. Shibuya: “Study of Proposed Specification results of the experiments will soon be reported at interna- for the Next Generation Terrestrial Broadcasting,” Proceedings tional conferences. of the ITE Annual Convention, 31 A-1 (2016) (in Japanese) 7) Ministry of Internal Affairs and Communications: “Public 5. Concluding remarks Offering of Research and Development Proposals for Radio- In this article, global trends towards the advancement of wave Resource Expansion in FY2016 (Attachment 5),” http:// terrestrial broadcasting were described, and the advanced www.soumu.go.jp/main_content/000404772.pdf (in Japanese) NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 9

FEATURE FEATURE

8) K. Tsuchida: “Research and Development aimed at UHD Digital Terrestrial Broadcasting –Expanding Transmission Capacity and Effective Utilization of Frequency–,” ITE Technical Report, Vol. 41, No. 43, BCT 2017-95, pp. 47-54 (2017) (in Japanese) 9) Y. Matsuo, K. Iguchi, and K. Kanda: “Coding Efficiency Improvement by Band-Limitation Equipment for Advanced Digital Terrestrial TV Broadcasting System” ITE Winter An- nual Convention, 14 C-3 (2017) (in Japanese) 10) M. Okano, A. Sato, N. Shirai, M. Nakamura, Y. Narikiyo, M. Nakamura, K. Suzuki, N. Sato, R. Watanabe, K. Tsuchida, and S. Nakahara: “Construction of Large-scale experimental environment for evaluating Advanced Digital Terrestrial TV Broadcasting System,” ITE Technical Report, Vol. 42 , No. 11, BCT 2018-49, pp. 47-50 (2018) (in Japanese) 11) N. Shirai, A. Sato, Y. Narikiyo, M. Okano, and K. Tsuchida: “A study on Bandwidth Extension for the Proposed Specifi- cation of the Next Generation Terrestrial Broadcasting,” ITE Technical Report, Vol. 42, No. 11, BCT 2018-48, pp. 43-46 (2018) (in Japanese) 12) Ministry of Internal Affairs and Communications: “Study on Effective Utilization of Frequency Focused on the Future of Broadcasting Services Subcommittee (4th), Document 4-2, Research and Development on Advancement of Broadcast- ing,” (in Japanese) http://www.soumu.go.jp/main_content/000539299.pdf 10 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

FEATURE FEATURE

Transmission System for Advanced Digital Terrestrial Television Broadcasting

Takuya Shitomi

With the aim of improving the quality and expanding the leaving, we can make ISDB-T robust against (i) multipath functions of digital terrestrial television broadcasting ser- interference waves*1 generated in the propagation path in vices, we have been developing an advanced transmission the case of terrestrial broadcasting and (ii) fluctuation of system that inherits key features of the current ISDB-T the received electric field strength in the case of mobile (Integrated Services Digital Broadcasting - Terrestrial) reception. Furthermore, it is compatible with the MPEG- system, which employs hierarchical transmission based on 2 (Moving Picture Experts Group 2) systems adopted for a segment structure. The advanced digital terrestrial televi- multiplexing video, audio, data, etc., constituting BS and sion broadcasting system has a new signal frame structure CS digital broadcast programs, so it is said to have high that enables flexible bandwidth allocation to multiple ser- interoperability. vices for different reception scenarios, such as fixed recep- At NHK, aiming to improve the functionality of digital tion and mobile reception compared with the ISDB-T sys- terrestrial television broadcasting and attain higher-quality tem. In addition, by introducing transmission technologies video and audio, we have been developing transmission such as the latest error correction code and modulation technologies inheriting the above-described features of the scheme, this specification has high frequency usage effi- ISDB-T scheme while considering the specifications of ciency and transmission robustness; i.e., the transmission the next-generation terrestrial-broadcasting transmission capacity increases by about 10 Mbps for the same required scheme (hereafter referred to as “advanced ISDB-T”)1). carrier to noise ratio (CNR) in comparison with the cur- When we formulated the specifications, we stipulated a rent ISDB-T system, or the required CNR can be reduced service requirement that Super Hi-Vision broadcasting by about 7 dB for the same transmission capacity. In this for fixed reception and high-definition broadcasting for paper, we describe an outline of the transmission system mobile reception can be simultaneously provided on one being studied for advanced terrestrial broadcasting. channel because the radio frequency spectrum is scarce in Japan. To satisfy this requirement, we introduced a new 1. Introduction signal structure and the latest technology for improving The current transmission scheme for digital terrestrial transmission characteristics while keeping in mind that TV broadcasting in Japan was first reported by the Tele- increasing the transmission capacity (bit rate) per unit communications Technology Council in May 1999, and it frequency (to increase the frequency-utilization efficien- was established as an ARIB (Association of Radio Indus- cy) and that any devised scheme must achieve excellent tries and Businesses) standard in May 2001. In the sense transmission robustness. Supposing that the MMT (MPEG of terrestrial integrated digital broadcasting, this scheme is Media Transport)*2 and TLV (type-length-value)*3 adopted called ISDB-T (Integrated Services Digital Broadcasting - for the 4K/8K satellite broadcasting, which started in De- Terrestrial). cember 2018, are used as the multiplexing schemes2), we A feature of the ISDB-T scheme—which handles differ- investigated the structure of the block of the error-correc- ent receiving modes (such as fixed reception and mobile tion code and the frame structure. In this report, the basic reception)—is that it can provide multiple services with configuration of the specifications of the physical layer of different transmission capacities and transmission robust- the advanced ISDB-T is outlined. nesses on one channel (6 MHz) of terrestrial broadcast signals, and currently, many broadcasting stations simul- *1 In addition to radio waves coming directly from the transmitter, ra- taneously provide high-definition broadcasts for fixed re- dio waves arrive via reflection by buildings, mountains, etc. ception and one-segment broadcasts for mobile reception. *2 A successor multiplexing technology of MPEG-2 systems, which was internationally standardized in 2014. By utilizing orthogonal frequency division multiplexing *3 A framework for multiplexing variable-length packets such as IP (In- (OFDM) as the modulation scheme as well as time inter- ternet Protocol) packets. NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 11

FEATURE FEATURE

2. Overview of advanced ISDB-T the functions, such as hierarchical transmission and partial reception, of ISDB-T. That is, the segment structure 2-1 Inheriting the functions (such as hierarchical trans- was adopted so that partial reception was possible in a mission) of the ISDB-T scheme manner that allows transmission from three layers (layers The current ISDB-T scheme utilizes OFDM, which has A, B, and C) with different transmission capacities and excellent multipath tolerance, as a modulation scheme and transmission robustnesses. Hierarchical transmission and adopts a segment structure*4 capable of partial reception*5. partial reception by the advanced ISDB-T are illustrated in By configuring the carriers of OFDM segments so that Fig. 1. For the advanced ISDB-T, the number of divisions multiple segments can be linked, we can set a transmission of the channel bandwidth is increased from 14 to 36, and bandwidth suitable for the target service on a segment ba- the maximum number of segments up to 35 is used for sis and simultaneously provide services for fixed reception signal transmission. A narrowband receiver that carries out and mobile reception on one channel. partial reception is supposed to receive nine OFDM seg- In the case of ISDB-T, the bandwidth of one OFDM ments (with a bandwidth of 1.50 MHz) in the center from segment is 1/14 of the bandwidth (6 MHz) of one channel among 35 OFDM segments, and the number of segments (hereafter referred to as “channel bandwidth”), and the in layer A can be set in the range from 1 to 9 to increase transmission signal of ISDB-T is composed of 13 OFDM the flexibility of the transmission capacity for partial segments. In addition, it is possible to perform hierarchi- reception. Moreover, by increasing the partial-reception cal transmission that simultaneously transmits up to three bandwidth to 3.5 times that of ISDB-T, the effect of fre- layers with different transmission parameters. Each layer quency interleaving for partial reception is enhanced3). is composed of one or a number of OFDM segments, and The example configuration in Fig. 1 shows hierarchical parameters such as the carrier modulation scheme, code transmission in which three segments are allocated to a rate of the error-correction code, and time-interleaving length can be set for each layer. Moreover, among the 13 *4 A structure by which signals are transmitted in units of blocks (called segments, the segment (0.43 MHz) in the center is set as a “OFDM segments”) in which the band of the OFDM signal is divided on the frequency axis. partial-reception band. *5 A function for demodulating only part of a signal to make it receiv- In regard to the advanced ISDB-T, we decided to inherit able.

0.43MHz 1 segment

Current ISDB-T scheme: 13 segments

Expand bandwidth for Fixed reception layer partial reception by 1.50MHz Power-boost 3.5 times function of Mobile reception layer mobile reception layer

Advanced ISDB-T: Partial reception band 35 segments 9 segments (normal mode)

(a) Configuration of Frequency bandwidth: 6 MHz To partial reception band OFDM segment in physical layer

Layer A (mobile reception layer) Layer B Layer C (b) Hierarchical structure in multiple layers

Figure 1: Hierarchical transmission and partial reception by the advanced ISDB-T (in the case of 3 segments in layer A) 12 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

FEATURE FEATURE

service for mobile reception and transmitted by layer A zation efficiency is improved by reducing the ratio of and the other segments are allocated to a service for fixed the guard interval (GI)*7 (which does not contribute to reception and transmitted by layers B and C with different information transmission). Although the current ISDB-T transmission parameters. In this, layer A accounts for the scheme can be operated with an FFT (fast Fourier trans- transmission capacity of three out of the nine segments of form) size*8 of 8,192 (213) points, the advanced ISDB-T the partial-reception band, and the carriers of layer A are can handle an FFT size of 32,768 (215) points. dispersed throughout the partial-reception band. The car- Increasing the FFT size reduces the carrierfrequency riers of layer A that perform partial reception have a func- spacing. Also, as the effective symbol length increases, tion (i.e., “power boost”) that sets the transmission power even when the GI is set to be the same, the ratio of the higher than that of the other carriers, thereby improving GI to the effective symbol length (hereafter referred to as the robustness of layer A4). In addition to this function, a “GI ratio”) becomes small, and the transmission capacity low-delay transmission path called “LLch” (low-latency can be increased. On the other hand, considering the com- channel) is provided in such a way that important infor- patibility the advanced ISDB-T with existing broadcast- mation (such as emergency earthquake warnings) can be ing, with this scheme, it is possible to select parameters transmitted with short delay. whose bandwidth and GI length match those of ISDB-T. Moreover, in the case of ISDB-T, the arrangement of the 2. 2 New signal structure pilot signals used for channel estimation is the same in As described in Section 2.1, in the advanced ISDB-T, the layers for fixed reception and for mobile reception; in the number of divisions of the channel bandwidth into contrast, in the case of the advanced ISDB-T, optimal ar- OFDM segments is increased from 14 to 36, of which 35 rangements can be separately selected for fixed reception segments are used for signal transmission. Comparing the and for mobile reception. advanced ISDB-T (which uses 35 of 36 segments of the 6-MHz channel) with ISDB-T (which uses 13 of 14 segments) reveals that together with increasing the number of *6 An unused frequency band provided for preventing interference with divisions of a segment to improve the flexibility of allo- a system that uses adjacent frequency bands. *7 In the case of the OFDM modulation scheme, redundant signal pe- cating bandwidth to fixed reception or mobile reception, riods that are inserted so that each transmission symbol does not in- reducing the guard band*6 makes it possible to increase terfere with the transmission symbols coming before or after it under multipath interference. capacity by about 5%. *8 Number of samples of FFT used for OFDM signal modulation/de- In the case of the advanced ISDB-T, frequency-utili- modulation.

Q axis Q axis

I axis I axis

(a) UC (b) NUC

Figure 2: Example of signal constellation of 64QAM NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 13

FEATURE FEATURE

2.3 Introduction of the latest technologies for improv- chical-frame signals (on layers A, B, and C), TMCC ing transmission performances (transmission and multiplexing configuration control) In the case of ISDB-T, a concatenated code composed information*10, and an LLch frame signal are output from of a convolutional code and a Reed-Solomon (RS) code the input interface. The former signals are input into the is adopted as an error-correction code. By correcting er- bit-interleaved coded modulation (BICM) blocks and rors that cannot be corrected by the convolutional code are subjected to error-correction coding and mapping to with the RS code, resistance to noise, interference, etc., is carrier symbols*11. Furthermore, the power for each layer increased. In the case of the advanced ISDB-T, to further is adjusted, e.g., the power of layer A is boosted, at the reduce the required carrier to noise ratio*9 (CNR), as a new level-adjustment blocks. After that, the hierarchical-com- error-correction code, an LDPC (low-density parity check) bining blocks combine the carrier symbols for layers A, B, code with higher error-correction capability than a convo- and C. Then, band splitting, time IL (time interleaving), lutional code is adopted 5), and transmission characteristics frequency IL (frequency interleaving), and band combin- are improved by using it concatenated with a BCH (Bose– ing are performed to form data segments*12. Chaudhuri–Hocqenghem) code. To achieve low-delay transmission, processing that Regarding the carrier modulation scheme in the case differs from the hierarchical frame of the three layers is of ISDB-T, three types are available: QPSK (quadrature applied to LLch frame, and DBPSK (differential binary phase shift keying), 16QAM (quadrature amplitude mod- phase-shift keying) modulation is performed after the ulation), and 64QAM, and the signal points are arranged differential reference bit is added to the beginning of the in a uniform constellation (UC). On the other hand, in the LLch frame. At the OFDM-frame-composition blocks, case of the advanced ISDB-T, by achieving multilevel a pilot signal, LLch signal, and TMCC information are modulation ranging from QPSK to 4096QAM, in addition added to the data segments to construct an OFDM frame. to a UC, robustness against noise in multilevel modulation OFDM modulation is then applied to the frame by using is improved by introducing a non-uniform constellation an inverse fast Fourier transform (IFFT), a GI is added, (NUC), which makes the arrangement of signal points and the OFDM frame is output as an IF (intermediate fre- non-uniform6). Examples of a UC and an NUC in the case quency) signal. The signal for SISO transmission is gener- of 64QAM are shown in Fig. 2. The NUC in Fig. 2 is an ated via path ①-②-③ in Fig. 3. example of a constellation of signal points, and in the case In the case of MISO transmission, MISO coding is ap- of the advanced ISDB-T, it was decided to design and use plied to the carrier symbols after band combining. The an optimal signal constellation according to the code rate OFDM frame is then formed and subjected to OFDM of the LDPC code. modulation. The signal of system 1 for MISO transmission For ISDB-T, which is a single-input single-output (SISO) is generated by the same path (①-②-③) as for SISO trans- transmission scheme, it is assumed that one transmitting mission, and the signal of system 2 is generated by path antenna and one receiving antenna are configured, and ①-④-⑤. The MISO encoding unit applies STBC (space– a signal is transmitted by either a horizontally polarized time block coding) or SFBC (space–frequency block cod- wave or a vertically polarized wave. In contrast, in the ing) only to the carrier symbols for data. case of the advanced ISDB-T, it is possible to improve STBC combines two data carrier symbols in the time transmission robustness by using the 2×1 MISO (multi- direction, SFBC combines two data carrier symbols in the ple-input single-output) scheme—using two transmitting frequency direction, the complex conjugate is obtained, antennas and one receiving antenna—in addition to the and sign inversion is applied to the data carrier symbols. SISO scheme, and increase transmission capacity and/or MISO coding aims to improve the transmission robustness transmission robustness by using the 2×2 MIMO (multiple-input multiple-output) scheme—using two transmit- *9 carrier to noise ratio necessary to achieve pseudo-error-free trans- ting antennas and two receiving antennas. mission (bit error rate of 1×10–12 or less) after error correction. *10 Information required for signal demodulation (transmission parame- 3. Outline of channel coding ters such as carrier modulation scheme and code rate of error-correction code). It is transmitted using specific carriers. *11 A unit of transmission consisting of one transmission symbol (in the 3.1 Basic configuration of channel coding time direction) and one carrier symbol (in the frequency direction) of OFDM (see Fig. 6). The basic configuration of the channel coding of the *12 Data such as video and audio transmitted in one OFDM segment is advanced ISDB-T is shown in Fig. 3. Three hierar- called a “data segment.” 14 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

FEATURE FEATURE

Input interface

Layer A Layer B Layer C

TMCC information LLch BICM BICM BICM

Level Level Level adjustment adjustment adjustment

Systematic Systematic Systematic separation separation separation

TMCC Differential information bit Pilot signal reference bit generation generation addition

Layer combining Layer combining

Synchroniza- Band splitting Band splitting TMCC tion bit DBPSK generation generation modulation

Time IL Time IL

Frequency IL Frequency IL

Band combining Band combining

MISO coding

STBC/SFBC SDM

OFDM OFDM frame composition frame composition

IFFT IFFT

GI GI Addition Addition

Signal output 1 Signal output 2

Figure 3: Transmission-path encoder of advanced ISDB-T

by exploiting the transmission-diversity effect in a poor Moreover, in the case of the advanced ISDB-T, it is reception environment keeping the transmission capacity possible to select MIMO of space-division multiplexing equivalent to that in the case of SISO. (SDM), which transmits different information from the NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 15

FEATURE FEATURE

system 1 and system 2 transmission antennas and expands nas, MISO or MIMO can be selected for each layer. For transmission capacity. For MIMO transmission, it is ad- example, it is possible to improve the reception robustness vantageous that the transmission capacity can be doubled with layer A (for mobile reception) using MISO and to compared with that possible with SISO; however, to expand the transmission capacity and/or robustness with achieve satisfactory transmission characteristics by reduc- layer B (for fixed reception) using MIMO. ing the correlation between propagation paths 1 and 2 in a line-of-sight reception environment, problems such as the 3.2 Block configuration of error-correction code need for transmitting and receiving antennas correspond- The detailed configuration of the BICM block shown in ing to orthogonal polarization (horizontal and vertical po- Fig. 3 is described in Fig. 4. For forward error correction larization) remain to be solved. (FEC), a combination of concatenated codes (with the In the case of MIMO transmission, the level-adjusted LDPC code as the inner code and the BCH code as the carrier symbols are separated by the systematic-separation outer code) is adopted by both the second-generation digi- blocks into two systems, OFDM frames are formed and tal terrestrial broadcasting system DVB-T2 (Digital Video subjected to OFDM modulation, and the two signals are Broadcasting - Terrestrial 2)7) and Japan’s 4K/8K satellite output on respective systems. The signals of the MIMO 1 broadcasting system ISDB-S38). An NUC (the constella- system and 2 system are generated via paths ①-②-③ and tion used for the mapping) is a technique also introduced ⑥-⑦-⑤, respectively. For the frequency IL in the MIMO in the second-generation digital terrestrial broadcasting transmission, different interleaving processes are applied system ATSC3.0 (Advanced Television Systems Commit- for systems 1 and 2, and the interleaving effect is im- tee 3.0)9) in the USA. Characteristics close to the Shannon proved. limit*13 can be obtained by optimizing bit interleaving and In the advanced ISDB-T, SISO, 2×1 MISO, and 2×2 using the NUC and an LDPC code in combination with MIMO are introduced. However, in the case of using one

transmitting antenna, it is supposed that all layers transmit *13 The theoretical upper bound of the transmission rate calculated from via SISO, and in the case of using two transmitting anten- the quality of the transmission channel and the bandwidth. Bit IL Mapping BCH encoding LDPC encoding Energy diffusion* FEC block conversion

*Signal processing that prevents the specific frequency component of the modulated signal from becoming large by converting the data using a predetermined generator polynomial and reducing successions of “0” and “1” Figure 4: Configuration of BICM block

FEC block header (16 bits) BCH parity (192 bits)

Main signal area LDPC (Concatenated TLV packet or part of TLV packet) Parity

Information bit length of BCH code

Information bit length of LDPC code

Code length of LDPC code（69,120 bits）

Figure 5: Configuration of FEC block 16 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

FEATURE FEATURE

multilevel modulation. bit length of each part of the FEC block of the advanced In the “FEC block converter” shown in Fig. 4, the ISDB-T is listed in Table 1. variable-length TLV packet (which is the input data) is encapsulated*14 in a fixed-length FEC block. The configu- 3.3 Design of pilot signal ration of the FEC block is shown in Fig. 5. In the case of In regard to the transmission of the OFDM signal by the the advanced ISDB-T, to achieve strong error correction, advanced ISDB-T, to enable the receiver to estimate the codes were designed with an FEC block length of 69,120 distortion of the amplitude and phase caused in the prop- bits. This block length is longer than that of DVB-T2 and agation path and equalize it, the so-called “scattered pilot ISDB-S3. The parity of the LDPC code input to the FEC (SP)” is applied, in which known signals (pilot signals) block changes with the length according to the code rate are arranged regularly. Regarding the arrangement of the of the LDPC code. For example, when the code rate is pilot signals used for estimating the channel response, it 14/16, the length of the parity of the LDPC is 69,120 × is important to optimally design the arrangement interval

(1−14/16) = 8,640 bits and the length of the information (Dx) in the frequency direction and the arrangement inter-

bits is 69,120−8,640 = 60,480 bits. The information bits val (Dy) in the time direction in accordance with the size of the LDPC code includes the parity of the BCH code, of the broadcast area and the reception environment. Here,

and the length of the BCH parity is 192 bits regardless of as Dx becomes smaller, it becomes possible to estimate the code rate. The variable-length TLV packets are stored multipath interference waves over a wide range in terms in the main signal area shown in Fig. 5. When the TLV of time and to create a large broadcast area. In contrast, as

packet cannot fit in this area, it is divided and partly stored Dy becomes smaller, the tracking ability with respect to the there, while the rest is stored in the main signal area of the time variation of the transmission channel increases, and next FEC block. At that time, the start position of the TLV high-speed mobile reception becomes possible. packets in each FEC block is designated by the TLV packet pointer in the FEC block header shown in Fig. 5. The *14 To fit the data in an optimal format according to its type.

Table 1: Bit length of each part of FEC block

LDPC code BCH code FEC block Main signal Parity bit Information bit Parity bit Information bit header Code rate Code length length length length length

2/16 69,120 60,480 8,640 192 8,448 16 8,432

3/16 69,120 56,160 12,960 192 12,768 16 12,752

4/16 69,120 51,840 17,280 192 17,088 16 17,072

5/16 69,120 47,520 21,600 192 21,408 16 21,392

6/16 69,120 43,200 25,920 192 25,728 16 25,712

7/16 69,120 38,880 30,240 192 30,048 16 30,032

8/16 69,120 34,560 34,560 192 34,368 16 34,352

9/16 69,120 30,240 38,880 192 38,688 16 38,674

10/16 69,120 25,920 43,200 192 43,008 16 42,992

11/16 69,120 21,600 47,520 192 47,328 16 47,312

12/16 69,120 17,280 51,840 192 51,648 16 51,632

13/16 69,120 12,960 56,160 192 55,968 16 55,952

14/16 69,120 8,640 60,480 192 60,288 16 60,272 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 17

FEATURE FEATURE

Frequency Layer B (3, 4) Layer A (3, 2)

Time Figure 6: Examples of SP arrangement (one square indicates one carrier symbol)

Table 2: Transmission parameters

Advanced ISDB-T ISDB-T

Channel bandwidth (MHz) 6

FFT size NFFT (transmission mode) 8,192 (3), 16,384 (4), 32,768 (5) 2,048 (1), 4,096 (2), 8,192 (3)

Number of segment divisions 36 14

33 + adjustment Number of segments of transmission signal 35 13 band*

5.57 5.83 Signal bandwidth (MHz) (compatibility 5.57 (normal mode) mode)

Number of layers Max. 3 levels

Number of segments in partial reception 1 - 9 1 band

6.321 8.127 FFT sample frequency (MHz) ( = 512/81) ( = 512/63)

Guard-interval ratio (GI ratio) 1/4, 1/8, 1/16, 1/32, 1/64, 1/256, 800/NFFT 1/4, 1/8, 1/16, 1/32

QPSK, 16QAM, 64QAM, 256QAM, Carrier modulation scheme 1024QAM, 4096QAM DQPSK, QPSK, 16QAM, 64QAM (16QAM NUC)

Error-correction code (inner code) LDPC code Convolutional code

Error-correction code (outer code) BCH code RS code

Transmission system SISO, 2 x 1 MISO, 2 x 2 MIMO SISO

*A band consisting of less than 1 segment of carriers. For each adjustment band at the right and left ends of the spectrum, a carrier with an integral multiple of the pilot signal placement interval (arrangement interval in the frequency direction × arrangement interval in the time direction) is arranged, and the signal bandwidth (5.57 MHz) equivalent to that used in ISDB-T is allocated.

For the advanced ISDB-T, on the premise that the broad- reception) and (Dx, Dy) = (3, 2) is set for the OFDM seg-

casting areas are equivalent in all layers, Dx is set to the ments of layer A (for mobile reception) are shown. In the

same value for all layers and Dy can be set to a different figure, blue SPs are inserted in both layers A and B, and

value for each layer. In Fig. 6, examples in which (Dx, Dy) green SPs are inserted in layer A only. = (3, 4) is set for the OFDM segments of layer B (for fixed 18 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

FEATURE FEATURE

40 the FFT size and reducing the GI ratio while inheriting the functions of hierarchical transmission), considering the advanced ISDB-T migration from the current terrestrial broadcasting to the 30 (SISO system) Shannon limit next-generation system, 800/NFFT was included in the spec- 10Mbps ification as the GI ratio for setting the GI length to 126 μs 20 (which is the same as the operational parameter used for 7dB ISDB-T (64 QAM, 3/4) ISDB-T). Here, NFFT indicates the FFT size, which is 8,192 10 (8k), 16,384 (16k), or 32,768 (32k) depending on the transmission mode. Transmission capacity(Mbps/1ch) Transmission Regarding the signal bandwidth, it is possible to choose 0 10 15 20 25 30 either the normal mode (bandwidth: 5.83 MHz, where the Required CNR (dB) guard band is reduced compared with that for ISDB-T) us- Figure 7: Relationship between transmission ing 35/36 of the channel bandwidth or a compatible mode capacity and required CNR (SISO) (5.57 MHz: the same bandwidth as for ISDB-T). The relationship between the transmission capacity per 3.4 Transmission parameters and transmission capac- channel of the advanced ISDB-T and the required CNR ity is plotted in Fig. 7. Under the supposition that the trans- Transmission parameters of the advanced ISDB-T are mission system is SISO, as calculated by computer sim- listed in Table 2. Although the new scheme attempts to ulation, the FFT size was 16k, the GI ratio was 1/16, the expand transmission capacity compared with that with number of segments was 35, and the SP ratio was 8.3%. ISDB-T by enhancing the frame structure (by increasing When the advanced ISDB-T (blue circles) and the ISDB-T

Table 3: Examples of transmission capacity of advanced ISDB-T

advanced ISDB-T ISDB-T

Mobile reception Fixed reception Mobile reception Fixed reception

FFT size 16,384 8,192

Bandwidth 5.83 MHz 5.57 MHz

Effective symbol length 2,592 μs 1,008 μs

Guard-interval length (GI ratio) 126 μs (800/16,384) 126 μs (1/8)

35 13 Number of segments 4 31 1 12

Pilot ratio 4.1% 4.1% 8.3% 8.3%

Carrier modulation scheme 64QAM 1024QAM QPSK 64QAM

Error-correction code LDPC code/BCH code Convolutional code/RS code

Code rate 7/16 11/16 2/3 3/4

Multiplexing scheme MMT-TLV MPEG-2 TS

In the case of SISO In the case of SISO transmission transmission 1.5 Mbps 31.4 Mbps Transmission capacity 0.4 Mbps 16.8 Mbps In the case of MIMO In the case of MIMO transmission transmission 3.0 Mbps 62.8 Mbps NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 19

FEATURE FEATURE

scheme (green diamond) are compared, it can be seen that K. Tsuchida, and K. Shibuya: “Study of Proposed Specifi- in the case of the former scheme (i) the transmission ca- cation for the Next Generation Terrestrial Broadcasting,” pacity increases by about 10 Mbps with the same required Proceedings of ITE Annual Convention, 31A-1 (2016) (in CNR and (ii) the required CNR can be reduced by about 7 Japanese) dB while keeping the same transmission capacity. 2) S. Aoki, T. Shitomi, and T. Takeuchi: “A Study on IP Multi- In Table 3, the transmission capacity of the advanced plexing Scheme in Next-Generation Terrestrial Broadcasting ISDB-T is compared with that of the current digital ter- System,” Proceedings of ITE Annual Convention, 14C-1 restrial broadcasting scheme in an example in which four (2017) (in Japanese) segments are used for mobile reception and 31 segments 3) H. Miyasaka, A. Sato, S. Asakura, T. Shitomi, N. Shirai, are used for fixed reception. In accordance with the current Y. Narikiyo, T. Takeuchi, M. Nakamura, K. Murayama, M. digital terrestrial broadcasting, the transmission capacities Okano, K. Tsuchida, and K. Shibuya: “A study on the inter- are compared in an example of two-layer transmission (i.e., leaver of partial reception toward the proposed specification layer A and layer B). For the advanced ISDB-T, compared of the next generation terrestrial broadcasting,” ITE Techni- with ISDB-T, the transmission capacity is greatly in- cal Report, Vol. 41, No. 11, BCT2017-46, pp. 29-32 (2017) (in creased owing to the increase in transmission bandwidth, Japanese) the reduction in the GI ratio (due to the increased FFT 4) Y. Narikiyo, A. Sato, H. Miyasaka, S. Asakura, T. Shitomi, S. size), the multileveling of the carrier modulation scheme, Saito, T. Takeuchi, M. Nakamura, K. Murayama, M. Okano, etc. K. Tsuchida, and K. Shibuya: “A study of a partial reception toward the proposed specification of the next generation ter- 4. Concluding remarks restrial broadcasting,” ITE Technical Report 31A-2 (2016) (in As stated in this report, aiming to enhance the function- Japanese) ality and quality of digital terrestrial TV broadcasting, 5) S. Asakura, T. Takeuchi, T. Shitomi, H. Miyasaka, A. Sato, Y. NHK are studying a new transmission scheme for simulta- Yamagami, T. Ijiguchi, N. Shirai, M. Nakamura, M. Okano, neously providing Super Hi-Vision broadcasting for fixed K. Murayama, and K. Tsuchida: “Performance Evaluation of reception and high-definition broadcasting for mobile the LDPC Codes in the Laboratory Experiment,” Proceed- reception on one channel. While inheriting features of the ings of ITE Annual Convention, 33D-3 (2018) (in Japanese) ISDB-T scheme (such as the hierarchical transmission 6) T. Shitomi, A. Sato, H. Miyasaka, S. Asakura, S. Saito, Y. function based on a segment structure), the new advanced Narikiyo, T. Takeuchi, M. Nakamura, K. Murayama, M. ISDB-T has newly introduced features, namely, a new Okano, K. Tsuchida, and K. Shibuya: “A Study on Carrier signal structure and the latest technology for improving Modulation for Draft Specification of Next Generation Ter- transmission performance. In particular, the flexibility restrial Broadcasting Transmission System,” Proceedings of of allocating bandwidth to multiple services for different ITE Annual Convention, 31A-3 (2016) (in Japanese) receiving modes (such as fixed reception and mobile re- 7) ETSI EN 302 755 V1.4.1, “Digital Video Broadcasting ception) is improved. Moreover, by introducing the latest (DVB) ; Frame Structure, Channel Coding and Modulation LDPC code, NUC, MISO/MIMO technology, etc., for im- for a Second Generation Digital Terrestrial Television Broad- proving transmission performance, the new transmission casting System (DVB-T2)” (2015) scheme attains higher frequency-utilization efficiency, 8) ARIB STD-B44 (2014) Radio Industry Association: “Trans- larger capacity, and better transmission robustness than mission System for Advanced Wide Band Digital Satellite ISDB-T. Broadcasting (Version 2.0),” ARIB STD-B44 (2014) From now onwards, aiming to further improve the 9) A/322:2017, “ATSC Standard: Physical Layer Protocol” quality and functionality of digital terrestrial television (2017) broadcasting, we will continue to improve the advanced ISDB-T by repeating performance evaluation and function verification using prototype modems.

References

1) T. Takeuchi, A. Sato, H. Miyasaka, S. Asakura, T. Shitomi, S. Saito, Y. Narikiyo, M. Nakamura, K. Murayama, M. Okano, 20 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

R & D F R & D Hybridcast Connect Library: For Easy Integration of TVs and Smartphones

Tohru Takiguchi

As a service for linking broadcasting and the internet, Connect Library is installed, a user can select the TV chan- “Hybridcast” provides a companion device communica- nel and watch the news on air by a simple one-tap action tion function that can display information related to pro- (see photo).Although the use of TVs compatible with the grams being watched on TV on a smartphone or tablet. To new standard protocol will become widespread from now, use the conventional companion device communication using the companion device communication function in function provided until now, it was necessary to install the manner described above will make it possible to easily a companion application (CA)*1 provided from each TV provide Integrated Broadcast-Broadband service and de- manufacturer on one’s smartphone, and the operation pro- velop services more flexibly. cedures were diverse. Moreover, to use the CA in conjunc- From now onwards, we will continue our research and tion with a TV, it was necessary to activate the Hybridcast development so that broadcasting services are coordinated application of the TV in advance, and linkage with various with telecommunications in a manner that creates a more internet services was inflexible and difficult. familiar medium. To overcome these problems, we aimed to create a com- mon CA, called “Hybridcast Connect®*2”, that can use a *1 A mobile application that acquires and displays information in companion device communication function regardless of cooperation with a Hybridcast-compatible TV 2 A registered trademark of the IPTV Forum (an association) the TV manufacturer, and a communication protocol for * *3 IPTVFJ STD-0010 Integrated Broadcast-Broadband System linking Hybridcast-compatible TVs and smartphones was Specification Version 2.2 and STD-0013 Hybridcast Opera- standardized at the IPTV Forum in 2016. Furthermore, tional Guideline Version 2.7 in September 2018, a function for selecting a TV channel from an application such as a mobile app and launching an Hybridcast app was added*3. At STRL, we proposed a companion screen architecture for linking various mobile apps and devices (such as IoT devices) with TVs and for establishing flow lines leading to broadcasting services (see the figure below), and we contributed to the standardization of a new communication protocol*3. Moreover, to make it easy to incorporate the new standard protocol into various mobile applications, we prototyped a “Hybridcast Connect Library”, which modularizes only necessary functions. This prototype is expected to reduce development costs and expand services. For example, if a developer adds a button called “Watch news on NHK General TV” to the mobile app in which Hybridcast Example of cooperative services between a mobile app and a TV

Hybridcast app Services via various applications and devices Companion HTML5 browser screen architecture

Launched from companion device 2 HybridcastConnect Library* App control function Companion device communication protocol Companion device Hybridcast-compatible TV Figure:Companion screen architecture and flow lines leading to broadcasting services

*Device-linking architecture and flow lines leading to broadcasting services Example of cooperative services between a mobile app and a TV NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 21

R & D F R & D Toward the realization of a Bendable Display: Extending the Lifetime using an Inverted-Structure Organic EL Device

Tsubasa Sasaki

For viewers to enjoy “8K Super Hi-Vision” images on * Organic EL (electroluminescence): The phenomenon by which a a large-screen TV even at home, flexible organic EL* dis- device emits light when an electric current is applied to its organic materials. plays—which are easy to carry into homes and install in various rooms as well as being thin, light, and bendable— are expected to be put to practical use. This kind of display is fabricated using a plastic film as the substrate. However, when a glass substrate is replaced with a film, oxygen and moisture in the atmosphere, which cause the deterioration of organic EL devices, are more likely to be incorporated in these devices, leading to the deterioration and darken- ing of the display over time. To overcome this problem, at Original electron-injection layer NHK’s STRL, we are developing organic EL devices that are resistant to oxygen and moisture. An organic EL device consists of multiple thin films. One of the thin films has an electron-injection layer through which electrons from the electrode are injected Conventional organic EL device Inverted-structure into the organic layer. In the case of conventional organ- organic EL device ic EL devices fabricated using glass substrates, an alkali Conventional Inverted-structure metal is used as the electron-injection layer. However, organic EL device organic EL device alkali metals are vulnerable to oxygen and moisture; thus, pixel defects will gradually form when they are used as a film substrate. Accordingly, aiming to use a material that is more stable than an alkali metal, we adopted an invert- Defect After After 5 days 35 days ed-structure organic EL device (Figure 1 top right), which is applicable to various fabrication methods, and we devel- Figure1:Structure of organic EL device (top) and formation of oped a novel electron-injection layer using a material suit- pixel defect (bottom) able for this device. The developed material can be formed uniformly over the entire display surface and it is stable in the atmosphere, and the display shows less reduction in luminance. To compare the performance of a conventional organic EL device with that of the inverted-structure organic EL device, we fabricated a simple flexible organic EL display using each device (Figure 2), and we evaluated the deterioration characteristics of both displays in the atmosphere (Figure 1 bottom). The results of the comparison demonstrated that the display with the inverted-structure organic EL device exhibited superior and uniform light emission with little degradation (such as the appearance of pixel defects) occurring over a long time. Henceforth, we will continue to work on improving the performance of the inverted-structure organic EL device while advancing our research to achieve practical flexible displays. Figure2:Flexible display fabricated using conventional and inverted-structure organic EL device 22 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019

R & D F R & D 3D Imaging Technology: Achieving High Resolution with a Camera Array

Masanori Kano

Aiming to develop new broadcasting services of the fu- “view interpolation*”. The camera array consists of 14 ture, we are promoting R&D on 3D televisions. To make cameras arranged horizontally and 11 cameras arranged 3D television a reality, technologies such as imaging tech- vertically, making it possible to simultaneously capture nology (for producing television programs), compression high-resolution images from 154 viewpoints (see Figure 1). and coding technology (for ensuring efficient transmission Then, on the basis of the video images (hereafter, “captured of video), and display technology (to enable viewing at image(s)”) captured by the camera array, the video images home) are necessary. In the following, a new imaging (“interpolation image(s)”) to be displayed are generated by technology is introduced. view interpolation processing (Figure 2). In the view inter- The method of 3D imaging being developed by NHK polation, by estimating the approximate depth of the sub- Science & Technology Research Laboratories (STRL) has ject, it is possible to determine the color of a pixel of the the features that it does not require special glasses and that interpolation image from the color of the corresponding the appearance of the subject changes according to the pixel of the captured images. It is possible to reproduce direction of viewing.This is made possible by reproducing 3D video with a resolution about three times higher than (displaying) the image of the subject viewed from various before by displaying the images generated by this imaging angles. On the imaging side, it is necessary to capture technology. all video images that need to be displayed from multiple From now onwards, we will work on further increasing viewpoints. So far, at STRL, we have captured video im- the resolution of 3D imaging and developing efficient images with a lens array (with many small lenses arranged aging technologies. Furthermore, we will aim to develop in a plane) in front of the camera; however, this method future 3D television in conjunction with developing dis- suffers from the problem that it is difficult to increase the play and coding technologies. resolution. To solve this problem, we have developed a novel 3D * On the basis of the images captured by the cameras, images imaging technology using a camera array (composed of from viewpoints from which they cannot be captured (such as a large number of cameras) and image processing called those between cameras) are generated by computer processing.

About 260cm

Front viewpoint

About 200cm

Top viewpoint

Left viewpoint Right viewpoint

Bottom viewpoint

Figure1:Camera array (top) and captured images (bottom) Figure2:Part of the generated 3D image (five viewpoints) NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 23

R & D JOURNALSF R & D

Required Bit Rate of 22.2 Multichannel Audio Signal Compressed by MPEG-H 3D Audio to Meet Trend in research on organic imaging devices Broadcast Quality Acoustical Science and Technology, Vol.39, No.3, pp.266-269 (2018) At NHK Science and Technology Research Laboratories, Takehiro Sugimoto and Tomoyasu Komori we are researching and developing organic imaging devices. An organic imaging device is a new kind of iming to achieve Super Hi-Vision broadcasting by terrestrial waves, NHK is promoting R&D for advancing ter- singlechip imaging device that consists of a stack of organic restrial broadcasting by developing technologies such as audio/video coding technology and channel coding tech- color imaging device that can independently extract the photoelectric conversion layers for each primary color, nology in collaboration with manufacturers and universities. In the case of terrestrial broadcasting, transmission photo-generated charges corresponding to each primary A each having a charge readout circuit that is transparent capacity is restricted, and compared to coding technology for the new 4K/8K satellite broadcasting, coding technology color by separating light into its constituent primary to visible light. We are using such devices as the basis for for terrestrial Super Hi-Vision broadcasting requires a higher compression efficiency. Accordingly, while promoting the colors as it passes through the device in the depth a new generation of super-compact high-quality color standardization of MPEG-H 3D Audio (3DA) (i.e., the latest audio coding scheme), we have been performing subjective direction. Since an organic imaging device can generate cameras. With the single-chip color imaging device, it evaluation experiments to determine the transmission bit rate for 22.2 multichannel sound that satisfies the broadcast qual- a color video signal by using all of the incident light, it is possible to separate light into different colors along ity. As evaluation sound sources, three types, applause, a music program, and SL , were selected and stimuli with five bit can in principle be used to implement a single-chip color the direction in which the light travels. This means it is rates (288, 512, 704, 880, and 1,200 kbit/s) were made. According to the results of evaluation by 12 listeners by using the imaging device that has the same picture quality as a possible in principle for it to have color imaging with double-blind triple-stimulus with hidden reference method [a subjective evaluation method for assessing hidden reference three-chip system. the same picture quality as a system that uses a color (original sound) and coded sound referring to known reference of the original sound], it was found that the coded sounds In this paper, after a discussion of the issues of modern separation prism. In this article, we discuss the concept satisfy the broadcast quality stipulated in the recommendation of ITU-R (International Telecommunication Union - Radio- camera imaging systems and the operating principles of of organic imaging devices and the characteristics of communication Sector) at 512 kbit/s and above. It can be concluded that since MPEG-4 AAC (Advanced Audio Coding), organic imaging devices, we describe the characteristics organic photoelectric conversion layers. We also describe which was evaluated at the same time, requires a bit rate of 1,200 kbit/s or more, MPEG-H 3DA improved the compression of a prototype organic photoelectric conversion layer the current status of device development. efficiency by about 2.5 times compared to that of MPEG-4 AAC. we made to verify these principles. Then, we discuss the 1. Introduction current development status of organic imaging devices Today, almost all cameras used in broadcasting are made by stacking such layers together. based on a three-chip color imaging method that uses Estimating Depth Range Required for 3-D Displays to Show Depth-Compressed Scenes Without 2. Modern camera imaging systems three imaging devices and a color separation prism Inducing Sense of Unnaturalness Figure 1 illustrates the color imaging systems used to obtain high-quality color images. The increasing IEEE Transactions on Broadcasting, Vol.64, No.2, pp.488-497 (2018) in modern cameras. A three-chip system obtains color diversity of program production has led to a strong Yasuhito Sawahata and Toshiya Morita information with three imaging devices, while a singlechip demand for broadcast cameras that are more compact system obtains color information with a single and lightweight and have greater mobility, but this is n this study, the required depth range for a light-ray-reproduction-type display (3D display), such as an integral display, imaging device. The three-chip system is mainly used difficult to achieve with three-chip cameras. A possible to reproduce 3D images with high spatial resolution were estimated. A feature of the 3D display is that it can reproduce in professional cameras. In this system, light that has solution to this problem is to use a single-chip color 3D images as if objects were actually in place; however, presenting the 3D image of a scene with great depth (like a passed through an optical lens is separated into the three I imaging device, which eliminates the need for a color landscape) was challenging. This is because the depth range (depth reproduction range) in which 3D images can be repro- primary colors - red (R), green (G) and blue (B) - by a color separation prism. However, single-chip technology has duced with high spatial resolution is limited and because the images reproduced at the near and far from the display surface separation prism, and it is then converted into electrical not been used much in broadcast cameras because are inevitably blurred. In this study, we introduced a “depth compression method” that deforms (“compresses”) the shape signals by three separate imaging devices (Figure 1(a)). its picture quality is still inferior to that of three-chip of the scene objects in the depth direction and places the entire scene within the depth reproduction range; as a result, it Although the three-chip system offers better sensitivity, technology. enables 3D image reproduction of a scene with great depth without blurring. The results of a subjective evaluation exper- resolution and color fidelity, it requires a color separation To comprehensively resolve this issue, STRL is iment using CG (computer graphics)-generated scenes showed that the scene depth can be greatly compressed—without prism and three imaging devices. This limits the extent researching and developing organic imaging devices. inducing unnatural feelings on the deformed shapes in viewers—by depth compression considering the characteristics of to which cameras based on this system can be made An organic imaging device is a new kind of single-chip human depth perception. Specifically, we found that unnaturalness is still acceptable even when a scene with a depth of more compact and lightweight. On the other hand, most 250 m is deformed and compressed into a depth of 1 m. It is therefore concluded that if the 3D display can physically re- consumer-oriented video cameras and digital cameras produce a depth of 1 m, scenes with any depth can be virtually shown with perceptually appealing quality. use a single-chip color imaging system (Figure 1(b)), where the surface of a single imaging device is covered with a mosaic of color filters corresponding to the three Pixel-Parallel 3-D Integrated CMOS Image Sensors for Next-Generation Video Systems ECS Transactions, Vol.85, No.8, pp.163-166 (2018) Masahide Goto, Yuki Honda, Toshihisa Watabe, Kei Hagiwara, Masakazu Nanba, Yoshinori Iguchi, Takuya Saraya*, Masaharu Kobayashi*, Eiji Higurashi*, Hiroshi Toshiyoshi*, and Toshiro Hiramoto* *The University of Tokyo

iming to create a next-generation image sensor that can achieve both a high resolution and a high frame rate, we have been studying a pixel-parallel 3D-integrated image sensor. A feature of this device is that the received sig- Anals for each pixel are processed in parallel and outputted in the depth direction of the substrate. By the direct bonding of SOI (silicon-on-insulator) substrates (in which 10-μm-diameter gold electrodes are embedded for each pixel) to three-dimensionally connect a photodiode, an A/D conversion circuit, and a 16-bit counter, we prototyped a sensor that performs A/D conversion by generating a number of pulses corresponding to the amount of incident light for a pixel. As a result, we confirmed its operation as a video image sensor (with 128 × 96 pixels) that performs pixel-parallel signal processing and attains excellent linearity corresponding to the amount of incident light, a 16-bit gradation output, and a wide dynamic range (96 dB). The developed 3D integration technology is useful not only for imaging devices but also for the high-density integration of devices such as integrated circuits, memories, and various sensors. 424 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 NHK STRL Bulletin, Broadcast Technology, No.76, Spring 2019 Science & Technology Research Laboratories NHK TECHNOLOGY Utilizing New Image Expression Technology for Sports Broadcasts Aiming to convey sports competitions in a more comprehensible manner, at NHK Science and Technology Research Lab- oratories (STRL), we are researching new image expression technologies, and in collaboration with production sites, we are utilizing these technologies for the production of programs in the quickest manner possible. In the following, we introduce two No.76 SPRING 2019 ISSN 1345-4099 technologies used for recent sports broadcasts. ■ Technology for displaying the trajectory of a golf putt In a putting scene during a golf broadcast, if the viewer knows the trajectory that the ball took on the green during the pre- vious player’s putt, the viewer can predict the trajectory of the ball of the next player to putt. We therefore developed a system that detects and tracks the ball on the green from camera images in real time and displays the trajectory of the putt with CG. The system tracks the ball while learning the image features of the ball in the images; consequently, the ball can be stably tracked even when the brightness of the image changes owing to changes in sunlight. The developed system was used at the 83rd Japan Open Golf Championship held from 11 to 14 October 2018, and viewers expressed opinions such as, “The trajectory of the ball is displayed in a way that makes the state of the green easy to understand, this makes the broadcast more enjoyable.” ■ Multi-motion technology “Multi-motion” is an image expression technology that can extract an indi- vidual moving subject (such as an athlete) from a scene and display that subject Putting trajectory display at the 83rd Japan sequentially in order. As a result, a series of movements of the subject can be Open Golf Championship presented in an easy-to-understand manner. We developed a system that can produce such multi-motion video at a live broadcast of a sports event in a short time and used it at the live broadcast of the 60th NHK Cup Jump (a ski jumping competition) held on November 4, 2018. By using the system to assist commen- tary during the broadcast (i.e., video replay), we communicated the movement of ski jumps more clearly to viewers in terms of the style of the ski jumper, the force applied to the skis, and so forth. From now onwards, we will continue to research and develop new image ex- Multi-motion imaging used at the 60th NHK pression technologies in cooperation with production sites. Cup Jump

FROM THE EDITORS

As you may have noticed, this issue came in with the renewed design. It is because this issue coincides with the transition from Japanese Heisei era (January 8, 1989 - April 30, 2019) to Reiwa era (May 1, 2019-), which took place along with the im- perial succession on May 1. The kanji characters of “Reiwa”, meaning “beautiful harmony”, were taken from a description of plum blossoms in a collection of Japanese classical poetry called “Manyoshu”, which led to this color tone. Upon this special occasion, we have renewed our determination to conduct R&D to create new broadcast technologies and services. STRL Open House 2019 to be held on May 30 through June 2 will reflect and convey our determination. The theme of Open House 2019 is “Taking media beyond the box.” We hope you visit and enjoy our Open House event as well as our website, https://www.nhk. or.jp/strl/open2019/index_e.html. (H.K.)