DOCSLIB.ORG

Information Theory and Its Application to Optical Communication

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Information Theory and Its Application to Optical Communication Information theory and its application to optical communication Citation for published version (APA): Willems, F. M. J. (2017). Information theory and its application to optical communication. In Signal Processing in Photonic Communications 2017, 24-27 July 2017, New Orleans, Louisiana [SpTu3E] Optical Society of America (OSA). https://doi.org/10.1364/SPPCOM.2017.SpTu3E.1 DOI: 10.1364/SPPCOM.2017.SpTu3E.1 Document status and date: Published: 01/01/2017 Document Version: Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers) Please check the document version of this publication: • A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal. If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement: www.tue.nl/taverne Take down policy If you believe that this document breaches copyright please contact us at: [email protected] providing details and we will investigate your claim. Download date: 03. Oct. 2021 Information Theory and its Application to Optical Communication FMJ Willems INTRODUCTION INFORMATION THEORY Information Theory and its Application to Optical WAVEFORM Communication CHANNELS AGN CAPACITY SOME CODES Frans M.J. Willems CODED MODULATION SHAPING CODES ICT Lab, SPS Group Department of Electrical Engineering REMARKS Eindhoven University of Technology SPPCom, Advanced Photonics 2017, July 25, New Orleans Introduction Information Theory and its Application to Optical Communication FMJ Willems Optical communication links carry most of the data that is transmitted INTRODUCTION around the world. Home connections are being replaced by optical links. INFORMATION THEORY We like to achieve the highest possible data rates for the smallest cost. Replacing links should be delayed as long as possible. WAVEFORM CHANNELS Therefore advance transmission protocols (equalisation, modulation, AGN CAPACITY coding) are required. SOME CODES INFORMATION THEORY tells us what the ultimate performances are CODED (e.g. capacity), and what the techniques are that achieve ultimate MODULATION performance. SHAPING CODES Wireless communication is characterized by major developments (coding, REMARKS mimo, cooperative communications, etc.), often boosted by information theoretical methods. Optical Communication is going through a similar innovation cycle now. Information theory can also be useful here. Claude Shannon (1916-2001) Information Theory and its Application to Optical Communication 1948: \A Mathematical Theory of Communication," FMJ Willems Bell Syst. Tech. J.: Shannon combined the noise power spectral density N0=2, the channel bandwidth INTRODUCTION W , and the transmit power P; into a single parameter INFORMATION C; which he called the channel capacity. More THEORY precisely Shannon Entropy, Conditional P Entropy, Mutual C = W log2(1 + ) Information N0W Capacity Discrete Memoryless Channel represents the maximum number of bits that can be Capacity Gaussian Channel sent per second reliably from transmitter to receiver. WAVEFORM Codes can be used to achieve capacity. CHANNELS 1938: Shannon also applied Boole's algebra to AGN CAPACITY switching circuits (MSc thesis, MIT). SOME CODES WW2: Shannon developed cryptographic equipment CODED for transoceanic conferences (Roosevelt-Churchill). His MODULATION ideas can be found in \Communication Theory of SHAPING CODES Secrecy Systems", a confidential report from 1945, REMARKS published in 1949. 1949: Shannon introduced the sampling theorem to the engineering community. Entropy Information Theory and its Application to Let X be a discrete random variable with alphabet X and probability mass Optical Communication function p(x) = PrfX = xg for x 2 X . FMJ Willems Definition INTRODUCTION The entropy H(X ) of discrete random variable X is defined as INFORMATION THEORY Shannon X 1 H(X ) = p(x) log [bit]: Entropy, Conditional 2 p(x) Entropy, Mutual x2X Information Capacity Discrete Memoryless Channel Capacity Gaussian Channel Example WAVEFORM CHANNELS Binary random variable X with alphabet X = f0; 1g. Let AGN CAPACITY 0 with probability 1 − p; X = SOME CODES 1 with probability p: CODED MODULATION Then the entropy of X is SHAPING CODES H(X ) = h(p); REMARKS where ∆ 1 1 h(p) = (1 − p) log + p log [bit]: 2 1 − p 2 p Entropy, Binary Entropy Function Information Theory and its Application to Optical Example Communication 1 FMJ Willems 0.9 INTRODUCTION 0.8 INFORMATION THEORY 0.7 Shannon Entropy, Conditional 0.6 Entropy, Mutual Information 0.5 Capacity Discrete Memoryless Channel h(p) (bit) 0.4 Capacity Gaussian Channel 0.3 WAVEFORM CHANNELS 0.2 AGN CAPACITY 0.1 SOME CODES 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 CODED p MODULATION SHAPING CODES Observe that h(p) = h(1 − p) and that e.g. h(0:1) = 0:4690. REMARKS Think of H(X ) as the uncertainty in X . It can be shown that 0 ≤ H(X ) ≤ log2 jX j. Conditional Entropy Information Theory Let X and y be discrete random variables with alphabets X and Y and its Application to Optical respectively and joint probability mass function p(x; y) = PrfX = x; Y = yg Communication for x 2 X and x 2 Y. P FMJ Willems Note that p(y) = x2X p(x; y) and p(xjy) = p(x; y)=p(y). INTRODUCTION Definition INFORMATION THEORY The conditional entropy H(X jY ) of discrete random variable X given Y is Shannon defined as Entropy, Conditional Entropy, Mutual Information X X 1 Capacity Discrete H(X jY ) = p(y) p(xjy) log2 [bit]: Memoryless Channel p(xjy) Capacity Gaussian y2Y x2X Channel WAVEFORM CHANNELS Think of H(X jY ) as the uncertainty in X when Y is given. It can be shown that 0 ≤ H(X jY ) ≤ H(X ). Conditioning can only reduce AGN CAPACITY entropy. SOME CODES Note also that CODED X MODULATION H(X jY ) = p(y)H(X jY = y); y2Y SHAPING CODES where REMARKS X 1 H(X jY = y) = p(xjy) log : 2 p(xjy) x2X Conditional entropy H(X jY ) is the expected value of entropies H(X jY = y) w.r.t. p(y). Example: Binary Symmetric Channel (BSC) Information Theory and its Application to Example Optical Communication Transition probabilities p(Y = 1jX = 0) = p(Y = 0jX = 1) = p. FMJ Willems X = 1 Y = 1 1 − p INTRODUCTION p INFORMATION THEORY p Shannon 1 − p Entropy, Conditional Entropy, Mutual X = 0 Y = 0 Information Capacity Discrete For uniform p(X = 0) = p(X = 1) = 1=2 we obtain that the entropy Memoryless Channel Capacity Gaussian Channel H(X ) = h(1=2) = 1: WAVEFORM CHANNELS Moreover: AGN CAPACITY SOME CODES p(Y = 1) = p(X = 0) · p + p(X = 1) · (1 − p) = 1=2; CODED p(X = 1; Y = 0) p(X = 1) · p MODULATION p(X = 1jY = 0) = = = p; p(Y = 0) p(Y = 0) SHAPING CODES p(X = 1; Y = 1) p(X = 1) · (1 − p) REMARKS p(X = 1jY = 1) = = = 1 − p; p(Y = 1) p(Y = 1) and the conditional entropy H(X jY ) = p(Y = 0)h(p) + p(Y = 1)h(1 − p) = h(p): Mutual Information Information Theory and its Application to Definition Optical Communication The mutual information I (X ; Y ) between the discrete random variable X and FMJ Willems Y is defined as I (X ; Y ) = H(X ) − H(X jY ) [bit]. INTRODUCTION INFORMATION THEORY Think of I (X ; Y ) as the decrease in uncertainty about X when Y is Shannon released. Equivalently it is the information that Y contains about X . Entropy, Conditional Entropy, Mutual It can be shown that always 0 ≤ I (X ; Y ) ≤ H(X ). Information I (X ; Y ) is also the decrease in uncertainty about Y when X is released. Capacity Discrete Memoryless Channel Capacity Gaussian Example Channel WAVEFORM Binary symmetric channel (BSC) with transition probability p: CHANNELS X = 1 Y = 1 AGN CAPACITY 1 − p SOME CODES p CODED p MODULATION 1 − p SHAPING CODES X = 0 Y = 0 REMARKS For uniform p(X = 0) = p(X = 1) = 1=2, we obtain that I (X ; Y ) = H(X ) − H(X jY ) = 1 − h(p): For p = 0:1 we obtain I (X ; Y ) = 1 − 0:4690 = 0:5310 bit. Discrete Memoryless Channel Information Theory and its Application to Optical Communication Definition FMJ Willems Channel input alphabet X , channel output alphabet Y. INTRODUCTION For each x 2 X the transition probabilities PrfY = yjX = xg for y 2 Y are P INFORMATION denoted by p(yjx), where y2Y p(yjx) = 1. THEORY Shannon Entropy, Conditional Entropy, Mutual Example Information Capacity Discrete Memoryless Channel p(yjx) Capacity Gaussian ∆ ∆ Channel jX j jYj Here X = f1; 2; ··· ; jX jg and Y = WAVEFORM f1; 2; ··· ; jYjg. CHANNELS AGN CAPACITY . SOME CODES . CODED MODULATION 2 2 SHAPING CODES REMARKS 1 1 Channel Capacity, Definition Information Theory and its Application to Observe that: Optical Communication the channel input distribution fp(x); x 2 X g determines the joint FMJ Willems distribution INTRODUCTION p(x; y) = p(x)p(yjx); for all x 2 X ; y 2 Y; INFORMATION THEORY and therefore the mutual information Shannon Entropy, Conditional Entropy, Mutual I (X ; Y ) = H(X ) − H(X jY ): Information Capacity Discrete Memoryless Channel The maximum value of I (X ; Y ) is called the channel capacity C.
Load more

Recommended publications
  • Exploring and Experimenting with Shaping Designs for Next-Generation Optical Communications Fanny Jardel, Tobias A
    SUBMITTED TO JOURNAL OF LIGHTWAVE TECHNOLOGY 1 Exploring and Experimenting with Shaping Designs for Next-Generation Optical Communications Fanny Jardel, Tobias A. Eriksson, Cyril Measson,´ Amirhossein Ghazisaeidi, Fred Buchali, Wilfried Idler, and Joseph J. Boutros Abstract—A class of circular 64-QAM that combines ‘geo- randomized schemes rose in the late 90s after the rediscovery metric’ and ‘probabilistic’ shaping aspects is presented. It is of probabilistic decoding [18], [19]. Multilevel schemes such compared to square 64-QAM in back-to-back, single-channel, as bit-interleaved coded modulation [20] offer flexible and and WDM transmission experiments. First, for the linear AWGN channel model, it permits to operate close to the Shannon low-complexity solutions [21], [22]. In the 2000s, several limits for a wide range of signal-to-noise ratios. Second, WDM schemes have been investigated or proved to achieve the simulations over several hundreds of kilometers show that the fundamental communication limits in different scenarios as obtained signal-to-noise ratios are equivalent to – or slightly ex- discussed in [23]–[27]. ceed – those of probabilistic shaped 64-QAM. Third, for real-life In the last years, practice-oriented works related to op- validation purpose, an experimental comparison with unshaped 64-QAM is performed where 28% distance gains are recorded tical transmissions have successfully implemented different when using 19 channels at 54.2 GBd. This again is in line – or shaping methods, from many-to-one and geometrically-shaped slightly exceeds – the gains generally obtained with probabilistic formats to non-uniform signaling. The latter, more often called shaping. Depending upon implementation requirements (core probabilistic shaping in the optical community [28]–[32], has forward-error correcting scheme for example), the investigated perhaps received most attention.
    [Show full text]
  • APSK Constellation Shaping and LDPC Codes Optimization for DVB-S2 Systems
    International Journal of Modern Communication Technologies & Research (IJMCTR) ISSN: 2321-0850, Volume-2, Issue-6, June 2014 APSK Constellation Shaping and LDPC Codes Optimization for DVB-S2 Systems VANDANA A, LISA C DVB-S [1] is an abbreviation for “Digital Video Abstract - Here an energy-efficient approach is presented for Broadcasting – Satellite”, is the original Digital Video constellation shaping of a bit-interleaved low-density Broadcasting Forward Error Correction and demodulation parity-check (LDPC) coded amplitude phase-shift keying standard for Satellite Television and dates from 1995, while (APSK) system. APSK is a modulation consisting of several development lasted from 1993 to 1997. DVB-S is used in concentric rings of signals, with each ring containing signals both Multiple Channel Per Carrier (MCPC) and Single that are separated by a constant phase offset. APSK offers an attractive combination of spectral and energy efficiency, and is Channel per carrier modes for Broadcast Network feeds as well suited for the nonlinear channels. 16 and 32 symbol well as for Direct Broadcast Satellite. DVB-S is suitable for APSK are studied. A subset of the interleaved bits output by a use on different satellite transponder bandwidths and is binary LDPC encoder are passed through a nonlinear shaping compatible with Moving Pictures Experts Group 2 (MPEG 2) encoder. Here only short shaping codes are considered in order coded TV services. to limit complexity. An iterative decoder shares information among the LDPC decoder, APSK de-mapper, and shaping Digital Video Broadcasting - Satellite - Second decoder. In this project, an adaptive equalizer is added at the Generation (DVB-S2) [2] is for broadband satellite receiver and its purpose is to reduce inter symbol interference applications that has been designed as a successor for the to allow recovery of the transmit symbol.
    [Show full text]
  • Coding and Equalisation for Fixed-Access Wireless Systems
    CODING AND EQUALISATION FOR FIXED-ACCESS WIRELESS SYSTEMS A Thesis presented for the degree of Doctor of Philosophy . In Electrical and Electronic Engineering at the University of Canterbury Christchurch, New Zealand by Katharine Ormond Holdsworth, B.E.(Hons), M.E. January 2000 ENGINEERING LIBRARY iK 5/ .1\728 ;; For my Parents 3 1 MAY 20aa ABSTRACT This Thesis considers the design of block coded signalling formats employing spectrally efficient modulation schemes. They are intended for high-integrity, fixed-access, wireless systems on line-of-sight microwave radio channels. Multi­ dimensional multilevel block coded modulations employing quadrature ampli­ tude modulation are considered. An approximation to their error performance is described and compared to simulation results. This approximation is shown to be a very good estimate at moderate to high signal-to-noise ratio. The effect of parallel transitions is considered and the trade-off between distance and the error coefficient is explored. The advantages of soft- or hard-decision decoding of each component code is discussed. A simple approach to combined decoding and equalisation of multilevel block coded modulation is also developed. This approach is shown to have better performance than conventional independent equalisation and decoding. The proposed structure uses a simple iterative scheme to decode and equalise multilevel block coded modulations based on decision feedback. System per­ formance is evaluated via computer simulation. It is shown that the combined decoding and equalisation scheme gives a performance gain of up to 1 dB at a bit error rate of 10-4 over conventional, concatenated equalisation and decoding. v VI CONTENTS ABSTRACT v ACKNOWLEDGEMENTS ri LIST OF FIGURES xiii LIST OF TABLES xvii GLOSSARY OF ABBREVIATIONS xix GLOSSARY OF MATHEMATICAL TERMS xxi Chapter 1 INTRODUCTION 1 1.1 General Introduction 1 1.2 System Goals .
    [Show full text]
  • Increasing Cable Bandwidth Through Probabilistic Constellation Shaping
    Increasing Cable Bandwidth Through Probabilistic Constellation Shaping A Technical Paper prepared for SCTE•ISBE by Patrick Iannone Member of Technical Staff Nokia Bell Labs 791 Holmdel Rd., Holmdel NJ 07733 +1-732-285-5331 [email protected] Yannick Lefevre Member of Technical Staff Nokia Bell Labs Copernicuslaan 50, 2018 Antwerp [email protected] Werner Coomans Head of Department, Copper Access Nokia Bell Labs Copernicuslaan 50, 2018 Antwerp [email protected] Dora van Veen Distinguished Member of Technical Staff Nokia Bell Labs 600 Mountain Ave., Murray Hill NJ 07974 [email protected] Junho Cho Member of Technical Staff Nokia Bell Labs 791 Holmdel Rd., Holmdel NJ 07733 [email protected] © 2018 SCTE•ISBE and NCTA. All rights reserved. Table of Contents Title Page Number Table of Contents .......................................................................................................................................... 2 Introduction.................................................................................................................................................... 3 1. Probabilistic Constellation Shaping ........................................................................................................ 4 2. PCS for Digital Optical Links in DAA HFC Networks ............................................................................. 5 3. PCS Over Coaxial Links ........................................................................................................................
    [Show full text]
  • UNIT-5 UNIT-5/LECTURE-1 INTRODUCTION to INFORMATION THEORY the Main Role of Information Theory Was to Provide the Engineering A
    UNIT-5 UNIT-5/LECTURE-1 INTRODUCTION TO INFORMATION THEORY The main role of information theory was to provide the engineering and scientific communities with a mathematical framework for the theory of communication by establishing the fundamental limits on the performance of various communication systems. Its birth was initiated with the publication of the works of Claude E. Shannon who stated that it is possible to send in-formation-bearing signals at axed rate through a noisy communication channel with an arbitrarily small probability of error as long as the communication rate is below a certain quantity that depends on the channel characteristics; he baptized. this quantity with the name of channel capacity. He further pro-claimed that random sources {such as speech, music or image signals } possess an irreducible complexity beyond which they cannot be compressed distortion-free. He called this complexity the source entropy. He went on asserting that if a source has an entropy that is less than the capacity of a communication channel, then asymptotically error free transmission of the source over the channel can be achieved. A simple model of a general communication system is depicted in Figure 1.1. Source: The source is usually modeled as a random process (the necessary background regarding random processes is introduced in Appendix B). It can be discrete (Finite or countable alphabet) or continuous (uncountable alphabet) in value and in time. Source Encoder: Its role is to represent the source in a compact fashion by removing its unnecessary or redundant content (i.e., compression). Channel Encoder: Its role is to enable we dont take any liability for the notes correctness.
    [Show full text]
  • Optimization of a Coded-Modulation System with Shaped Constellation
    Graduate Theses, Dissertations, and Problem Reports 2013 Optimization of a Coded-Modulation System with Shaped Constellation Xingyu Xiang West Virginia University Follow this and additional works at: https://researchrepository.wvu.edu/etd Recommended Citation Xiang, Xingyu, "Optimization of a Coded-Modulation System with Shaped Constellation" (2013). Graduate Theses, Dissertations, and Problem Reports. 207. https://researchrepository.wvu.edu/etd/207 This Dissertation is protected by copyright and/or related rights. It has been brought to you by the The Research Repository @ WVU with permission from the rights-holder(s). You are free to use this Dissertation in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you must obtain permission from the rights-holder(s) directly, unless additional rights are indicated by a Creative Commons license in the record and/ or on the work itself. This Dissertation has been accepted for inclusion in WVU Graduate Theses, Dissertations, and Problem Reports collection by an authorized administrator of The Research Repository @ WVU. For more information, please contact [email protected]. Optimization of a Coded-Modulation System with Shaped Constellation by Xingyu Xiang Dissertation submitted to the College of Engineering and Mineral Resources at West Virginia University in partial fulfillment of the requirements for the degree of Ph.D. in Electrical Engineering Brian D. Woerner, Ph.D. Daryl S. Reynolds, Ph.D. John Goldwasser, Ph.D. Natalia A. Schmid, Ph.D. Matthew C. Valenti, Ph.D., Chair Lane Department of Computer Science and Electrical Engineering Morgantown, West Virginia 2013 Keywords: Information Theory, Constellation Shaping, Capacity, LDPC code, Optimization, EXIT chart Copyright 2013 Xingyu Xiang Abstract Optimization of a Coded-Modulation System with Shaped Constellation by Xingyu Xiang Ph.D.
    [Show full text]