INFORMATION THEORY in the Benelux and FIRE CONTROL Han Vinck Univ. Duisburg‐Essen Univ. Johannesburg, South Africa IT starts with Shannon The book co‐authored with Warren Weaver, The Mathematical Theory of Communication,reprintsShannon's1948 article and Weaver's popularization of it, which is accessible to the non‐specialist.[5] In short, Weaver reprinted Shannon's two‐part paper, wrote a 28 page introduction for a Weaver changed the title from“ "transformed cryptography 144 pages book, and changed the title from Amathematicaltheory...“ from an art to a science." "A mathematical theory..." to "The to "The mathematical theory..." mathematical theory..." BUT ... WIC meeting Gent May 2019 2 BEFORE ... (1946) “there is an obvious analogy between the problem of smoothing the data to eliminate or reduce the effect of tracking errors and the problem of separating a signal from interfering noise in communications systems.” WIC meeting Gent May 2019 3 RVO –TNO (Rijks Verdedigings Organisatie RVO ‐ Nederlandse Organisatie voor Toegepast‐ Natuurwetenschappelijk Onderzoek TNO. Ir. J.l. van Soest: Director RVO‐TNO 1927 – 1957 Extra ordinary Professor TU Delft Information and Communication Theory 1949 ‐ 1964 Task: NEW Research Directions. From mechanical (pre‐war) via analogue to digital methods for fire control Prof. van Soest, started (1955) a group on Information theory (Topics: game theory, cryptography and correlators WIC meeting Gent May 2019 4 But also this: hearing aids WIC meeting Gent May 2019 5 #1 Prof. IJ. Boxma successor of van Soest(director) in 1957: • After 1947: Digital Fire‐control development by ir. IJ. Boxma in the group Electronic computing, later Information Processing Systems (Militaire Spectator, 1958) 1950, Head Engineer: E.W. Gröneveld 1970 Ordinary Prof in Information and Communication theory at TU Delft. WIC meeting Gent May 2019 6 #2 Willem Gröneveld, successor Boxma in 1957: (Militaire Spectator, 1960) 1965 Ordinary Professor in InformationWIC and meeting Communication Gent May 2019 theory at TU Twente 7 #3: Piet (J.P.M.) Schalkwijk 1959‐1961 HSA Hengelo, 1st lieutenant Dutch armed forces Project digital fire control system DIPHYSA developed together with HSA Hengelo. 1961‐1963 RVO‐TNO (Rijks verdedigings organisatie) 1972 Ordinary Professor in InformationWIC theory meeting Gent at May TU2019 Eindhoven 8 Piet Schalkwijk 1965! Kailath’s first Ph.D student, is said to have complained that he was made to rewrite his thesis 6 times! In response Kailath (Stanford) says: “That’s an exaggeration. After all he graduated in two years after his Master’s degree. Almost all my students graduated within 3‐4 years from their Master’s degrees.” Best IEEE IT paper 1970 WIC meeting Gent May 2019 9 All chairs in information theory in the Netherlands are occupied in 1972 •Prof.ir. IJ. Boxma, Delft University of Technology (deceased) •Prof.ir. E. Willem Gröneveld, University of Twente (deceased) WIC founding fathers1986 •Prof.dr. Edward C. van der Meulen, KU Leuven •Prof.dr.ir. J.Pieter.M. Schalkwijk, TU Eindhoven Edward van der Meulen, 1976, KUL IT TOPICS COVERED: Data Processing (Boxma, Delft), Coding techniques (Schalkwijk, Eindhoven, Estimation and detection (Gröneveld, Twente) Algebraic Coding theory (Van Lint, Eindhoven and Goethals, MBLE Brussels), Multi User theory (vd Meulen, Leuven), WIC meeting Gent May 2019 10 Thanks Ludo! ISIT Sorento, ISIT Nice WIC meeting Gent1st May Benelux 2019 Japan workshop, 1989 (now AEW, 2019 R‘dam) 11 Next topic: memory with restrictions;. Some examples ‐ limit the number of changes to pn (Phase Change) ‐ memory with defects or permanent error ‐ memory with periodic writing direction: MO disk ‐ memory with erase phase: flash memory ‐ write once memory (WOM): only change 0 into 1 WIC meeting Gent May 2019 12 We start with defects: What is a defect? • defect‐00 0 1 • Image of a Poly‐Silicon defect‐10 Bridging Defect 1 1 Writing does not change the memory cell content We assume a fraction pn defects in a word of length n 50% defect‐0; 50% defect‐1 WIC meeting Gent May 2019 13 Key result for memories with a fraction p‐defects writer reader CAPACITY knows knows bits/cell YY 1 ‐ p(obvious) Sacha Kuznetsov YN 1 ‐ psurprise! (K‐T) In a word of length n with pn defects we can write (1‐p)n bits! Boris Tsybakov Kuznetsov‐Tsybakov (1974!) show that codes exist (45 years ago) A. V. Kuznetsov, B. S. Tsybakov , Coding in a Memory with Defective Cells , Probl. Peredachi Inf., 1974, Pages 52–60 (Mi ppi1029) ‐"On the General Defect Channel wi th Informed Encoder and Capacities of Some Constrained Channels A.V.Kuznetsov and A.J. Han Vinck, IEEE Trans. on Information Theory, Vol. 40, November 1994, pp. 1866‐1871. 14 WIC meeting Gent May 2019 2 examples: problem to find codes/coding methods Example 1: 1 defect ( 1 ) ‐ store x or its complement x Example: ‐ efficiency: (n‐1)/n = 1 –1/n = optimum! Defect‐0 in 2nd position 000 111 001 110 010 101 100 011 Example 2: (n‐1) defects ( 1 0 1 1 ? 0) ‐ 1 position left to change the word parity ‐ efficiency: (1/n) = 1 ‐ (n‐1)/n = optimum ! In General: Linear binary codes can be used. (MDS, or RS (dmin = n‐k+1) codes are optimum) WIC meeting Gent May 2019 15 An application: Writing more than once in a write‐once memory! FLASH: we can change 0 into 1 only. • First writing: write words with pn „1“s 0 0 1 0 0 0 1 0 • write nh(p) bits pn defects 0 0 1 0 0 0 1 0 • Second writing: consider „1“s as fixed (1‐p)n „free“ positions • write (1‐p)n bits Sum rate: { h(p) + (1‐p) } log23 = 1.58 bits/memory cell We cannot do better! WIC meeting Gent May 2019 16 Writing twice: 2 bit in 3 cells (R = 67%) • First writing 2 bits: 000 001 010 100 0 1 2 3 After inspection of the content, write again 2 bits • Second writing: 0 111 111 111 111 1 110 001 110 110 2 101 101 010 101 3 011 011 011 100 Uniquely direct decodable: unique rows! Efficiency: R = 2 bits /3 cells = 2/3. At the cost of only 1 additional read! WIC meeting Gent May 2019 17 General result for q‐ary cells and T writings –before erase‐: T = 1 2 3 4 … Level Example: q = 4 Increment any step size 0 1 2 3 • Capacity: log2|C| ≈ (q‐1) log2 (1 + T/(q‐1)); for q =2, |C| = log2(T+1) On the Capacity of Generalized Write‐Once Memory with State Transitions Described by an Arbitrary Directed Acyclic Graph Fang‐Wei Fu and A. J. Han Vinck, IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 1, JANUARY 1999 WIC meeting Gent May 2019 18 q‐ary flash: different type of errors need for special codes • Writing errors (incorrect level) • Leaking cells • Interference • Cell Wear 3214 5 4 error ‐1 +1 Codes over the Ring of Integers Modulo, IEICE Transactions, November 1998 pp. 2013–2018, (A.J. Han Vinck and Hiro Morita) Perfect (d,k)‐Codes Capable of Correcting Single Peak‐Shifts, IEEE Trans. on Inf. Th., pp. 656‐662, March 1993, (V.I. Levenshtein and A.J. Han Vinck) WIC meeting Gent May 2019 19 Applications using error correcting codes (ECC)– coding techniques • Write‐once memories (WOM): FLASH. Reuse old locations • Principle: only use clean pages. Delay erase cycle (expensive in time + ..) • Use multi level Flash: more bits per cell. Write more than once! • Problems: leaking, overshoot. Use of special ECC, integer codes. • Capacity improvements? • Phase change: much better performance. Use defect codes • Problems: do not overheat. Use fraction p of the storage capacity • Writing capacity? • Write unidirectional: Technical writing constraints. Needs special coding techniques • Problem: calculating capacity. WIC meeting Gent May 2019 20 Multi user theory in Eindhoven • Frans Willems as staff member from Leuven (vdMeulen) • Example: Coding for Optical Rewriting Disk (Sony) After a visit to the Walhalla at TUE About 40 years At Philips research: try to prove that .5 efficiency is optimal good colleagues We showed the .51 can be reached in a simple way Coding problem for higher efficiency still open. "Repeated recording for an optical disk", Proc. 7th Symp. Information Theory in the Benelux, pp.49 ‐53 1986 WIC meeting Gent May 2019 21 Recent: Inf. Th. applications of defect matching codes: old stuff in new jacket Fraction T position in a word should not be changed: too hot Fraction W changes in the rest are allowed To be published IEEE IT transactions Capacity: (1‐T)h(W) for W = .5, we have (1‐T) To be published IEEE IT transactions, and ISIT 2019 Paris ex: for q‐ary cells, maximum change of charge < q WIC meeting Gent May 2019 22 Single cell storage: one step‐up strategy average increase 2p(1‐p) per writing q 1 • Example: average number of writes to reach top level T 2p(1 p) Input Sequence 7 = 1 6 = 0 1, 0, 1, 0, 0, 0, 1, 1, 0, 1 5 = 1 4 = 0 1, 2, 3, 4, 4, 4, 5, 5, 6, 7 3 = 1 Level 2 = 0 1 = 1 0 = 0 8 level flash T q 1 the # of bits stored is : log 2 log 2 (T 1) pT 2 which is 50% of the capacity! WIC meeting Gent May 2019 23 Our research approach: Find appropriate model ( use of the Walhalla) Find codes/coding techniques Calculate capacity/efficiency Use un‐ or informed writer/reader WIC meeting Gent May 2019 Often very unpractical! Memory research examples: • Magnetic optical rewriting: Write Unidirectional Codes (WUM) • Peak‐shift error correcting codes: especiall in magnetic, but also in flash • Life time prediction for memories with defects: improvement from coding? 24 One more picture WIC meeting Gent May 2019 25 1990, IEEE IT workshop Paul de Bot, Ineke van Overveld, Han Vinck Door „gebrek“ aan mankracht naar ULM 1997 in Duitsland WIC meeting Gent May 2019 26 The first Japan Benelux Workshop on IT, 1989 now Asia‐Europe Workshop (AEW), AEW‐11 in R‘dam, July 3‐5, 2019 WIC meeting Gent May 2019 27 Large non‐volatile memories (RAM) are based on the principle of a variable resistor NEW RAM MEMORY TYPES: CHANGE RESISTOR Flash: change FET resistance Phase Change RAM: heating DRAM, capacitor load Magnetic RAM: magnetization charge WIC meeting Gent May 2019 28 Example: Flash memory.