IEEE Information Theory Society Newsletter

Vol. 67, No. 2, June 2017 EDITOR: Michael Langberg, Ph.D. ISSN 1059-2362 Editorial committee: Alexander Barg, Giuseppe Caire, Meir Feder, Joerg Kliewer, Frank Kschischang, Prakash Narayan, Parham Noorzad, Anand Sarwate, Andy Singer, and Sergio Verdú.

President’s Column Rüdiger Urbanke

Some of you might have noticed a monthly itsoc.org/news-events/recent-news/call-for- email from ITSOC in your mail box with the short-video-proposal table of contents of the upcoming IT Transac- tions (if not, check you spam filter and make Talking about movies. Mark Levinson, the di- sure you paid your dues!). This was suggest- rector of the Shannon movie, has started edit- ed by Prakash Narayan, our Editor-in-Chief. ing the material from the February shoot and Thanks also to Elza Erkip, our First Vice Presi- is making plans for additional shooting of dent, for her support, and to Anand Sarwate, flashback scenes and interviews. He has also the Chair of the Online Committee, for posting begun speaking to a composer and graphics this information on our web page. We hope people. The role of Shannon has already been that you like the idea. The early feedback has cast—sorry, you can stop practicing juggling. been very positive. But we can do even better. But if you have ideas about how best to ex- Dave Forney has suggested that we bring the plain central concepts of information theory format into the 21st century and perhaps com- to a large audience, let us know. bine with regular communication to our members. After playing around with various HTML Unfortunately, I also have to bring you very versions and mailers all I can say: Who knew that spamming sad news. Mary Elizabeth (Betty) Moore Shannon, the re- could be so complicated! Please keep the comments coming. markable wife of Claude, has died. You can find a reprint of the obituary that has appeared in the Boston Globe and online On a related note, if you are looking for papers from our com- in the New York Times in this Newsletter. munity make sure that once in a while you check out Xplore. And if you are writing a paper ensure that you also include Besides the Shannon movie, a Shannon biography, authored the reference to the final version of the paper. This is a small by Jimmy Soni and Rob Goodman, has just been published. step for any author, but has potentially a large impact for our You can order it at https://www.amazon.com/Mind-Play- society since it will ensure that our click rates adequately re- Shannon-Invented-Information/dp/1476766681 flect our work and contributions. Read it, and if you enjoy it, recommend it to your friends, like As part of the Shannon Centennial Celebration, the IEEE In- it, and up-vote it. An enthusiastic support by our community formation Theory Society launched last year a pilot project to will help push the book onto bestseller lists, boosting our out- create short videos. The objective was to showcase intriguing reach efforts. results from Information Theory (in a broad sense) to a large non-expert audience. Two videos, one on Network Coding And if you want to do more, follow Christina Fragouli’s sug- and one on Space Time codes, have almost been completed. gestion. Start your Christmas shopping early this year and get a We have decided to extend this project and we are now look- few copies for your loved ones or anybody who is digitally chal- ing for volunteers to help define and create the next three vid- lenged but not entirely hopeless. Want more ideas? How about eos! If there is a concept or an idea, rooted in information the- donating a copy to your local library or placing a copy in the ory, that you think the whole world should know about, this waiting room of a train station or dentist office. You never know is your chance. A call for video proposal has been released where the next Claude will pass by, looking for inspiration. on the ITSOC website with more information. We look forward to your creative ideas and suggestions. http://www. (continued on page 10) 2 From the Editor Michael Langberg

Dear colleagues, fellow colleagues for their outstanding research accom plish- ments and service recognized by our community. Spe- This issue opens with an intriguing survey cifically, congratulations to Ido Tal and Alexander Vardy article by Adam Smith, “Information, on their 2017 IEEE Communications Society and Informa- Privacy and Stability in Adaptive Data tion Theory Society award winning paper “List Decoding Analysis,” addressing the subtleties in of Polar Codes,” and to the members of our society that data analytics when collected data is re- have recently been named Distinguished Lecturers. We used. The survey connects the study of continue with a special “Behind the scenes Q&A” with adaptive data analytics with certain in- the authors of the upcoming Claude Shannon biography. formation measures and measures of dis- Thanks to authors Jimmy Soni and Rob Goodman for pre- tributional stability. The latter, termed paring the Q&A and for pursuing this impressive and im- “differential privacy,” has seen a sig- portant project. nificant amount of interest lately and has recently awarded Adam Smith (together We conclude with Tony Ephremides’s Historian’s column; our “Students’ Corner’’ with Cynthia Dwork, Frank McSherry, column, by Mine Alsan and Basak Güler, presenting the recent initiatives of the In- and Kobbi Nissim) with the prestigious formation Theory Student Subcommittee; and a note by Emanuele Viterbo and Elza Gödel Prize on their outstanding work Erkip, “Thinking about organizing an ITW or ISIT?,” outlining the ins and outs of introducing differential privacy. Many organizing a workshop or conference in our society. Thanks to the contributors for thanks to Adam Smith for his generous their efforts. willingness and significant efforts in pre- paring the survey. Continuing our remembrance and honoring of Sol Golomb, an extraordinary scholar and long time newsletter contributor, the fourth and final collection of Sol’s earlier The issue continues with a number of newsletter puzzle columns appears in this issue. This last collection includes solu- announcements, regular columns, and tions to the second collection of puzzles published in the December 2016 issue of the reports. We start by congratulating our newsletter.

continued on page 15

IEEE Information Theory Table of Contents Society Newsletter President’s Column...... 1 IEEE Information Theory Society Newsletter From the Editor ...... 2 (USPS 360-350) is published quarterly by the Information Theory Society of the Institute of Information, Privacy and Stability in Adaptive Data Analysis ...... 3 Electrical and Electronics Engineers, Inc. IEEE Communications Society and Information Headquarters: 3 Park Avenue, 17th Floor, New York, NY 10016-5997. Theory Society Joint Paper Award for 2017 ...... 11 Cost is $1.00 per member per year (included IEEE Information Theory Society’s New Distinguished Lecturers...... 11 in Society fee) for each member of the Information Theory Society. Printed in the Behind the Scenes: A Q&A with the Authors of Claude U.S.A. Periodicals postage paid at New York, NY and at additional mailing offices. Shannon’s New Biography ...... 11 Postmaster: Send address changes to IEEE The Historian’s Column...... 13 Information Theory Society Newsletter, IEEE, 445 Hoes Lane, Piscataway, NJ 08854. Students’ Corner: The Information Theory Student © 2016 IEEE. Information contained in this Subcommittee Goes Social! ...... 14 newsletter may be copied without permission provided that the copies are not made or Thinking About Organizing an ITW or ISIT?...... 16 distributed for direct commercial advantage, and the title of the publication and its date Golomb’s Puzzle Column™ Collection, Part 4 ...... 17 appear. In Memoriam: Mary Elizebeth (Betty) Moore Shannon ...... 41 IEEE prohibits discrimination, harassment, and bullying. For more information, visit http://www.ieee.org/web/aboutus/ Nominations Sought for 2018 and 2019 Leaders ...... 41 whatis/policies/p9-26.html. Recent Publications...... 42

Promoting Sustainable Forestry Call for Papers ...... 47

SFI-01681 Conference Calendar ...... 52

IEEE Information Theory Society Newsletter June 2017 3 Information, Privacy and Stability in Adaptive Data Analysis Adam Smith*

Abstract To formalize our situation somewhat, imagine there is a population that we wish to study, modeled by a probability distribution P. Traditional statistical theory assumes that the analysis to be per- An analyst selects a sequence of analyses M1, M2, ..., that she wishes formed on a given data set is selected independently of the data to perform (we specify the type of analysis to consider later). In an themselves. This assumption breaks downs when data are re-used ideal world (Figure 1, left), the analyst would run each analysis Mi (i) across analyses and the analysis to be performed at a given stage on a fresh sample X from the population. For simplicity, we only depends on the results of earlier stages. Such dependency can arise discuss i.i.d samples in this article; we may assume that each sample (i) when the same data are used by several scientific studies, or when a X has n points, drawn independently from P. In the real (adaptive) single analysis consists of multiple stages. setting, the same data set X gets used for each analysis (Figure 1, right). The challenge is that we ultimately want to learn about P, How can we draw statistically valid conclusions when data are re- not X, but adaptive queries can quickly overfit to X. used? This is the focus of a recent and active line of work. At a high level, these results show that limiting the information revealed by Our goal is to relate these two settings—to develop techniques that earlier stages of analysis controls the bias introduced in later stages allow us to emulate the ideal world in the real one, and understand by adaptivity. how much accuracy is lost due to adaptivity. As mentioned above, merely setting aside a holdout to verify results at each stage is not Here we review some known results in this area and highlight the sufficient, since the holdout ends up being re-used adaptively. If the role of information-theoretic concepts, notably several one-shot no- number k of analyses is known ahead of time, one can split the data tions of mutual information. into k pieces of n/k points each (assuming i.i.d. data, the pieces are independent). This practice, called data splitting, provides clear va- 1 Introduction lidity guarantees, but is inefficient in its use of data: data splitting requires n to be substantially larger than k, while we will see tech- How can one do meaningful statistical inference and machine learn- niques that do substantially better. Data splitting also requires an ing when data are re-used across analyses? The situation is common agreed upon partition of the data, which can be problematic with in empirical science, especially as data sets get bigger and more com- data shared across studies. plex. For example, analysts often clean the data and perform various exploratory analyses—visualizations, computing descriptive A line of work in computer science [21, 28, 20, 19, 40, 41, 6, 39, statistics—before selecting how data will be treated. Many times 46, 23, 24] initiated by Dwork, Feldman, Hardt, Pitassi, Reingold, the main analysis also proceeds in stages, with some sort of feature and Roth [21] and Hardt and Ullman [28] provides a set of tools selection followed by inference using the selected features. In such and specific methodology for this problem. This article briefly settings, the analyses performed in later stages are chosen adaptively surveys the ideas in these works, with emphasis on the role of based on the results of earlier stages that used the same data. Adap- several information-theoretic concepts. Broadly, there is a strong tivity comes into even sharper relief when data are shared across mul- connection between the extent to which an adaptive sequence of tiple studies, and the choice of the research question in subsequent analyses remains faithful to the underlying population P, and the studies may depend on the outcomes of earlier ones. Adaptivity has amount of information that is leaked to the analyst about X. In been singled out as the cause of a “statistical crisis” in science [27]. particular, randomization plays a key role in the state of the art methods, with a notion of algorithmic stability—differential There is a large body of work in statistics and machine learning on privacy —playing a central role. preventing false discovery, for example by accounting for multiple hypothesis testing. Classical theory, however, assumes that the analy- Another approach, with roots in the statistics community, seeks to sis is fixed independently of the data—it breaks down completely model particular sequences of analyses, designing methodology to when analyses are selected adaptively. Natural techniques, such as adjust for the bias due to conditioning on earlier results (e.g., [36, 30, separating a validation set (“holdout”) from the main data set to 26, 22, 34, 32]). The specificity of this line of work makes it hard verify conclusions or the bootstrap method, do not circumvent the to compare with the more general approaches from computer sci- issue of adaptivity: once the holdout has been used, any further hy- ence. Other work in statistics hews an intermediate path, allowing potheses tested using the same holdout will again depend on ear- the analyst freedom within a prespecified class of analyses [7, 10]. lier results. Blum and Hardt [8] point out that this issue arises with There are intriguing similarities between these lines of work and the leaderboards for machine learning competitions: they observe that work surveyed here, such as the use of randomization to break up one can do well on the leaderboard simply by using the feedback dependencies (e.g., [42, 43, 29]); understanding these connections provided by the leaderboard itself on an adaptively selected sequence more deeply is an important direction for future work. of submissions—that is, without even consulting the training data! 2 The Lessons of Linear Queries *Computer Science and Engineering Department, Pennsylvania State University, University Park, PA, USA. [email protected]. Supported by A simple but important setting for thinking about adaptivity, in- NSF award IIS-1447700, a Google Faculty Award and a Sloan Foundation troduced by Dwork et al. [21] and Hardt and Ullman [28], is that of research award. an analyst posing an adaptively selected sequence of queries, each of

June 2017 IEEE Information Theory Society Newsletter 4

err,X MA=-max ziiP a . (1) i X M1 X M1 ^ h ^ h

P P which depends on X as well as the coins of M and A.

(2) X M2 M2 Definition 1. A query answering mechanism M is (ab, )-accurate on i.i.d. data for k queries if for every data analyst A and distribution P, we have Statistical ideal: Adaptive Fresh Sample Setting for Each Pr err,x MA#$ab1 - . (k) ^^ hh Analysis X Mk Mk + Figure 1. Ideally, we would collect fresh data from the The probability is over the choice of the dataset X iid.. . . P and the randomness of the mechanism and the analyst. Similarly, the expected same population P for each analysis (left). In many real settings, we have only a single data set that must error of M is the supremum, over distributions P and data analysts A, of E err,X MA . We sometimes fix the distribution P and take the be re-used, leading to adaptively selected analyses ^^ hh (right). The arrows pointing into the top of the analyses supremum only over analysts A. indicate that each analysis is selected based on the results of previous stages. It is important to note that this definition makes no assumptions on how the analyst selects queries, except that the selection is based on the outputs of M and not directly on the data. The aim of this which asks for the expectation of a bounded function in the popula- line of work is to design mechanisms with provable bounds on ac- tion. Such queries capture a wide range of basic descriptive statistics curacy. We aim for mechanisms that are universal, in the sense that (the prevalence of a disease in a population, for example, or the they can be used in any type of exploratory or adaptive workflow. average age). Many inference algorithms can also be expressed in terms of a sequence of such queries [31]; for example, optimization 2.1 Failures of Straightforward Approaches algorithms that query the gradient of a Lipschitz, decomposable loss function. As mentioned above, there are a couple of natural approaches to this problem. The first is to answer queries using each query’s Suppose each data point lies in a universe |, so that a data set empirical mean z X . When queries are specified nonadaptively, a n ^h lies in | and the underlying population is a distribution on |. standard argument shows that the population error of that strat- A bounded linear query is specified by a function z|: " 01, . The egy is H log kn population value of a linear query is simply the expected value6 @of the ` ^hj function when evaluated on an element of the data universe drawn In contrast, in the adaptive setting, the empirical mechanism’s er- 1 according to P, denoted z P = Ex + P z x . ror may be unbounded even with just two queries. For example, ^ h 6 ^ h@ the query z may be selected such that the low-order bits of z1 x ^h Consider now an interaction between an analyst wishing to pose reveal all the entries of the data set x. In that case, the analyst may such queries and an algorithm M (called the mechanism) holding a construct a query z2 which takes the value 1 for values in the data data set X sample i.i.d from P that attempts to provide approxi- set x, and 0 otherwise. The empirical mean z2(x) will be 1, while mate answers to the queries zz12, , .... This is illustrated in Figure 2, the population mean z2 P will be close to 0 for any distribution ^h where we use subscripts (as in M1, M2, ...) to distinguish k different P with sufficiently high entropy. rounds of M . In general, neither the mechanism nor the analyst knows the exact distribution P (otherwise, why collect data?), so This last example seems contrived, since it requires seemingly atyp- the mechanism cannot always answer z P . A natural approach ical structure from the initial query z . For example, constraining ^h 1 = is to answer with the empirical mean zzXn1 / ! Xi the queries z to be predicates taking values in {0, 1} seems to elimi- ^^hh^ h xi x When queries are selected nonadaptively, this is the best estima- nate the problem. However, the example is instructive for at least tor of z(P). We shall see, however, this is not the best mechanism two reasons. First, it illustrates the role that information about the for estimating the expectations of adaptively selected queries! data set can play: learning x allows the analyst to pose a query that is highly overfit to the data set, and thus difficult for the mechanism Given a query answering mechanism M , a data analyst A, and a to answer accurately. Conversely, we will see that limiting the in- distribution P on the data universe | , consider a random interac- formation revealed about the data strongly limits overfitting. tion defined by selecting a sample X of n i.i.d. draws from P, and then having A interact with M (X) for k rounds, where in each Second, when the analyst asks more queries, one can construct much round i, (i) A selects zi (based on a1, . . . , ai-1), (ii) M answers ai. more natural examples of analyses that go awry when using the em- The (population) error of M is the random variable pirical mean. For instance, consider a data set where each k-individual data point lies in | = {0, 1}k-1 × {0, 1}, where we think of the first k - 1 bits as a vector of binary features, and the last bit as a label. Consider a particular analyst (from [20]) aiming to find a good clas- 1The “linear” in “bounded linear query” refers to the fact that we care k - about the expectation of a function, so the resulting functional is a sification rule for the label. The analyst’s first 1 queries ask for the linear map from the set of distributions om X to [0, 1]. In contrast, success rate of each of the k - 1 features in predicting the label. In some statistics, such as the variance of a random variable, are not linear. the k-th query, the analyst constructs a classifier that takes a major- “Bounded” refers to the image being limited to [0, 1] (or, equivalently, ity vote among those features that had success rate greater than 50%. any other finite interval). On uniformly random data (where the label is independent of the

IEEE Information Theory Society Newsletter June 2017 5 features), the mechanism will report the success rate of this last classi- φ fier to be 55% when k = n 10 , and 67% when k = n (even though its 1 ^h X M1 success rate on the population would be 1/2). Generalizing the example a1 somewhat, one can show that even with very simple data distribu- P Analyst tions, the error of empirical mechanism scales as H kn—expo- i.i.d nentially larger than the error one gets with nonadaptively` jspecified Sample φ2 Probability M queries. Encapsulating this discussion, we have: from P 2 Distribution a2 Proposition 1. When answering k nonadaptively specified queries, the on Set X empirical mechanism has expected error H log kn When answer- ^h ing k adaptively selected queries, the empirical` mechanismj has expected error X kn, even for predicate queries on uniformly random data in `{0, 1}k. j φk Data Splitting Another natural approach for handling adaptively Mk a specified queries is data splitting : when k is known in advance, k one may divide the data set into k subsamples of n/k points each, Figure 2. Adaptively selected linear queries. and answer the i-th query using its empirical mean on the i-th data set. This approach means that we can truly ignore adaptivity and use all the tools of classical statistics; the downside is that we are The ideas underlying the two previous algorithms can also be adapt- limited to the accuracy one can get with sample size n/k. The fact ed to give better results when we make further assumptions about that we want a bound that is uniform over all k queries adds a the class of allowed queries, or the universe from which the data are further logarithmic factor to the final error bound: drawn. One such result, due to Dwork et al. [21] (and tightened in [6]), recovers a logarithmic dependence on k in exchange for a depend- Proposition 2. When answering k adaptively specified queries, the data ence on the size of the universe | in which the data lie. splitting mechanism has expected error H kklog n . ` ^hj Theorem 5 ([21, 6]). There is a computationally inefficient mecha- For both of these natural mechanisms, answering queries with nism with expected error Ok6 logl| 3 og n . The mechanism ^h error a, even with constant probability, requires n to grow at least runs in time linear in || | ( a` n d n o t log || | as onej would naturally as fast as k a2 . Can we do better? How good a dependency on a want). and k is possible? None of these upper bounds is known to be tight in all parameter 2.2 A Sample of Known Bounds regimes, but some lower bounds are known, in particular showing that the scaling n = Ω ( k ) cannot be improved, and that inef- In fact, there are mechanisms that can answer a sequence of k adap- ficiency of the mechanism in Theorem 5 is necessary. tively selected linear queries with much higher accuracy than that provided by the straightforward approaches. Namely, for a given Theorem 6 (Hardt and Ullman [28], Steinke and Ullman [41]). accuracy a, we can get mechanisms that work for n that scales For every mechanism M that an swers k adaptively selected linear que- only as k a2 —a quadratic improvement in k. The bounds be- ries, for a sufficiently large universe | (with log ||| exponential low are stated in terms of expected error for simplicity; the un- in n), there exist a distribution P and an analyst A for which the mechanism’s error err,x MA is X kn with constant probabil- derlying arguments also provide high-probability bounds on the ^h^h tail of this error. ity. Furthermore, for mechanisms that answer faithfully with respect to both the distribution and the data set (that is, they provide answers Theorem 3 ([21, 6]). There is a computationally efficient mechanism for close to both zi X and zi (P)), the bound can be strengthened to 4 ^h k statistical queries with expected error O 4 kn/ . X k n ^h ^h A simple mechanism that achieves this bound is one that adds Gauss- Finally, if we assume that one-way functions exist, then the bounds con- ian noise with standard deviation about 4 kn/ to each query. tinue to hold when log ||| has polynomial size, for polynomial-time mechanisms (but not for those that can take exponential time). One can give a different-looking mechanism—which we do not describe in this survey—to automatically adjust to the actual We won’t discuss the proof of these lower bounds here, but we “amount” of adaptivity in a given sequence of queries. Specifi- note that closing the gap between the upper and lower bounds cally, imagine that the k queries are grouped into r batches, where remains an intriguing open problem. the queries in a given batch depend on answers to queries in previous batches but not on the answers to queries in the same batch. For 2.3 Privacy and Distributional Stability example, in the classification example of the previous section, the number of rounds r is only 2. The upper bounds above are obtained via a connection between adaptive analysis and certain notions of algorithmic stability. Theorem 4 ([21]). If there are at most r rounds of adaptivity, then Broadly, algorithmic stability properties limit how much the out- there is a computationally efficient mechanism with expected error put of an algorithm can change when one of its inputs is changed. O rklog n The algorithm is not given the partition of the queries Different notions of stability correspond, roughly, to different ` ^hj into batches. measures of distance between outputs. There is a long-standing

June 2017 IEEE Information Theory Society Newsletter 6 connection between algorithmic stability and expected generalization error (e.g., Devroye and Wag ner [14], Bousquet and Elisseeff a X M [9]). Essentially, stable algorithms cannot overfit. It seems that if we could design adaptive query-answering mechanisms that are sta- Analyst ble in an appropriate sense, we could get validity guarantees for P φa adaptive data analysis. Figure 3. A two-stage overfitting game. Alas, there is a hitch. Recall that our goal is to design mechanisms that provide statistically valid answers no matter how the analyst selects queries. Even if each stage of the mechanism is stable, the Theorem 9 (Main Transfer Theorem [21, 6]). Suppose a statistical overall process might not be—in an adaptive setting, the analyst estimator M is (ff, $ d)-accurate with respect to its sample, that is, for ends up being part of the mechanism. all data sets x,

The resolution is to consider a distributional notion of stability. We Pr max aii--zdx #$e 1 e $ . will require that changing any single data point in x have a small ` i ^h j effect on the distribution of the mechanism’s outputs. If we choose a distance measure on distributions that is nonincreasing under If M is also (e,e · d)-differentially private, then it is (O (e ), O (d))- postprocessing, then we can limit the effect of the analyst’s choices. accurate with respect to the population (Definition 1).

Specifically, we work with “differential privacy,” a notion of sta- This theorem underlies two of the three upper bounds of the pre- bility introduced in the context of privacy of statistical data. Dif- vious section (Theorems 3 and 5). Each is derived by using exist- ferential privacy seeks to limit the information revealed about any ing differentially private algorithms together with Theorem 9. For single individual in the data set. Theorem 4, Dwork et. al. [21] used a different argument, based on compressing the output of the algorithm to a small set of possibili- Definition 2 ([17, 16]). An algorithm M:|n " O is e, d -differen- ties; see Section 3. ^hn tially private if for all pairs of neighboring data sets x,xl ! | , and for all events S 3 O: 2.4 A Two-stage Game, Stability and “Lifting”

We conclude this section with an outline of the proof of the main Pr MSx.!#expPe r MSxl ! + d 6 ^ h @ ^ h 6 ^ h @ transfer theorem (Theorem 9). That theorem talks about, analyses with many stages of interaction, but it turns out that the core of the Differential privacy makes sense even for interactive mechanisms argument lies in understanding a seemingly much simpler, two- that involve communication with an outside party: we simply stage process. think of the outside party as part of the mechanism, and define the final output of the mechanism to be the complete transcript of Consider a two-stage setting in which an analysis M is run on data the communication between the mechanism and the other party. set x, and the analyst, selects a linear query z : | " [0, 1] based on M(x) (Figure 3). We say M robustly (a, b)-generalizes if for all Differential privacy is a useful design tool in the context of adaptive distributions P over the domain |, for all strategies (functions) A data analysis because it is possible to design interactive differen- employed by the analyst, with probability at least 1 - b over the tially private algorithms modularly, due to two related properties: choice of X ~ P n and the coins of M, we have that zzX - P # a ^ h ^ h closure under postprocessing, and composition: where z = A(M(X)). (Similarly, we may talk about, the expected generalization error, that is, the maximum over A and P of Proposition 7. If M:|n " O is ( e, d)-differentially private, and f:O " Ol E zzX.- P ^ ^ h ^ h hh is an arbitrary (possibly randomized) mapping, then fM% :|n " Ol is (e, d )-differentially private. The quantification over all selection functions A here is critical— when the first, phase of analysis satisfies the definition, then a

Proposition 8 (Adaptive Composition [18, 35]—informal). Let M1, query asked in the following round cannot, overfit to the data, M2, ..., Mk be a sequence of (e, d)-differentially private algorithms that are (except, with low probability), no matter how it is selected. all run on the same data set, and selected adaptively (with the choice of Mi depending on the outputs of M1, ..., Mi-1, but not directly on x). Then no Differential privacy (and a few other distributional notions of stabil- matter how the adaptive selection is done, the resulting composed process ity, such as KL-stability [6, 47]) limits the adversary’s score in this is (el, d’)-differentially private, for eel . k and ddl . k . game. This connection had been understood for some time— for example, McSherry observed that it could be used to break up depend- Taken together, these two properties mean that in order to design encies in a clustering algorithm, and Bassily, Smith, and Thakurta differentially private algo rithms for answering linear queries, it [5] used a weak version of the connection to bound the population is sufficient to make sure the mechanism run at each stage is dif- risk of differentially private empirical risk minimization. ferentially private. However, the application to adaptive data analysis-and especially Perhaps even more importantly, in order to ensure statistical va- the understanding of the importance of post-processing to the de- lidity—that is, accuracy with respect to the underlying popula- sign of universal mechanisms—came recently in [21]. Their initial tion—it suffices to design differentially private algorithms that are result was subsequently sharpened, to obtain the following tight accurate with respect to the sample x: connection:

IEEE Information Theory Society Newsletter June 2017 7

Theorem 10 (Differentially Private Algorithms Cannot Overfit [6]). φ1 x M If M is (e, ed)-differentially private, then it is (O(e), O(d))-robustly 1 a 1 1. Find i ∗ generalizing. n P ~P = argmaxi |ai – φi(P)| φ 2. Return φ Lifting to Many Stages Bounds on the two-stage game can be M 2 i 2 a “lifted” to provide bounds on the k-phase game either through a 2 sequential application of the bound to each round [21] or through φi ∗ a more holistic argument, called the monitor technique [6], that P yields Theorem 9 (and Theorems 3 and 5). φk M k a The monitor argument is a thought experiment—we argue that k for any multi-stage process, there is a two-stage process in which Figure 4. The “monitor” argument for lifting results about the error on the population equals the maximum population error a two-stage game to many stages. over all k stages of the original process. The argument applies quite generally, but it is a bit simpler to explain under the assumption that the mechanism answers queries accurately with respect to the M(X), the probability that the test rejects the null hypothesis given data set X. The idea, given an interaction between an analyst and X might be much higher than c even if P lies in H. adaptively selected mechanisms M1, M2 ..., Mk, is to encapsulate the analyst and mechanisms into a single fictional entity M which gets, How much higher it can be depends on M and—as we will see— as additional input, the underlying distribution P. The fictional M on several measures of the information leaked by M. To formalize executes an interaction and then outputs a single query z*—the this, consider a game similar to the overfitting game, in which the one which maximizes the population error over all stages i. analyst A, given Y = M(X), selects an arbitrary event TY = A(Y) (which depends on Y). For a particular output Y of M, the analyst’s Beyond Linear Queries The techniques described in this section “score” is extend to problems that are not described by estimating the mean scoreP,,MA,c y of a bounded linear functional. One important class is minimizing ^ h PrXX++PPnnXX!!TTyyYy= if Pr # c, a decomposable loss function where each individual contributes a = ^ h ^ h )0 if PrX+ Pn X ! Ty 2 c. bounded term to the loss function [6]. ^ h

A Re-usable Holdout The techniques described in this section can Now consider the analyst’s expected score in this game: hcPP,,MA = E score ,,MA y . As we will see below, the analyst’s appear somewhat onerous for the analyst, since they require ac- ^ h Y ^^hh cessing data via differentially private algorithms. As pointed out by score in this game can be bounded using various definitions of the Dwork et al. [20], however, one need not limit, access to the entire information leaked about X by Y. This score also plays a key role in data set in this way. In fact, a more pragmatic approach is to give controlling false discovery: most, of the data, “in the clear” to the analyst, and protect, only a small holdout set via the techniques discussed here This still allows Proposition 11. Bounding the score η has several important implications: one to verify conclusions soundly, but additionally allows full explor- 2 atory analysis, as well as repeated verification (“holdout re-use”). 1) (False discovery [39]) If Y = M(X) is used by A to select a hypothesis test with signifi cance c, then the probability of false discovery is at 3 The Intrigue of Information Measures most hP,,MA. 2) (Robust generalization [19]) If Y = M(X) is used by A to se- Despite the generality of the approach of the previous section, lect a bounded linear query zc, then zzYYXPr - P $ tn many important classes of analyses are not obviously amenable to - t2 3 ^ ^ h ^ h h # hP,,MA e those techniques; in particular, problems that are not easily stated ^h in terms of a numerical estimation task. We are interested in universal bounds that hold no matter how the analyst uses the output Y, and no matter the original input distri- Consider the problem of hypothesis testing. Crudely, given a set bution. To this end, we define of distributions H (called the null hypothesis), we ask if the data set is “unlikely to have been generated” by a distribution P ! H . hcM = sup hcP,,MA More precisely, we select an event T (the acceptance region) such ^ hhP,A ^ ^ h that PrX + Pn X ! T is at most a threshold c (often 0.05) for all dis- ^h tributions in H. If it happens that the observed data X lie outside 3.1 Information and Conditioning of T, the null hypothesis is said to be rejected. If this happens when the true distribution is actually in , then we say a false dis- P H The function hΜ measures how much probabilities less than c can covery occurs. Hypothesis tests play a central role in modern em- be amplified by conditioning on Y, on average over values of Y. pirical science (for better or for worse), and techniques to control false discovery in the classic, nonadaptive setting are the focus of For several one-shot notions of mutual information, we have that if k intense study. Despite this, very little is known about hypothesis a procedure leaks k “bits” of information we have hcP,,MA . c $ 2 . tests in adaptive settings. ^h

Adaptivity arises when the event T is selected based on earlier 2The definitions here differ somewhat from those of [39]; in particular, our analysis of the same data—conditioned on those earlier results Y = function h is the inverse of a “p-valuecorrection function” from [39].

June 2017 IEEE Information Theory Society Newsletter 8

Unfortunately, such a clean relationship is not known for the stand- Proposition 13 ([40, 39]). If I (X; M (X)) < k, then (i) for every analyst A, hp,M,A(c) # k + 11log2 c , and (ii) for every b > 0, M ard notion of Shannon mutual information. Instead, we consider ^ ^ hh robustly (a, b)-generalizes for ab= Okn on P. two other notions here. ` j

Fix two random variables X, Y with joint distribution given by pXY 3.2 What procedures have bounded one-shot (x, y) = Pr(X = x,Y = y) and marginals pX(·) and pY(·). Consider the information measures? information loss The information-theoretic framework of the previous subsection pxXY ,y Ixy, = log ^ h . captures several other classes of algorithms that satisfy robust e pxXYpyo ^ ^h h generalization guarantees. In addition to unifying the previous The standard notion of mutual information is the expectation of work, this approach shows that these classes of algorithms allow for principled post-selection hypothesis testing. this variable: I(X; Y) = E (IX,Y). The max-information [12, 19] between X and Y is the supremum of this variable. Unfor tunately, for many interesting procedures, the max-information is either unbounded The most important of these, currently, is for the class of differen- or much larger than the mutual information. tially private algorithms:

Theorem 14 (Informal, see [39]). If M is ( ,de )-differentially private, One can get a more flexible notion by instead considering a high- b 2 and the entries of X are independent, then IM3 XX; = One for probability bound on the information loss: we say the β-approximate ^^hh ^ h b bd= On e . max information between X and Y is at most k (written IX3 :Yk# ) ^h 3,4 ^h if Pr xy,,+ xy Ikxy, #$1 - b. ^^hh^ h This result, together with Theorem 12, implies that differentially- private algorithms are OOee, n d -robustly generalizing for For many algorithms of interest, the approximate max-informa- ^^hh^ tion turns out to be very close to the mutual information but, by data drawn i.i.d from any distribution P. It essentially recovers providing a bound on the upper tail of Ix,y, allows for more precise the results of the previous section on linear queries (with a worse control of small-probability events. value of b), but additionally applies to more general problems such as hypothesis testing. We can also define a related quantity, which we call the expected log-distortion: Description length [19] In many cases, the outcome of a statistical analysis can be compressed to relatively few bits—for example, XY Ixy, px,y when the outcome is a small set of selected features. If the output LX3 ;Y ==logsE up2 logsE up ^ h · ^ h yY++x ^ h yYe x pxXYpyo of M can be compressed to k bits, then the expected log-distortion ` j ^ h ^ h L3 (X; M(X)) is at most k bits. An argument along these lines was This notion of information leakage is not symmetric in X, Y. It is used implicitly in [21] to prove Theorem 4. closely related to, but in general different from, the min-entropy leakage HY33XH- X [15, 1, 25, 2, 3]. Compression Schemes Another important class of statistical ^^hh analyses that have good (and robust) generalization properties Of these notions, expected log distortion is the strongest are compression learners [33]. These process a data set of n points since it upper bounds the other two: I(X; Y) # L3 (X; Y) and to obtain a carefully selected subset of only k << n points, and b IX3 ;;YL# 3 XY+ log 1 b . finally produce an output fit to those points. A classic example is ^^hh^ h support vector machines: in d dimensions, the final classifier is Theorem 12 For every mechanism M and distribution P s.t. X + P: determined by just d + 1 points in the data set. b k 1) If IY3 X; # k, then for every analyst A, hP,,AM cc# $ 2 + b, ^h ^h and M robustly (α,2β)-generalizes for ab=+kn21ln Cummings, Ligett, Nissim, Roth, and Wu [13] used classic gener- on P. ^^hh alization results for such learners to show that they satisfy robust generalization guarantees. The classic results as well as those of k 2) If L3 (X ; M(X)) # k, then for every analyst A, hcP,,AM # c $ 2 , Cummings et al. [13] can be rederived from the following lemma ^h and for every β > 0, M robustly (α,β)-generalizes for (new, as far as we know) bounding the information leaked by a ab=+kn21ln on P. compression scheme about those points that are not output by the ^^hh scheme. We know much weaker implications based only on bounding the nk mutual information. Most significantly, the bounds for general hy- Lemma 15 (Compression schemes). Let M : XX" be any algo- pothesis testing are exponentially weaker than those one gets from rithm that takes a data set X of n points and outputs a subset xout = M nk- the one-shot measures above (x) of k points from x. Let xin 2 ! X denote the remaining data points, so that xin , xout = x (as multisets). For any distribution P on X , if X + Pn 3This condition is not exactly the definition of max-information of [19] , then (which requires that for all events E, Pr((X,Y) ! E) ≤ 2k Pr((Xl, Y ) ! E) n where Xl is identically distributed to X but independent from Y). The LX3 ino;lX ut # og2 . ^ h ck m definition here implies that of [21]. 4The b-approximate max information is equivalent to a smoothed version of max-information [37, 38, 44, 45, 12], in which we ask that the pair X, Y be Cummings et al. [13] used the robust generalization properties b within statistical distance b of a joint distribution with IY3 X; # k. See of compression learners to give robustly generalizing algorithms Corollary 8.7 in Bun and Steinke [11] for details. ^h for learning any PAC-learnable concept class. In particular, this

IEEE Information Theory Society Newsletter June 2017 9 implies robustly generalizing algorithms for tasks that do not [8] A. Blum and M. Hardt. The ladder: A reliable leaderboard for have differentially private algorithms, such as learning a thresh- machine learning competitions. In Proc.32 nd International Confer- old classifier with data from the real line. ence on Machine Learning, 2015. arXiv:1502.04585.

Notes and Acknowledgments [9] O. Bousquet and A. Elisseeff. Stability and generalization. Journal of Machine Learning Research, 2: 499–526, 2002. I am grateful to many colleagues for introducing me to this topic and insightful discussions. Among others, my thanks go to Raef [10] A. Buja, R. Berk, L. Brown, E. George, E. Pitkin, M. Traskin, Bassily, Cynthia Dwork, Moritz Hardt, Kobbi Nissim, Ryan Rog- L. Zhao, and K. Zhang. Models as approximations—a conspiracy ers, Aaron Roth, Thomas Steinke, Uri Stemmer, Om Thakkar, Jon of random regressors and model deviations against classical infer- Ullman, Yu-Xiang Wang. Paul Medvedev provided helpful com- ence in regression. Statistical Science, 1460, 2015. ments on the writing, making the survey a tiny bit more accessible to nonexperts. Finally, I thank Michael Langberg, this newsletter’s [11] M. Bun and T. Steinke. Concentrated differential privacy: Sim- editor, for soliciting this article and graciously tolerating delays in plifications, extensions, and lower bounds. In Theory of Cryptography its preparation. Conference (TCC) 2016-B, 2016. arxiv:1605.02065.

Although I tried to cover the main ideas in a recent line of work, [12] N. Ciganovic, N. J. Beaudry, and R. Renner. Smooth max- that line is now diverse enough that I could not encapsulate information as one-shot generalization for mutual information. everything here. Notable omissions include the “jointly Gauss- IEEE Trans. Information Theory, 60(3):1573–1581, 2014. ian” models of Russo and Zou [40] and Wang et al. [46], which assume that the analyst is selecting among a family of statis- [13] R. Cummings, K. Ligett, K. Nissim, A. Roth, and Z. S. Wu. tics that are jointly normally distributed under P, work of El- Adaptive learning with robust generalization guarantees. In 29th An- der [23, 24] on a Bayesian framework that encodes a further nual Conference on Learning Theory, pages 772–814, 2016. restriction that the mechanism “know as much” as the analyst about the underlying distribution P, and the “typical stabil- [14] L. Devroye and T. Wagner. Distribution-free performance ity” framework [4, 13].There are no doubt other contributions bounds for potential function rules. IEEE Transactions on Informa- that I lost in my effort to consolidate. Some ideas in this survey, tion Theory, 25(5):601–604, 1979. notably the information-theoretic viewpoint on compression- based learning, have not appeared elsewhere; they have not been [15] Y. Dodis, R. Ostrovsky, L. Reyzin, and A. Smith. Fuzzy extrac- peer-reviewed. tors: How to generate strong keys from biometrics and other noisy data. SIAM J. Comput., 38(1):97–139, 2008. References [16] C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. [1] M. S. Alvim, K. Chatzikokolakis, C. Palamidessi, and G. Smith. Naor. Our data, ourselves: Privacy via distributed noise genera- Measuring information leakage using generalized gain functions. tion. In Advances in Cryptology - EUROCRYPT, pages 486–503, St. Pe- In 25th IEEE Computer Security Foundations Symposium (CSF), pag- tersburg, Russia, 2006. es 265–279, 2012. [17] C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating [2] M. S. Alvim, K. Chatzikokolakis, A. McIver, C. Morgan, C. noise to sensitivity in private data analysis. In Theory of Cryptography Palamidessi, and G. Smith. Additive and multiplicative notions Conference, pages 265–284. Springer, 2006. of leakage, and their capacities. In 27th IEEE Computer Security Foundations Symposium (CSF), pages 308–322, 2014. [18] C. Dwork, G. N. Rothblum, and S. Vadhan. Boosting and differential privacy. In Proceedings of the 2010 IEEE 51st Annual Sym- [3] M. S. Alvim, K. Chatzikokolakis, A. McIver, C. Morgan, C. posium on Foundations of Computer Science, FOCS ‘10, pages 51–60, Palamidessi, and G. Smith. Axioms for information leakage. In Washington, DC, USA, 2010. IEEE Computer Society. 29th IEEE Computer Security Foundations Symposium (CSF), pages 77–92, 2016. [19] C. Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, and A. Roth. Generalization in adaptive data analysis and holdout [4] R. Bassily and Y. Freund. Typical stability. arXiv:1604.03336 [cs. reuse. In Advances in Neural Information Processing Systems, pages LG], April 2016. 2350–2358, 2015.

[5] R. Bassily, A. Smith, and A. Thakurta. Private empirical risk mini- [20] C. Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, and mization: Efficient algorithms and tight error bounds. In IEEE Sympo- A. Roth. The reusable holdout: Pre- serving validity in adaptive data sium on the Foundations of Computer Science (FOCS), pages 464–473, 2014. analysis. Science, 349(6248):636–638, 2015.

[6] R. Bassily, K. Nissim, A. Smith, T. Steinke, U. Stemmer, and [21] C. Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, and J. Ullman. Algorithmic stability for adaptive data analysis. In A. L. Roth. Preserving statistical validity in adaptive data analy- 48th Annual ACM SIGACT Symposium on Theory of Computing, sis. In Proceedings of the Forty-Seventh Annual ACM on Symposium on pages 1046–1059, 2016. Theory of Computing, pages 117–126. ACM, 2015.

[7] R. Berk, L. Brown, A. Buja, K. Zhang, and L. Zhao. Valid post- [22] B. Efron. Estimation and accuracy after model selection. Journal selection inference. The Annals of Statistics, 41(2):802–837, 2013. of the American Statistical Association, 109(507):991–1007, 2014.

June 2017 IEEE Information Theory Society Newsletter 10

[23] S. Elder. Challenges in bayesian adaptive data analysis. arX- [36] B. M. P¨otscher. Effects of model selection on inference. Economet- iv:1604.02492, 2016. ric Theory, 7(2):163–185, 1991.

[24] S. Elder. Bayesian adaptive data analysis guarantees from sub- [37] R. Renner and S. Wolf. Smooth R´enyi entropy and applica- gaussianity. arXiv:1611.00065 [cs.LG], 2016. tions. In IEEE International Symposium on Information Theory — ISIT, page 233, 2004. [25] B. Espinoza and G. Smith. Min-entropy as a resource. Inf. Com- put., 226:57–75, 2013. [38] R. Renner and S. Wolf. Simple and tight bounds for information reconciliation and privacy amplification. In Advances in Cryp- [26] W. Fithian, D. Sun, and J. Taylor. Optimal inference after tology - ASIACRYPT, pages 199–216, 2005. model selection. arXiv preprint arXiv:1410.2597, 2014. [39] R. Rogers, A. Roth, A. Smith, and O. Thakkar. Max-informa- [27] A. Gelman and E. Loken. The statistical crisis in science. Amer- tion, differential privacy, and post-selection hypothesis testing. ican Scientist, 102(6):460, 2014. In Foundations of Computer Science (FOCS), 2016 IEEE 57th Annual Symposium on, 2016. [28] M. Hardt and J. Ullman. Preventing false discovery in interactive data analysis is hard. In Foundations of Computer Science [40] D. Russo and J. Zou. Controlling bias in adaptive data analysis (FOCS), 2014 IEEE 55th Annual Symposium on, pages 454–463. using information theory. In 19th International Conference on Artificial IEEE, 2014. Intelligence and Statistics, pages 1232–1240, 2016. arXiv:1511.05219.

[29] X. T. Harris, S. Panigrahi, J. Markovic, N. Bi, and J. Taylor. Selec- [41] T. Steinke and J. Ullman. Interactive fingerprinting codes and tive sampling after solving a convex problem. arXiv: 1609.05609, 2016. the hardness of preventing false discovery. In Proceedings of The 28th Conference on Learning Theory, pages 1588–1628, 2015. [30] C. M. Hurvich and C.-L. Tsai. The impact of model selection on inference in linear regression. The American Statistician, 44(3):214–217, [42] X. Tian and J. E. Taylor. Selective inference with a randomized 1990. response. arXiv:1507.06739, 2015.

[31] M. Kearns. Efficient noise-tolerant learning from statistical que- [43] X. Tian, N. Bi, and J. Taylor. Magic: a general, powerful and ries. Journal of the ACM (JACM), 45 (6):983–1006, 1998. tractable method for selective inference. arXiv: 1607.02630, 2016.

[32] J. D. Lee, D. L. Sun, Y. Sun, and J. E. Taylor. Exact post-selec- [44] M. Tomamichel, R. Colbeck, and R. Renner. Duality between tion inference, with application to the lasso. The Annals of Statistics, smooth min- and max-entropies. IEEE Trans. Information Theory, 44(3):907–927, 2016. 56(9):4674–4681, 2010.

[33] N. Littlestone and M. Warmuth. Relating data compression [45] A. Vitanov, F. Dupuis, M. Tomamichel, and R. Renner. Chain and learnability. Technical report, 1986. rules for smooth min- and max-entropies. IEEE Trans. Information Theory, 59(5):2603–2612, 2013. [34] R. Lockhart, J. Taylor, R. J. Tibshirani, and R. Tibshirani. A significance test for the lasso. The Annals of Statistics, 42(2):413, [46] Y.-X. Wang, J. Lei, and S. E. Fienberg. A minimax theory for 2014. adaptive data analysis. arXiv:1602.04287 [stat.ML], 2016.

[35] S. Oh and P. Viswanath. The composition theorem for differ- [47] Y.-X. Wang, J. Lei, and S. E. Fienberg. On-average KL-privacy ential privacy. CoRR, abs/1311.0776, 2013. URL http://arxiv.org/ and its equivalence to generalization for max-entropy mechanisms. abs/1311.0776. arXiv:1605.02277 [stat.ML], 2016.

From the President continued from page 1

This is not the only book project in the works by the way. I hope to goes into such an event. Therefore, a great thank you to the orga- have more to report in the next column. nizing team! Hopefully their heroic efforts inspire you to check out the article written by Emanuele Viterbo, the head of our con- By the time you read this, it will be time for ISIT in Aachen, Ger- ference committee, and Elza Erkip entitled “Thinking about orga- many. No doubt it will be perfectly organized and smoothly run nizing an ITW or ISIT?”. like a well-oiled engine, emitting a maximal amount of excite- ment. If you have ever been involved in organizing one of these I hope to see many of you in Aachen! large conferences, you know the incredible amount of effort that Ruediger

IEEE Information Theory Society Newsletter June 2017 11 IEEE Communications Society and Information Theory Society Joint Paper Award for 2017

The paper “List Decoding of Polar Codes”, IEEE Transactions on Communications Society and Information Theory Society Joint Information Theory, Vol. 61, No. 5, pp 2213–2226, May 2015 by Paper Award for 2017. Ido Tal and Alexander Vardy has been awarded the 2017 IEEE Congratulations!

IEEE Information Theory Society’s New Distinguished Lecturers

The Information Theory Society established the Distinguished Lec- Congratulations to five new lecturers that have been named by turer Program to promote interest in information theory by sup- the Society as Distinguished Lecturers for 2017–2018: porting chapters who wish to invite prominent information theory researchers to give talks at their events. Distinguished Lecturers are Tara Javidi, Navin Kashyap, Chandra Nair, Osvaldo Simeone selected by the Membership and Chapters (MC) Committee in con- and Ram Zamir. sultation with the Board of Governors. The Society aims to maintain ten Distinguished Lecturers each serving for two year terms.

Behind the Scenes: A Q&A with the Authors of Claude Shannon’s New Biography Jimmy Soni and Rob Goodman

We’ve spent the prior five years writing a Our agent connected us with Alice Mayhew at biography about Claude Shannon. We are Simon & Schuster, the editorial force behind Steve unexpected chroniclers of his life: We aren’t Jobs and Einstein by Walter Isaacson and A Beauti- engineers or mathematicians or even scien- ful Mind by Sylvia Nasar. She’s shepherded books tists; we are biographers (our first book was of this kind before, and she understood this idea about an ancient Roman senator named instantly. Five years later, we have ourselves a Cato). It took a good deal of research—and book. And as interesting as we thought Shannon lots of help from people more knowledge- was when we knew only the barest details of his able than us!—to put together what we life and work, we find him all the more compelling think is a compelling window into the life now. He was a once-in-a-generation figure, and we of one of the 20th century’s great minds. hope our biography manages to capture that.

Many people who work in information the- What Did You Learn About Him ory have, understandably, wondered how That Surprised You? two non-technical writers came to this project. We figured we would share how we got Those who know Shannon’s work appreciate its here, what we learned along the way, and rigor, thoughtfulness, and depth. Those are his hall- how this book came to be: marks and they are why he still commands such respect today. What’s less understood, or what How Did You Come to Learn surprised us, at least, was Shannon’s artistic and About Claude Shannon? What creative bent. He had a flair for the visual—and it’s Got You Hooked? telling that, alongside the landmark papers published in journals, he constructed objects that are, to The idea came from a friend, who sent Jimmy a copy of Jon Gert- this day, featured in museums. His son, spoke with us at length about ner’s book, The Idea Factory, a narrative history of the Bell Labora- this, probably because he was one of the few who had a front-row seat tories. The gift included a note that said, half-jokingly, “Your next to all the tinkering and building that Shannon did as an adult. book ought to be about Claude Shannon.” Jimmy found Shannon to be an engaging figure. He went looking for a biography of him Many of the objects Shannon made have a theatrical quality: The and couldn’t find one. And thus a book idea was born! fire-breathing trumpet, for instance, or the Ultimate Machine,

June 2017 IEEE Information Theory Society Newsletter 12 whose only purpose is to turn itself off. Even Theseus the maze- to absorb Shannon lore from the people who experienced it first- solving mouse, his best-known creation, is designed for maxi- hand, and to help preserve it for posterity. mum visual impact: the mouse is just an accessory to the real “artificially intelligent” system, the 75 relays that operate Having Closely Studied His Life, What Do You below the maze and allow the mouse to locate a metal piece Think is the Key to His Breakthroughs? of cheese. His contraptions may have been private curiosities, but they long for an audience—and there’s real artistry and cre- It’s hard to point to one thing. There are a few elements that run ativity baked into this work. That same artistry, we would con- through his body of work, and each one, we think, is important. tend, drove Shannon’s most groundbreaking theoretical work, including his innovations in information theory. An engineer we He was a tinkerer from his earliest days. As a boy, he built a spoke with called this quality “thinking not only about things, barbed-wire telephone line and a makeshift elevator in a friend’s but through things.” barn; his hands were always busy, taking things apart and putting them back together. He brought this tinkering spirit to much of his Who Do You Think Influenced His Work? work, building with his hands to test and refine what he dreamt up with his mind. Shannon was the sort of student whose talent marked him out almost immediately as a promising protege, and he crossed paths Shannon was persistent, especially early in his career. Even though with many of the most influential scientists and engineers of his he gave the impression of a carefree scholar, Shannon was phe- day. Perhaps his most important early mentor was Vannevar nomenally hard-working, and he stuck with problems long past Bush, the MIT engineering professor who went on to play a cru- the point at which others might have given up. Shannon’s work on cial role in coordinating America’s scientists during WWII. It was information theory is so elegant that we can sometimes overlook Bush who hired Shannon to work on MIT’s differential analyzer, the fact that his “Mathematical Theory of Communication” was a massive analogue computer, as Shannon completed his Master’s developed over the course of a decade—and he did the bulk of the degree; and it was during his work for Bush that Shannon wrote work on it in his spare time. his famous thesis on the use of digital switches for Boolean logic, which Walter Isaacson called “the basic concept underlying all There was, as our title suggests, an element of play throughout digital computers.” Another important early influence for Shan- the work. You get the impression that each of these intellectual non was his dissertation supervisor, the geneticist Barbara Stod- puzzles were a joy to figure out rather than a bore. Codebreaking, dard Burks, who oversaw his work developing what he called “an chess-playing machines, information theory—it all came from the algebra for theoretical genetics.” same joyful place.

Shannon also drew inspiration from the histories of the fields he And finally, he had some gifts and some luck. A gift for simplifica- worked in. “A Mathematical Theory of Communication,” his land- tion, for instance, which led him to the essence of things. He’d also mark work on information theory, cites the earlier work of Harry been fortunate: fortunate to have earned the early backing of Van- Nyquist and Ralph Hartley in developing the scientific concept nevar Bush; lucky to have found his way to Bell Labs, which might of information; our book describes how their work at Bell Labs have been the one place on earth that would tolerate him; blessed shaped the field as Shannon found it. In another unpublished pa- to have spent World War II on cryptography and the mathematics per on, of all things, juggling, he cites a wide range of sources: the of anti-aircraft fire rather than fighting in combat. science fiction author Robert Silverberg, Captain James Cook, Xe- nophon, W.C. Fields, and the jazz drummer Gene Krupa, among Why Do You Think He Doesn’t Have the Name many others—which should give you an idea of just how eclec- Recognition of an Einstein or a Newton? tic Shannon’s influences and interests were. Shannon read very widely, and he found inspiration in equal parts from T.S. Eliot (his Partly because he didn’t chase the attention. Shannon had numer- favorite poet) and Lewis Carroll. He was moved by music and art ous opportunities to become a scientific celebrity. In fact, in the as much as by mathematics and engineering, and it’s important to 1950s and 1960s, he got a taste of it: he made the rounds of media include those among his influences. and was written up in Life and Time Magazine. But playing the part of celebrity intellectual was an odd fit for him, and he gave it up What was the Process of Writing the almost as soon as he got it. He preferred to follow his own curios- Biography? ity rather than burnish a public profile.

It involved a mix of in-depth research and interviews with Shan- Outside of his own indifference to the attention, it’s easy to take non’s family, friends, and associates. We were fortunate to have the information revolution for granted—to simply accept that the strong guides through the world of information theory, people world is as it was destined to be. That goes double for contribu- like Dr. Sergio Verdú, as well as access to the people who knew tions like information theory, which, even to the initiated, can be Shannon and his work. Andrew and Peggy Shannon, his son and difficult to understand. The reason we wrote the book is because daughter, were very generous with their time. Betty Shannon we thought there was something wrong about that: here we’ve opened up to us about the Claude she knew. Robert Gallager, Len been given this bounty of digital information and instant com- Kleinrock, Henry Pollak, Bob Fano, Ed Thorp, and many others munication, but we only have a vague appreciation for how we gave us vivid descriptions of what it was like to study under or got here and who laid the theoretical groundwork for the world work with Shannon. The interviews were the most useful, and we now live in. Hopefully the book will help us all appreciate the most enjoyable, part of writing the book. It was a pleasure Shannon—and many of his contemporaries—a bit more.

IEEE Information Theory Society Newsletter June 2017 13 The Historian’s Column Anthony Ephremides

This is it! It is time to say loud and clear that the King has no Commercial publishers do share a clothes! I am using this metaphor to address an issue that has been great deal of responsibility for these enveloping our profession (and not only). The proliferation of con- developments. At best, they are guid- ferences and journals that we have been experiencing over several ed simply by the lure of the bottom- years (if not decades) already, and which we have been ignoring line and they place quality at a sec- and/or tolerating, has reached the point where it needs to be con- ondary level. Beyond them, there is fronted. Or, to use another cliché, we cannot ignore anymore the also a growing set of individuals and elephant in the room. other organizations that have been polluting the international scientific It used to be that scientific gatherings and publications were se- scene. Without naming them, I can rious affairs that were organized by reputable organizations and report that there is an individual, with claimed positions between were indeed serving the needs for communication amongst sci- a northern Swedish University and a University in the Persian entists and engineers. To be sure, the level of activity in numer- Gulf, who has an h-index in the thousands! And there has been an ous fields, and especially in the areas that deal with communi- organization in Europe, operating out of northern Italy, that has cation, computation, control, and information, has been soaring been running dozens (if not hundreds) of “fake” conferences after and, hence, it is reasonable to expect that there would be more a carefully planned exploitation of originally legitimate channels workshops, conferences, and symposia and more journals; espe- of Universities and official European funding agencies. cially so, since activity in these fields is no more confined to North America and Europe. In many parts of the world the needs for Friends, this is dangerous and needs to be controlled. Of course harnessing the creative talents and resources of local populations we are blessed to belong to an organization whose symposia, have yielded genuinely productive and constructive activities in workshops, and publications do remain at a level of the highest intellectual and scientific discourse that has translated into more standard. Nonetheless, as most of us are also active in areas out- (and bigger) meetings and more periodicals and books. The free- side mainstream Information theory, we are undoubtedly aware dom from the constraints that the use of traditional “print” media of this emerging phenomenon. Perhaps not all of us have encoun- used to entail has led to massive use of web-based formats and has tered cases of egregious abuse. However, most of us do perceive, I accelerated this growth. believe, a dangerous trend that is changing the landscape.

Like any activity that acquires momentum, this one has attracted For balance, I would like to acknowledge that not all new and the attention of business oriented outfits (publishing houses, inter- unusual activities are deplorable. Among the many exceptions, I national meeting organizers, and creative new format inventors) would like to mention some that I happen to be involved in (not and has been slowly altering the landscape we have been used to imply that they are worthwhile because of my involvement but, to. What is worse, for the younger amongst us, there has been no rather, that I am involved because they are worthwhile). One is alternative experience and the emerging new ecosystem is taken the Journal of Communications and Networks that has been actu- as the accepted status quo. And what is worst, the crowd attracted ally on the “scene” for many years. It emerged out of the desire by the new opportunities of abuse and corruption in the existing of the Korean community, active in the areas of communications system has included outright fraudsters who have grown from be- and networking, to harness the talents of their members to lead a ing a nuisance or a subject of jokes to becoming a serious threat to world-class publication that would meet the standards of high in- the foundations of the practice of science. ternational quality. The main mover and shaker of this effort was Byung Lee, a leader in the Korea Information and Communication Every day we are bombarded with announcement of new meet- Society who saw that the way to establish a high quality venture ings and symposia that clearly smack of fraud or, at least, de- was to gain the support of an organization like the IEEE. As a re- graded imitations of erstwhile serious activities. Many of us result of a long and sustained effort by him and others who worked ceive “invitations” to organize sessions an ANY topic we choose with him, he managed to elevate the status of this journal through in conferences with ambiguous names and organized by outfits a technical sponsorship by the IEEE Communication Society and of dubious stature. The reviewing process which is supposed to through Editorial Boards that included people like Ray Pickholtz, provide a filter for preserving quality through the seal of approval Steve Weinstein, Ezio Biglieri, Vince Poor, and several other well- by reputable and recognized experts has been diluted to the point respected members of our community, of becoming meaningless. I am serving as Editor-in-Chief of two publications (more on that later) and I am in disbelief when I re- Another one is the NOW publishing company, based in the Nether- ceive messages that in essence say something like “I am “so-and- lands, which aspired to replace Springer Verlag as the brand-name so” and I am delighted to report to you that I am ready to edit a of quality in assembling monographs of exceptional standard. Its special issue in your esteemed journal”! Not only, in most cases, endeavors include a set of Series of periodical monographs (not haven’t I ever heard of these individuals but sometimes they dare a contradiction of terms) called “Foundations and Trends in X” to include their resumes which are of unbelievably unacceptable where X takes the values “Information Theory,” “Networking,” nature. It seems that we have hit a rapid slope that threatens to “Signal Processing,” and several others. Many of our readers are lead us to a world of “alternative scientific facts” or, worse, “fake familiar with these series since NOW is usually present, and ex- scientific news.” hibits its products, during our major conference activities.

June 2017 IEEE Information Theory Society Newsletter 14

Finally, I would also like to put a plug for some conferences that profit this is a regional effort to establish a model for worthwhile might appear of questionable motivation and quality, but for activities that meet solid standards of quality. I hope that its future which there are strong supportive arguments that are accom- will be similar to that of the WiOpt conference that Eitan Altman panied by careful implementation. One of these, in which I was and I founded in 2003 and which has been thriving and enjoying recently involved, is the BALKANCOM, a small new conference respect and international recognition ever since. on Communications and Networking that aspires to harness the talents of a developing area of the world which includes the states Our community needs to engage in a vigorous effort to moni- of the Balkan Peninsula. By the time you read this, its first edition tor and maintain the quality that has made our Society so pro- will have taken place in Tirana, Albania with the participation of foundly respected, as we engage in parallel activities; these ac- people like Gerhard Kramer, Alexandre Proutiere, George Gianna- tivities occur amidst the cataclysmic environment of a veritable kis, Petar Popovski, and other well-known members of our com- jungle of symposia and products that threaten the integrity of munity. Without any sponsor, totally self-sustained, and not-for- our profession.

Students’ Corner: The Information Theory Student Subcommittee Goes Social! Mine Alsan ([email protected]) and Basak Güler ([email protected])

In order to facilitate interaction and networking opportunities r Have you ever found yourself in a situation where you between our student and postdoctoral members, the Information wished you could have presented your work to a broader Theory Student Subcommittee is now present on two online plat- audience during a conference? You can now advertise your forms: Facebook and Slack. talk to your peers via the channel #come-and-see-my-talk.

The Facebook group was created in May 2016 with the goal to r Have you ever wished there was a section on Quora for ask- create a social media domain to connect students and postdoc- ing one of the many questions swirling in your head at the toral scholars interested in information theory. The group is intersection of IT, research, academia, publications, etc., but called the “IEEE Information Theory Society Student Group,” thought that no one would probably be able to generate a which is a public group and can be found easily by searching useful answer? In 2016, a set of more senior IT Society mem- its name on Facebook. Members of this group can connect with bers kindly agreed to answer questions and give career each other and learn about important news and events targeted advice in #ask-me-anything. Use this opportunity to reduce at the student members of the information theory community, (or increase!) your uncertainties. including student events in upcoming conferences as well as travel support deadlines. Interested students can send a mem- r Have you ever found yourself in a situation you wanted to get bership request using a Facebook account to join the group. more out of a conference, be more productive, and get in touch with more colleagues to discuss their research interests, but In addition to the Facebook page, a Slack group with the name did not get the opportunity in the midst of the many parallel “ISIT 20**: What’s up, Docs?” was initiated to create a conference- sessions? As a first step, you can now start a conversation in wide channel which connects the student and postdoctoral attend- #post-an-open-problem or #tutorial-materials. Alternatively, ees of the main annual conference in our field. Slack is a commu- you can use the application to create a separate channel to nication platform exclusive for “teams” where members can post initiate a discussion group on a topic of your choice and even messages or share files within topic-specific channels. Attendees organize a reading group around the conference dates. of ISIT can use this communication channel (which goes with a smartphone app) to engage more effectively with their fellow col- r Have you ever found yourself in a situation where you wished leagues before, during, and after the conference. Just to highlight there were a couple of activity buddies among the other a few possible use cases: attendees who share similar interests with you? If you don’t want to miss out on the opportunity to enjoy what the conference location has to offer, we suggest that you create your own r Have you ever found yourself in a situation where you #activity channel to find like-minded people to organize needed to find a shared accommodation to attend a confer- group activities. In 2016, #run-a-centennial-run and #isit- ence, potentially with a person outside your own institute, bouldering were created to organize running and bouldering but did not know who to ask? Next time, feel free to drop a activities. In addition, local suggestions for restaurants around message to the #logistics channel to quickly reach out to the conference venue were shared in #dinner-meet-ups. Take your colleagues. a holistic view during the next ISIT; you won’t regret it!

IEEE Information Theory Society Newsletter June 2017 15

In a nutshell, the goal of the Slack group is to provide the younger ming raises questions like: How do you manage to be a better members of our community with a practical tool to manage more researcher and be more successful? How can you become more easily, shape more creatively, and enjoy to the fullest their confer- productive even in a non-ideal environment? What are the ence experience. To join this group, simply send a private message most important characteristics of a good researcher? Serving to either Mine Alsan ([email protected]) or Bernhard Geiger as panelists, Professors Dan Costello, Michelle Effros, Emina ([email protected]) with your name, surname, affiliation, and email Soljanin and Emre Telatar answered these thought-provoking address, and we will invite you! questions and more.

Other than going social on Facebook and Slack, the IT Student Moreover, as in previous years, we are organizing our annual Subcommittee continues to organize exciting events for IT stu- “Meet the Shannon Awardee’’ event at ISIT 2017. This is a great dents. For example, at ITA 2017, the Information Theory Stu- opportunity for students to learn more about this year’s Shannon dent and Outreach Committees, together with the NSF Center Award Winner Professor David Tse. The event will take place over for Science of Information, co-hosted the lunchtime panel “You lunchtime and students will be provided with free lunch. Last and Your Research,” which discussed Richard Hamming’s re- year’s event with Professor Alexander Holevo can be viewed by nowned lecture on how to do great work. In his lecture, Ham- following the link https://vimeo.com/185470703

From the Editor continued from page 2

With sadness, we conclude this issue with a tribute to Mary Eliza- The next few deadlines are: beth (Betty) Moore Shannon who passed away on May 1st at the age of 95. The tribute appeared in the Boston Globe and is repro- July 10, 2016 for the issue of Sep. 2017. duced here with permission. Oct. 10, 2016 for the issue of Dec. 2017. Please help to make the newsletter as interesting and informative as possible by sharing with me any ideas, initiatives, or potential newslet- Please submit plain text, LaTeX, or Word source files; do not worry ter contributions you may have in mind. I am in the process of search- about fonts or layout as this will be taken care of by IEEE layout ing for contributions outside our community, which may introduce our specialists. Electronic photos and graphics should be in high reso- readers to new and exciting problems and, as such, broaden the influ- lution and sent as separate files. ence of our society. Any ideas along this line will also be very welcome. I look forward to hearing your suggestions and contributions. Announcements, news, and events intended for both the printed newsletter and the website, such as award announcements, calls With best wishes, for nominations, and upcoming conferences, can be submitted Michael Langberg. at the IT Society website http://www.itsoc.org. Articles and col- [email protected] umns can be e-mailed to me at [email protected] with a subject line that includes the words “IT newsletter.”

June 2017 IEEE Information Theory Society Newsletter 16 Thinking About Organizing an ITW or ISIT? Emanuele Viterbo and Elza Erkip

One of the most rewarding experiences as volunteer in our Society The budget—A simplified budget with the key items template can is the organization of a workshop or conference. In this short note be used to get an idea of the registration fees. Please contact the we would like to give some basic information and tips on how to conference committee for a copy of this template. approach this seemingly daunting task. The financial co-sponsorship (FCS)—The Information Theory Society The team and key roles—To start, as the promoter you should try is the default financial co-sponsor with IEEE of ITW and ISIT’s to put together a small team of volunteers possibly including taking responsibility for all gains and losses. someone with some prior experience, someone local (or as close as possible) to the venue, someone thinking about the technical The technical co-sponsorship (TCS)—The Information Theory Soci- program. Consider diversity and gender balance. The key roles ety is the default technical co-sponsor of ITW and ISIT’s. that should be identified as early as possible are: General Chair (typically 1–2 persons), TPC Chair (typically 2–4 persons), Local The Conference Committee—The IT Society’s Conference committee Arrangements, Finance Chair, Publications. Other roles are also is in charge of collecting the expressions of interest from the pro- very important but can be assigned at a later stage. moters, give advise and feedback for the preparation of the final bid. The timeline—BoG meetings are scheduled three times a year around February, June/July (at the ISIT), and October. The final selection for Resources—The most useful resource is the website of the pre- the next available ISIT is made only at the June/July BoG meeting. vious editions of the conference. Previous organizers are an in- ITW’s can be selected at any BoG meeting. In order to prepare your valuable source of information and full manual for organizers bid for an ITW or ISIT you will need to submit an expression of in- is available from the Conference Committee. If you have further terest to the Conference committee chair at least one to two months questions please contact any one of the members of the Confer- before the BoG meeting preceding the one where the selection is to be ence Committee (http://www.itsoc.org/people/committees/ made (for an ISIT the expression of interest is due in early January). conferences):

The location—The selection of the location within a country should r Emanuele Viterbo, Monash University, Melbourne (Chair), be made in close contact with the local organizers. Many factors [email protected] should be considered for both ITW and ISIT: easy accessibility r Ruediger Urbanke, EPFL, Lausanne (Ex Officio), ruediger. from main hubs, accommodation availability for different price [email protected] levels (including student accommodation where possible). r Elza Erkip, New York University, (Ex Officio), [email protected] r Daniela Tuninetti, University of Illinois at Chicago (Ex The venue—The venue can be a hotel with conference facilities, a con- Officio ), [email protected] ference center or a university campus. The venue should be checked r Albert Guillen i Fabregas, Univ. Pompeu Fabra, Barcelona, to verify the quality of the rooms and AV equipment. The plenary [email protected] space should be sufficient to accommodate all the participants with r Urbashi Mitra, University of Southern California, [email protected] a good margin. A desirable features for the venue is to have all the r Brian Kurkoski, Japan Advanced Institute of Science and rooms in short range to facilitate switching sessions, and single space Technology, [email protected] for coffee breaks where people can easily meet. Additional, break-out r Alfonso Martinez, Univ. Pompeu Fabra, Barcelona, alfonso. spaces for participants to sit and discuss are also useful. [email protected]

IEEE Information Theory Society Newsletter June 2017 171 GOLOMB’SGOLOMB’S PUZZLEPUZZLE COLUMN™COLUMN™ COLLECTION,COLLECTION, Part 4

Beyond hishis extraordinary extraordinary scholarly scholarly contributions, contributions, Sol Sol Golomb Golomb was was a totion the to newsletter, the newsletter, a collection a collection of his of hisearlier earlier puzzles puzzles dated dated back back to longa long time time newsletter newsletter contributor contributor enlightening enlightening us all, us youngall, young and old,and 2001to 2001 appears appears in in 4 4compiled compiled parts parts over over theprevious, previous current, and currentand up- withold, with his beautiful his beautiful puzzles. puzzles. In honor In honor of Sol’s of Sol’s immense immense contribution con-tribu- issues.coming Part issues. 4 is Partgiven 4 isbelow. given He below. will beHe greatly will be missed. greatly missed.

Reprinted from Vol. 53, No. 3, September 2003 issue of Information Theory Newsletter

GOLOMB’S PUZZLE COLUMN™ Latin Squares and Transversals Solutions

1. Given a pair of orthogonal Latin Squares, L and L′, of where { g1, g2,...gn },{h1, h2,...,hn },and{t1, t2,...,tn }are order n, each symbol in L′ occurs in the positions which each permutations of { 1, 2, . . . , n }.ByWilson’sTheoremof form a transversal in L. Thus, the n symbols in L′ specify elementary number theory, n!=(p–1)! ≡ –1(mod p). Multi- n disjoint transversals in L. Conversely,if L has n disjoint plying all n equations (*)togethermodulop,weget transversals, each transversal of L can be used to correspond to a different symbol in L′. ()()()ghgh1122××"" gnn ×≡××× h tt12 tn (mod p ) For example: ()!(pp−⋅−11)!≡− (pp1 )!(mod ) ()()−⋅−≡−11 1 (mod)p +11≡−(modp )

a contradiction. 4. “The number of mutually (pair-wise) orthogonal Latin Squares (MOLS) of order n cannot exceed n – 1”. 2. If L is the Cayley table of a group G of order n, Proof. It is no loss of generality to rename the ele- and it has a transversal, we can represent this as ments in each of the Latin Squares so that each top row follows. Let G ={g1, g2,...,gn },andindexthe consists of 1, 2, 3, · · ·, n. Next, we simultaneously per- rows of L with g1, g2,...,gn. For a transversal in L, mute the remaining rows (which disturbs neither each row gi must be paired with a column hj to get an en- Latin-ness nor orthogonality) so that the second row of try gi × hj = tk, where “×” is the group operation, and all the first Latin Square begins with “2”. We now ask what three of { g1,g2,...,gn},{h1,h2,...,hn }, and { t1, t2,...,tn } are the possibilities are for the left-most elements of the sec- permutations of the n elements of G.Ifnowwe ond rows of the remaining, mutually orthogonal Latin right-multiply each equation gi × hj = tk by a fixed ele- Squares. From the corresponding elements in the top ment p ∈ G, we get a “new” transversal (truly new if p is rows, all the ordered pairs 11, 22, 33, ...,nn have already not the identity element in G), because occurred, so these left-most positions in the second rows must all be distinct from each other, and from the “2” in ghptpght11××()()= 1× 111 ×′=′ the first Latin Square. Moreover, since each sits below a ghptpgh22××()()= 2× 22 ×′=t′2 “1” in the top row of its Latin Square, no entry in row 2, ! column 1, can be “1”. This leaves the n–2 values 3, 4, ..., ghptpght××()()=× ×′=′ n, and limits the number of mutually orthogonal Latin nn n nnn Squares of order n to at most 1+(n – 2) = n – 1.

where { h′1 , h′2 ,...,h′n} is a new permutation of the ele- Note: A construction for n – 1 MOLS of order n is known k ments of G, and { t′1 , t′2 ,...,t′n} is a new permutation of whenever there is a field of n elements (thus, n = p , p the elements of G, disjoint respectively from { h1, h2,..., prime and k≥1). Whether this can happen for any other hn } and{ t1, t2,...,tn } . Thus, each group element p ∈G values of n is unknown in general, has been shown to be generates a new transversal, disjoint from the others. impossible for infinitely many n ≠ pk , and has never been (We used the associative law for groups when we took successfully constructed for any n ≠ pk . (gi × hj)×p =gi ×(hj × p).) Thus, if L is a Cayley table, one transversal gives a “complete set” of n disjoint trans- 5. “If a Latin Square L of order n has n–1 disjoint transversals, and hence an “orthogonal mate” L′. versals, then it has n disjoint transversals.” The converse is trivial. If L has an “orthogonal mate” L′, Proof. Each transversal of L consumes one cell in each it has n disjoint transversals, so surely at least one row, one cell in each column, and one of each of the n transversal. symbols. Hence, n – 1 disjoint transversals consume n –1 x cells in each row, n – 1 cells in each column, and n – 1 of 3. If p=n+1isprime,n>1,andLis the Cayley table of Zp ,the each of the n symbols. Thus, what remains in L is one cell multiplicative group modulo p (which has order n), we will in each row, one cell in each column, and one each of the assume that L has a transversal, and obtain a contradiction. n symbols, i.e. an nth disjoint transversal. As in the previous problem, a transversal of L looks like: 6. Here is a Latin Square of order 6 with four disjoint ght111×= transversals, whose elements are enclosed in the fig- ght222×= ures,circle, triangle, square, and diamond, respectively. (*) ! ght×= Note: Euler further conjectured that no pair of orthogo- nnn nal Latin Squares of order n exists when n =2m, where m

IEEE Information Theory Society Newsletter September 2003 June 2017 IEEEIEEE Information Theory Society Newsletter 2 218 9 Reprinted from Vol. 53, No. 3, September 2003 issue of Information Theory Newsletter continued 9 8 Reprinted from Vol. 53, No. 3, September 2003 issue of Information Theory Newsletter continued 8

GOLOMB’S PUZZLE COLUMN™ is odd. This was shown to be false, in 1959, for all n > 6, GOLOMB’S PUZZLE COLUMN™ byis odd. R.C. This Bose, was S.S. shown Shrikhande, to be false, and in E.T. 1959, Parker. for all Inn > fact, 6, it hasby R.C. been Bose, shown S.S. that Shrikhande, if M(n) isand the maximum E.T. Parker. number In fact, of it MOLShas been of shown order thatn ifwhichM(n) is the actually maximum occur, number then of OVERLAPPING SUBSETS SOLUTIONS limMOLS infMn of ( ) order=∞. n which actually occur, then OVERLAPPING SUBSETS SOLUTIONS limn→∞ infMn ( ) =∞. n→∞ Solomon W. Golomb Solomon W. Golomb

1. a. By statistical independence, the expected number of c. Numerically, pr(9) 10 0.1477564. ≈ 27√102π = 1. a. Byoverlaps statistical is M independence,( a )( b )N ab the. expected number of c. Numerically, pr(9) ≈ = 0.1477564. overlaps is M N N N N . ≈ 27√2π = Nelson Blachman - continued overlaps is M = ( a )( b )N = ab . Nelson Blachman - continued a N a Nb NNb N (k)( b−k )= (k)( a−k ) = a!b!(N a)!(N b)! Nelson Blachman - continued k b−k k a−k ! ! ! ! b. pr(k) a NN− a b NN− b − − . ( )(( )− ) ( )(( )− ) k!(a ka)!!b(!b(Nk)!aN)!!((NN ba)! b k)! 9 = k ( bb) k = k ( aa) k = − − − − + 9 9 b. pr(k) N− N− − − − −− − + . 4. a. Pr(9) e 0.13175564. sulted in many of the foregoing papers and others applying [6] C. E. Shannon, “Communication in the presence of noise,” Proc. ( ) ( ) k!(a k)!(b k)!N!(N a b k)! 4. a. Pr 9 e− 9!9 0 13175564. pr(k 1)= b (a =k)(b k)a = − − − − + = 9 · 9 = sulted in many of the foregoing papers and others applying Inst.[6] C. Radio E. Shannon, Engrs., “Communication vol. 37, no. 1, pp. in10-21, the presence Jan. 1949. of noise,” Proc. c. + − − . 4. a. Pr(9) e− 9! 0.13175564. these ideas as well as some on random processes, informa- Inst. Radio Engrs., vol. 37, no. 1, pp. 10-21, Jan. 1949. prpr(k(k)1) (k 1()(aNk)(ab bk)k 1) = · = + = + − − + + these ideas as well as some on random processes, informa- Inst. Radio Engrs., vol. 37, no. 1, pp. 10-21, Jan. 1949. c. pr(k) (k 1)(N− a −b k 1) . b. The largest source of error in 3.c. was using Stirling’s tion theory, et al. For a complete list of my publications, see = + − − + + [7] H. P. Manning, Geometry of Four Dimensions. New York: Do- b. formulaThe largest to approximatesource of error 9! inin the3.c. denominatorwas using Stirling’s of 3.a., 8http://home1.GTE.net/blachman/tion theory, et al. For a complete list . of my publications, see [7] H. P. Manning, Geometry of Four Dimensions. New York: Do- Reprinted from Vol. 53, No. 4, December 2003 issue of Information Theory Newsletterver, 1956 (reprint of Macmillan, 1914). whichformula gives to approximate 9! 359, 536 9!.873in .the This denominator is only about of 99%3.a., 8Fromhttp://home1.GTE.net/blachman/ 1951 to 1954 I’d worked at the Office . of Naval Research ver, 1956 (reprint of Macmillan, 1914). ≈ ReprintedFrom 1951 from to Vol. 1954 53, I’d No. worked 4, December at the 2003 Office issue of of Naval Information Research Theory Newsletter 2. a. M 9. whichof the truegives value 9! (3599! , 536362.,873880. )This. This is “correction” only about 99% From 1951 to 1954 I’d worked at the Office of Naval Research [8] D. M. Y. Somerville, An Introduction to the Geometry of N Di- = ≈ = (ONR) in Washington, and for occasional relaxation from 2. a. M 9. ofwould the true only value reduce (9! the 362estimate, 880) .in This 3.c. “correction” to pr(9) 0. 146; (ONR) in Washington, and for occasional relaxation from mensions.[8] D. M. Y. New Somerville, York: Dover, An Introduction 1958 (reprinted to the from Geometry Methuen, of 1929). N Di- = = ≈ my forty years at GTE I took leaves of absence to do scientific b. pr(k 1) pr(k 1) pr(k 1) wouldso 3.c. isonly almost reduce certainly the estimate a better in estimate 3.c. to pr (than9) ≈ 4.a.0.146; mensions. New York: Dover, 1958 (reprinted from Methuen, 1929). b. k + k + k + so 3.c. is almost certainly a better estimate than≈ 4.a. liaisonmyGOLOMB’S forty at years ONR’s atPUZZLE GTE London I tookCOLUMN™ Branch leaves and of absence to work to at do the scientific Naval b. prpr(k(k)1) prpr(k(k)1) prpr(k(k)1) so 3.c. is almost certainly a better estimate than 4.a. [9] H. S. M. Coxeter, Regular Polytopes, 3rd ed. New York: Dover, k (+) k (+) k (+) liaisonGOLOMB’S at ONR’s PUZZLE London COLUMN™ Branch and to work at the Naval pr k pr k pr k Pr(k 1) λ Research Laboratory as well as to teach in Madrid under the 1973.[9] H. S. M. Coxeter, Regular Polytopes, 3rd ed. New York: Dover, 011.2344 42.0403 81.0284 c. Since + for the Poisson distribution, if we 1973. PrPr(k(k)1) kλ1 Irreducible Divisors of Trinomials Solutions 011.2344 42.0403 81.0284 = + FulbrightResearchIrreducible Laboratory program. as I feel well that asDivisors to I’ve teach been in Madridthe beneficiary of under Trinomials the of 1973. Solutions 15.4855 51.6586 90.8999 c. take Since λ Pr9(+k) and 7k 1 kfor 11the, we Poisson find distribution, if we Irreducible Divisors of Trinomials[10] N. M. Solutions Blachman, “The output signal-to-noise ratio of a = = +≤ ≤ muchFulbright good program. luck during I feel the that course I’ve of been my the career, beneficiary including of [10] N. M. Blachman, “The outputSolomon signal-to-noise W. Golomb ratio of a 2315.57034855 6151.38656586 1090 0.79598999 take λ 9 and 7 k 11, we find power-law[10] N. M. device," Blachman, J. Appl. “The Phys., output vol. signal-to-noise24, no. 6, pp. 783-785, ratio Juneof a = ≤ ≤ mymuch poor good vision’s luck notduring utterly the failing course until of my 2000. career, Fortunately including I Solomon W. Golomb 3223.61365703 7161.18293865 1011 0.79597105 1953.power-law device," J. Appl. Phys., vol. 24, no. 6, pp. 783-785, June k 9 my poor vision’s not utterly failing until 2000. Fortunately I 32.6136 71.1829 11 0.7105 k (k 1) was able to acquire software by then that speaks aloud my 1953. k +9 c. The mode is 9 (same as the mean, in this case), since pr(k) (k 1) keystrokeswas1. able “A primitive to and acquire whatever’s polynomial software on byf ( myx) thenof computer degree that speaksn screen.2 divides aloud So infi- I’ve my 3.[11] “If W. p B.5 Davenport,is a prime for Jr., “Signal-to-noisewhich 2 is primitive ratios modulo in bandpass p, then limit- c. The is 9 (same as the , in this case), since 71.125+ keystrokes and whatever’s on my computer≥ screen. So I’ve ≥ c. The mode is 9 (same as the mean, in this case), since pr(k) keystrokesnitely1. “A primitivemany and trinomials whatever’s polynomial over on fGF( myx)(2)”.of computer degree n screen.2 divides So infi- I’ve 3.ers,”[11]f ( x“If) W. J.p (≥Appl. B.xp5 Davenport,is1 a )/(Phys., primex 1vol. )for Jr., 24, “Signal-to-noisewhich1 no.x 26, xis 2pp. primitive 720-727, ratiosxp modulo 1June in bandpass 1953. p, then limit- is increasing up to k 1 9, but decreasing thereafter. 71.125 notnitely been stoppedmany trinomials in my research over GF(2)”. work although≥ it’s slowed ers,”f (x) J. ≥(Appl.x 1 )/(Phys.,x 1vol.) 24,1 no.x 6,x pp. 720-727,x − Juneis 1953. an irre- + = 81.000 nitely many trinomials over GF(2)”. ers,”f (x) = J. (Appl.xp − 1 )/(Phys.,x − 1vol.) = 24,1 + no.x + 6,x 2pp.+···+ 720-727,xp 1Juneis 1953. an irre- is increasing up to k 1 9, but decreasing thereafter. downnot been somewhat stopped inwhile my researchInternet communicationwork although it’s continues slowed ducible polynomial which divides no trinomials.”− + = 9081.900000 down somewhat while Internet communication continues [12] N.= M.− Blachman,− “The= + signal+ times+···+ signal, noise times noise, 90900 downProof. somewhat Let α be a while root of Internet f (x). Bycommunication “primitivity”, all continuesthe values ducible polynomial which divides no trinomials.” 90.900 unabated. At GTE inn Mountain View there were few other [12] N. M. Blachman, “The signal times signal, noise times noise, 10 0.818 2 3α 2 2 ( ) and signal times noise output of ap nonlinearity,” IEEE Trans. In- 90 810 1Proof.,α,α Let,α ,...be ,aα root− ofare f distinct,x . By “primitivity”, and are all all the the non-zero values Proof. For each prime p,"p(x) (x 1)/(x 1) is the “cyclo- (90)( 810 ) 2 2 unabated. At GTE inn Mountain View there were few other p ( k )(90 k) (90!) (810!) 10 0.818 communication2 3 theorists2 n 2 with whom to interact, butn I’m form.and signal Theory, times vol. noise IT-14, output no. 1,= pp. of ap 21-27,− nonlinearity,” Jan.− 1968. IEEE Trans. In- ( ) 90 810− 11 0.750 1,α,α ,α ,... ,α n− are distinct, and are all the non-zeron Proof.form. ForTheory, each vol.prime IT-14, p," no.p(x) 1,= pp.(x 21-27,− 1)/( Jan.x − 11968.) is the “cyclo- 3. a. pr(k) 900 − (( ) )22 ( 2 ) elements of GF(2 ). Therefore, for each j, 0 < j < 2 1, tomic polynomial” over the rational field Q, whose roots are = ( k()(90) k) = k!((90 (90!k)!)) (900!810!()720 k)! . gratefulcommunication to the many theoristsn friends with in many whom places, to interact, such as but Lornen − I’m form. Theory, vol. IT-14, no. 1,= pp. 21-27,− Jan.− 1968. 3. a. pr(k) = 90 − = − + 11 0 750 grateful1elementsα toj theα k ofwith manyGF (02< friends).k Therefore,< 2n in1 manyand for j places, eachk. Hence, j, such0 < f (jx< as) divides2 Lorne− 1, thetomic φ( ppolynomial”) p 1 primitive over the prationalthroots offield unity. Q, whoseWhile allroots cyclo- are (900) k!((90 k)!)2900!(720 k)! grateful1 α to theα with many 0 < friendsk < 2 in1 manyand j places,k. Hence, such f (x as) divides Lorne− the[13] φ N.(p) M. Blachman,p 1 primitive “The uncorrelatedp roots of unity. output While components all cyclo- of a = 90 = − + Campbell,1 + α j = withαk with whom 0 5,3thenon-zero minimum nonlinearities, transform," terms. poly- By IEEE and b. Except for a factor of 2 in the denominator, all the [1] S.+ O. Rice,+ “Mathematical= analysis= of random noise,” Bell Sys. = ≥ b. Exceptirrational for numbers a factor ofdisappear. √2π in the(The denominator, powers of e allcancel the 2. “If f (x) is irreducible with primitivity t and f (x) divides no nomialResultTrans.their optimization: Inform. 2,for above, such Theory, a ifroot inversionthis ofvol. funity(x IT-17,) divides of has the no. p Chebyshev >4,any pp.3 non-zerotrinomial, 398-404, transform," terms. July it must 1971. By IEEE irrational numbers disappear. (The powers of e cancel [1] S. O. Rice, “Mathematical analysis of random noise,” Bell Sys. irrational numbers disappear. (The powers of e cancel 6 Tech. J., vol.f (x 23,) no. 3, pp. 282-332, 1944, and vol.t 24, no.f (x 1,) pp. 46-157, ResultTrans. Inform. 2, above, Theory, if this vol. f (x IT-17,) divides no. 4,any pp.trinomial, 398-404,n a July it must 1971. completely between numerator and denominator. 5. Pr(25) 5.712 10− when λ 9. (The true value of pr(25) trinomials2. “If is of irreducible degree < twith, then primitivity f (x) divides and no trinomials.”divides no divide a trinomial of degree ed., Itst, and Effect Malabar, letf ( on xα) be Communication.divides Fla.: a root Krieger, theof f trinomial( x1982.). Since New degreep 1 withp 1α, whereasas a root the is unique the minimal polynomialpolynomial of degree of of 10.) When all the smoke clears, all that remains is N A AES-4, no.≤ −4, p. 635, July 1968. of 10.) When10 all the smoke clears, all that remains is York:x McGraw-Hill,+ x + 1 of degree 1966; 2ndNt > ed.,t, and Malabar, let α be Fla.: a root Krieger, of f ( x1982.). Since ≤p − 1 with α as a root is the minimal polynomial of pr(9) . f (x) has primitivity t, α 1. Since f (α) 0, where α is a root α[16], f (x J.) Kuhar,"p(x), Jr.,of and degree D. L. p Schilling,1,which “Signal-to-noise has more than ratiosthree for ≈ 27√102π [3] R. B.+ Blackman+ and J. W.t = Tukey. “The measurement= of power ≤ − = − pr(9) ≈ . off ( xf)(hasx), anyprimitivity polynomial t, α g(1x.) Since divisible f (α) by 0f,(wherex) also α hasis a αrootas αterms., f (x) "p(x), of degree p 1,which has more than three 27√2π [3] R.of B.f ( Blackmanx), any polynomial and J. W.= Tukey.g(x) divisible “The measurement by= f (x) also ofhas power α as terms.bandpass[16] J. Kuhar,= limiters," Jr., and IEEE D. L.Trans. Schilling,− Aerosp. “Signal-to-noise Electron. Systems, ratios vol. for ≈ spectraof f from(x), any the polynomial point of view g(x of) divisible communications by f (x) also engineering," has α as terms. spectraa root, from sincethe point g(x) of viewf (x) ofq( communicationsx) gives g(α) engineering,"f (α) q(α) AES-4,bandpass no. limiters,"1, pp. 125-128, IEEE Trans.Jan. 1968. Aerosp. Electron. Systems, vol. Bell Sys. Tech. J., vol. 37,pp.=N 185-282A· and 485-569, 1958;= reprintedn· a by= 0a q root,(α) since0. Thus, g(x )αN f (xα)A q(1x) gives0, from g( α which) f ( αα) n q(αα)a Note.AES-4,There no. 1, are pp. also 125-128, many Jan. other 1968. irreducible polynomials Dover,Bell Sys.· New Tech.= York, J., vol. 1958. 37,pp.=N 185-282+ A·+ and= 485-569, 1958;= reprintedn· + a by+= 01 q(0α)where0. Thus,n N α(modα t) and1 a0, fromA (mod which t), whereα α we whichNote.[17] S. There O.divide Rice, are no ``Noise also trinomials. many in FM other receivers,’’These irreducible three ch. problems 25, pp.polynomials 395-422 are the in M. Dover,·= New= York, 1958.≡ + + = ≡ + + [4] S.1choose= O.0 Rice,where both “Statistical n nand≡ N (mod properties t) and of at a≡ sineA (mod wave t plus),f ( wherex) random we easywhichRosenblatt,[17] S.results. O.divide Rice, ed., no ``NoiseTime-Series trinomials. in FM Analysis. receivers,’’These three New ch. problems York, 25, pp. Wiley, 395-422 are 1963. the in M. [4] S.choose= O. Rice, both “Statistical n and≡ a to propertiesbe less than of t, a≡ from sine which wave plus f (x) randomdivides easy results. noise,”[4] S.choosethe O. Belltrinomial Rice, bothSys. “Statistical Tech.n xandn J.,xaa tovol. propertiesbe1, 27, ofless no.degree than 1, of pp. t < a, from tsine109-157,. wavewhich Jan. plus f ( 1948.x) randomdivides easyRosenblatt, results. ed., Time-Series Analysis. New York, Wiley, 1963. noise,”the Belltrinomial Sys. Tech. x J.,x vol.1, 27,of no.degree 1, pp. < t109-157,. Jan. 1948. [18] N. M. Blachman, ``FM reception and the zeros of narrow-band noise,”the Belltrinomial Sys. Tech. xn + J.,xa vol.+ 1, 27,of no.degree 1, pp. < t109-157,. Jan. 1948. [18] N. M. Blachman, ``FM reception and the zeros of narrow-band [5] C. E. Shannon, “A+ mathematical+ theory of communication,” Gaussian[18] N. M. noise,’’ Blachman, IEEE ``FM Trans. reception Inform. and Theory, the zeros vol. ofIT-10, narrow-band no. 3, pp. Bell[5] C.Sys. E. Tech. Shannon, J., vol. “A 27, mathematical pp. 379-423 and theory 623-656, of communication,” 1948. 235-241,Gaussian July noise,’’ 1964. IEEE Trans. Inform. Theory, vol. IT-10, no. 3, pp. Bell Sys. Tech. J., vol. 27, pp. 379-423 and 623-656, 1948. 235-241, July 1964. Gerard J. Foschini Named Bell Labs Fellow Gerard J. Foschini Named Bell Labs Fellow Alex Dumas Alex Dumas

Bell Labs President Bill O'Shea and Lucent's R&D *Gerard J. Foschini, Bell Labs Research, for his break- leadershipBell Labs Presidenthave chosen Bill seven O'Shea employees and Lucent's as 2002 R&D Bell through*Gerard J.invention Foschini, of Bell the Labs BLAST Research, concept for that his has break- the Labsleadership Fellows. have The chosen annual seven award, employees Bell Labs' as 2002 highest Bell potentialthrough invention to revolutionize of the BLAST wireless concept technology. that has the IEEELabs Information Fellows. The Theory annual award,Society Bell Newsletter Labs' highest potential to revolutionize wireless technology.March 2017 IEEEhonor,IEEE InformationInformation recognizes sustainedTheoryTheory SocietySociety research NewsletterNewsletter and develop- MJarchune 2017 Septembermenthonor, contributions recognizes 2003 sustainedto the company. research The and 2002 develop- awards *Theodore IEEE M. Information Lach, Theory Integrated Society Newsletter Network Septembermarkment contributionsthe 20th 2003 year ofto the the program. company. Since The 2002it began, awards 198 Solutions,*Theodore for IEEE M.his Information innovation Lach, Theory Integrated and Societytechnical Newsletter Network leader- IEEE Information Theory Society Newsletter March 2004 mark the 20th year of the program. Since it began, 198 Solutions, for his innovation and technical leader- IEEE Information Theory Society Newsletter March 2004 scientistsmark the 20thand yearengineers of the have program. joined Since this it elite began, group. 198 shipSolutions, in switching for his innovationsystem component and technical and product leader- Eachscientists new and Fellow engineers will receive have ajoined sculpture, this elite a personal group. reliability,ship in switching silicon fabricationsystem component techniques and and product con- FellowsEach new plaque Fellow and will a receivecash award. a sculpture, A plaque a personal of each tributionsreliability, tosilicon industry fabrication standards. techniques and con- willFellows be addedplaque to and the a “wall cash ofaward. honor” A plaquein the Murrayof each tributions to industry standards. Hill,will beN.J., added lobby. to A the formal “wall luncheon of honor” to honor in the the Murray recip- *Rajeev R. Rastogi, Bell Labs Research, for contri- ientsHill, N.J.,is scheduled lobby. A formalfor September. luncheon to honor the recip- butions*Rajeev R.in theRastogi, areas Bell of network Labs Research, management for contri- and ients is scheduled for September. databasebutions in systems, the areas and of network the successful management application and The 2002 winners are: databaseof these innovations systems, and to Lucent the successful products. application The 2002 winners are: Gerard J. Foschini of these innovations to Lucent products. *Alvin Barshefsky, Lucent Worldwide Services, Gerard J. Foschini *William D. Reents, Supply Chain Networks, for for*Alvin his sustained Barshefsky, performance Lucent Worldwide in the area Services, of software and services pioneering work*William in analytical D. Reents, science Supplyand development Chain Networks, of leading- for development.for his sustained performance in the area of software and services edgepioneering characterization work in analytical tools and science methodologies. and development of leading- development. edge characterization tools and methodologies. *Young-Kai Chen, Bell Labs Research, for his pioneering working *Joseph A. Tarallo, Mobility Solutions, for sustained contributions in*Young-Kai developing Chen, high-speed Bell Labs devices Research, and circuits.for his pioneering working to*Joseph second- A. andTarallo, third-generation Mobility Solutions, wireless for technology, sustained contributionsand base sta- in developing high-speed devices and circuits. tionto second- architecture and third-generation and design. wireless technology, and base station architecture and design. continued on page 9 continued on page 9 IEEE Information Theory Society Newsletter December 2003 IEEE Information Theory Society Newsletter December 2003 19 3

Reprinted from Vol. 54, No. 1, March 2004 issue of Information Theory Newsletter Reprinted8 from Vol. 54, No. 1, March 2004 issue of Information Theory Newsletter

GOLOMB’S PUZZLE COLUMN™

OVERLAPPINGOVERLAPPING SUBSETSSUBSETS SOLUTIONS SOLUTIONS Solomon W. Golomb

1. a. By statistical independence, the expected number of c. Numerically, pr(9) 10 0.1477564. ≈ 27√2π = overlaps is M ( a )( b )N ab . = N N = N a N a b N b (k)( b −k ) (k)( a−k ) a!b!(N a)!(N b)! b. pr(k) N− N− − − . k!(a k)!(b k)!N!(N a b k)! 9 = ( b) = ( a ) = − − − − + 9 9 4. a. Pr(9) e− 9! 0.13175564. pr(k 1) (a k)(b k) = · = c. + − − . pr(k) = (k 1)(N a b k 1) + − − + + b. The largest source of error in 3.c. was using Stirling’s formula to approximate 9! in the denominator of 3.a., which gives 9! 359, 536.873. This is only about 99% ≈ 2. a. M 9. of the trtrueue value (9! 362, 880). This “correction” = = would only reduce the estimate in 3.c. to pr(9) 0.146; pr(k 1) pr(k 1) pr(k 1) ≈ b. + + + so 3.c. is almost certainly a better estimate than 4.a. k pr(k) k pr(k) k pr(k) Pr(k 1) λ 011.2344 42.0403 81.0284 c. Since + for the Poisson distribution, if we Pr(k) = k 1 15.4855 51.6586 90.8999 take λ 9 and 7 + k 11, we find = ≤ ≤ 23.5703 61.3865 10 0.7959

32.6136 71.1829 11 0.7105 9 k (k 1) + mode mean pr(k) c. The is 9 (same as the , in this case), since 71.125 is increasing up to k 1 9, but decreasing thereafter. + = 81.000 90.900 10 0.818 90 810 2 2 ( k )(90 k) (90!) (810!) ( ) − 11 0.750 3. a. pr k 900 k!((90 k)!)2900!(720 k)! = ( 90 ) = − + At k 9, = 2 90 180 810 1620 which are fairly close to the values in 2.b. (The values (2π) 90 810 ( e ) ( e ) pr(9) 5 · · · · . will not be as close for k farther from λ.) ≈ (2π)2 √9 812 900 729( 9 )9( 81 )162( 900 )900( 729 )729 · · · e e e e b. Except for a factor of √2π in the denominator, all the irrational numbers disappear. (The powers of e cancel completely between numerator and denominator. 5. Pr(25) 5.712 10 6 when λ 9. (The true value of pr(25) = × − = Fortuitously, 9, 81, 729, and 900 are all perfect squares; is about 2.2 10 7, and is actually much smaller than the × − and everything surviving involves only powers of 3 and Poisson approximation.) The student’s intuition was correct. of 10.) When all the smoke clears, all that remains is pr(9) 10 . ≈ 27√2π

June 2017 IEEE Information Theory Society Newsletter June 2017 IEEE Information Theory Society Newsletter

IEEE Information Theory Society Newsletter March 2004 it0604.qxd 6/3/04 9:45 AM Page 11

420 11 Reprinted from Vol. 54,54, No.No. 2,2, JuneJune 20042004 issueissue ofof InformationInformation TheoryTheory Newsletter Newsletter

GOLOMB’S PUZZLE COLUMN™ An InverseInverse Problem—Solutions Problem—Solutions Solomon W. Golomb

n We are asked to reconstruct a set S of n distinct positive real numbers, given only the set T consisting of the k sums of n 1 the k-element subsets of S. Let the elements of S be a1 < a2 < a3 < < an. Each ai occurs in exactly k−1 of the k-element ··· − n ! " subsets of S. Hence, the sum, a1 a2 an, of all the elements of S can be obtained by summing all k elements of T n 1 + +···+ ! " and then dividing by k−1 . − ! " ! " If n > k, the smallest element of T is a1 a2 ak, and the next-smallest element of T is a1 a2 ak 1 ak 1 . + +···+ + +···+ − + + Similarly, the largest element of T is an an 1 an k 1 , and the next-largest element of T is an an 1 + − +···+ − + + − + an k 2 an k. The remaining elements of T are partially ordered by magnitude. This partial ordering can usefully ···+ − + + − be shown by a graph, where the nodes are the elements of T (increasing in numerical magnitude from left to right), and the edges are labeled with the difference of the magnitudes of the nodes they connect. The only distinct edge labels will be α a a ,β a a ,γ a a , etc. These facts, and the corresponding graphs, will be used to solve problems 1 = 2 − 1 = 3 − 2 = 4 − 3 to 4 as follows.

1. n 4, k 2, T 24, 28, 30, 32, 34, 38 . = = ={ } + α a2 a3 γ β β a1 + a2 a1 + a3 a2 + a4 a3 + a4

γ a1 + a4 α

We know a a 24, a a 28, a a 34, a a 38, and β a a 4. There are two possibilities, 1 + 2 = 1 + 3 = 2 + 4 = 3+ 4 = = 3 − 2 = leading to two solutions. Either a a 30 and a a 32, or a a 32 and a a 30. In the former case 2 + 3 = 1 + 4 = 2 + 3 = 1 + 4 = α 2,γ 4, while in the latter case α 4,γ 2. In the former case, a a 30, a a β 4, and a 17, = = = = 3 + 2 = 3 − 2 = = 3 = giving a 13, a 11, and a 21. That is, a first solution is S 11, 13, 17, 21 . In the latter case, 2 = 1 = 4 = ={ } a a 32, a a β 4, and a 18, from which a 14, a 10, and a 20. That is, the second solution is 3 + 2 = 3 − 2 = = 3 = 2 = 1 = 4 = S 10, 14, 18, 20 . ={ }

2. n 5, k 2, T 21, 26, 28, 29, 31, 34, 36, 37, 42, 44 . With S a1, a2, a3, a4, a5 with a1 < a2 < a3 < a4 < a5, we have = = ={ } ={ } 5 1 a a a a a 328/4 82, where 328 is the sum of the elements in T, and 4 − . We know 1 + 2 + 3 + 4 + 5 = = = 2 1 a a 21, a a 26, a a 44, and a a 42. Also, a 82 (a a ) (a − a ) 82 21 44 17. 1 + 2 = 1 + 3 = 5 + 4 = 5 + 3 = 3 = − 1 + 2 − 4!+ "5 = − − = Then a 26 17 9 and a 42 17 25. Finally, a 21 9 12, and a 44 25 19. Hence the unique 1 = − = 5 = − = 2 = − = 4 = − = solution is S 9, 12, 17, 19, 25 . ={ } 3. n 6, k 2, T 32, 35, 37, 39, 41, 43, 44, 45, 48, 49, 51, 52, 54, 58, 62 . Here the graph is = = ={ }

ϵ a1 + a6 α + β δ a1 + a5 α ϵ a2 a6 γ a1 + a4 a3 + a6 α a2 + a5 β ϵ γ β a1 + a2 a1 + a3 a2 + a4 δ a3 + a5 a4 + a6 δ a5 + a6 γ γ ϵ α a2 + a3 β a3 + a4 δ a4 + a5

We know a a 32, a a 35,β 3, a a 62, a a 58,δ 4. Also, a a a a a 1 + 2 = 1 + 3 = = 5 + 6 = 4 + 6 = = 1 + 2 + 3 + 4 + 5 a 690/5 138, from which a a 138 (a a ) (a a ) 138 32 62 44 a a + 6 = = 3 + 4 = − 1 + 2 − 5 + 6 = − − = ; 2 + 5 = 138 (a a ) (a a ) 138 35 58 45, and a a 138 (a a ) (a a ) 138 45 44 49. Our − 1 + 3 − 4 + 6 = − − = 1 + 6 = − 2 + 5 − 3 + 4 = − − = graph now becomes

IEEEJune Information 2004 Theory Society Newsletter IEEE Information Theory Society NewsletterJune 2017 IEEE Information Theory Society Newsletter June 2017 it0604.qxd 6/3/04 9:45 AM Page 12

215

Reprinted12 fromfrom Vol.Vol. 54,54, No.No. 2, 2, June June 2004 2004 issue issue of of Information Information Theory Theory Newsletter Newsletter continued continued

ϵ 49 α 4 + + 3 a1 a5 α ϵ a2 a6 a1 + a4 a3 + a6 γ α 4 45 3 ϵ γ 4 323 35 41 48 58 62 γ 3 4 γ ϵ α a2 + a3 44 a4 + a5

From nodes 35 to 41, α γ 6. From nodes 48 to 58, γ ϵ 10. From nodes 45 to 49, α ϵ 4. (This is a dependent + = + = − + = set of 3 equations, so we do not yet have a unique solution.) The third-smallest element of T, 37, is either a a or 1 + 4 a a , so either γ 2 or α 2. The third-largest element of T, 54, is either a a or a a , so either γ 4 or ϵ 4. 2 + 3 = = 3 + 6 4 + 5 = = From the last two statements, there are three possibilities: i) γ 2,ϵ 4; ii) α 2,γ 4; iii) α 2,ϵ 4. Since ======γ ϵ 10 we rule out i). Since α ϵ 4 we rule out iii). That leaves only ii), with α γ 2 4 6. We can now + = − + = + = + = uniquely fill in the entire graph.

6 49 2 4 43 2651 3 39 54 4 4 2 4 45 3 6 3 32 35 41 48 584 62 4 6 2 37 4 3 44 4 52

Knowing which two elements of S were summed to obtain each element of T, we have 15 consistent linear equations in only 6 unknowns. One easy way to solve this system: a a β 3, a a 37, hence a 20, a 17, and 3 − 2 = = 3 + 2 = 3 = 2 = a 32 17 15. Also a a δ 4, a a 52, a 28, hence a 24, and a 62 28 34. Thus, the unique 1 = − = 5 − 4 = = 5 + 4 = 5 = 4 = 6 = − = solution is S 15, 17, 20, 24, 28, 34 . ={ } 4. n 6, k 3, T 49, 54, 56, 57, 58, 60, 61, 65, 66, 67, 68, 69, 70, 74, 75, 77, 78, 79, 81, 86 . We have a a a 49, = = ={ } 1 + 2 + 3 = a a a 54, a a a 86, a a a 81. Thus a a a a a a 135, and a a γ 5. The 1 + 2 + 4 = 6 + 5 + 4 = 6 + 5 + 3 = 1 + 2 + 3 + 4 + 5 + 6 = 4 − 3 = = graph, using only the subscripts of the summed ai’s to label the nodes, shows its central symmetry, and in fact it has the V4 symmetry group of the rectangle.

γ ϵ 126 β 136 146 δ 156 α α 125 ϵ α 256 δ β ϵ δ β γ γ γ γ 123 124 135 145 236 246 β 356 456 β δ α δ 134 α ϵ 346 α ϵ ϵ 234 235 245 345 δ γ β It is relatively easy to show that there are only two sets of six positive real numbers for which the sums of all 3-subsets yield the twenty elements of T. One is S 15, 16, 18, 23, 27, 36 , and the other is S 9, 18, 22, 27, 29, 30 . These cor- 1 ={ } 2 ={ } respond to two mirror-image assignments to the edges of the graph. S uses α 1,β 2,γ 5,δ 4,ϵ 9, while S 1 = = = = = 2 uses the reverse assignment α 9,β 4,γ 5,δ 2,ϵ 1. = = = = =

5. If k′ n k, the k′-subsets of S a1, a2,... ,an are precisely the complements (relative to S) of the k-subsets. Since we = − ={ } n 1 showed how to obtain the sum τ of all elements in S by dividing the sum of all elements of T by k−1 , if we replace the elements of T by τ minus each of these elements to obtain T , we see the equivalence of the two problems.− ! ′ " #$ 6. For n k 2, we have S a , a and T a a . From the single positive element in T, there are infinitely many = = ={ 1 2} ={ 1 + 2} ways (a continuum of ways) to represent it as a sum of two elements. For n 3, k 2, we have S a , a , a and = = ={ 1 2 3} T a a , a a , a a . If a < a < a then a a < a a < a a , so if we are given T r, s, t with r < s < t, ={ 1 + 2 1 + 3 2 + 3} 1 2 3 1 + 2 1 + 3 2 + 3 ={ } then we have a solvable system of three linear equations in three unknowns, with a unique solution. We saw in Problem 1 that the case n 4, k 2, has two solutions for S, while in Problems 2 and 3 we saw that n 5, k 2, and n 6, k 2, ======have unique solutions. For k 2 and n > 4 the reconstruction of S from T is unique. = 7. If n k > 1, then S contains at least two elements while T contains only one (the sum of all elements of S), and as in the = case n k 2, there are infinitely many solutions. = = 8. For k 2 and n 2k there will be two solutions for S, given T. This is the case where k k (in Problem 5), and the graph ≥ = ′ = has V4 symmetry, whereby each solution has a complementary solution. (In Problems 1 and 4, we saw the special cases n 4, k 2, and n 6, k 3.) = = = =

JuneIEEE 2017 Information Theory Society Newsletter IEEE Information Theory SocietyJune Newsletter 2004 June 2017 IEEE Information Theory Society Newsletter 10

226

Reprinted12 from Vol. 54, No. 3, September 2004 issue of Information Theory Newsletter 8Reprinted from Vol. 54, No. 3, September 2004 issue of Information Theory Newsletter

ReprintedParticipants from in Vol. the 54, 4th No. Asia-Europe 4, December 2004Workshop issue of in Information Lucca, Italy. Theory Newsletter

GOLOMB’S PUZZLE COLUMN™ GOLOMB’S PUZZLE COLUMN™ SOMESOME PRIME NUMBER PRPROPERTIES—SolutionsOPERTIES—Solutions Countable or Uncountable Solutions Solomon W. Golomb Solomon W. Golomb

In all three problems, S A is a collection of infinite sub- becomes 1, 2, 5, 11, 23, 47,... . For the present con- ={ i} { } setsHere (or, p infinitenth primesubsequences) number, andA of π the(x) =positive number integers. of primes struction,π(m 1) weπ ( canm). Thus use either the integers of these; skipped and in in fact the the n = i − = x, for positive real x. argumentsequence n is strengthenedπ(n) are preciselyif we the use terms both.) of Suppose the ≤ { + } 1. If A A whenever i j, then S is (at most) βsequenceα where n βpn 0.1b. b b b ... is the binary expan- i ∩ j =∅ ̸= ̸= { + =− } 2 3 4 5 countably infinite. n sion of β. The sequence Aβ has only finitely many terms 1. “Prove that the ratio π(n) , for n 2, takes every ≥ Note inthat common in problems with 1 and the 2, sequence the fact thatA ; p because,n is the since integer value > 1 at least once.” α { } Proof. Let M be the collection of smallest elements of sequenceα ofβ, thethere prime is a smallestnumbers t (ratherfor which than a tsomebt. otherThen not Proof. It was given in the Puzzle Column that ̸= ̸= all the sets Ai. Since the sets Ai are pairwise disjoint, subsequenceonly 1a 2ofa3 the positiveat 1b2 bintegers3 bt, butthat allbecomes subsequent less inte- π(x) π(x) ··· ̸= ··· eachlim one contributes0 and lim a differentpn positive. Thus the integer ratio to M; dense)gers plays in almostAα and no A role.β are different. Thus, Aα Aβ is a x x = n =∞ x ∩ soultimately →∞M is (at most) becomes countably→∞ and remains infinite; less but than the any elements finite set for every α β, and there is such a set Aα for 3. “Givenα positive integers̸= a and b, there exists a posi- of M are inϵ one-to-one> correspondenceπ (2with) 1the ele- every in the uncountably infinite set of real numbers assigned 0, as x . It starts at 2 2 . tive integer c such1 that infinitely many numbers of ments Ai of S. →∞ = in the interval ( 2 , 1). For any m 2, there is a unique largest prime the form an b (n a positive integer) have all their ≥ pk + 2. If pAk pAk(mis) finitefor which (or empty) π(pk) wheneverk . Thus, i j, it is pos- 3. Ifprime A Afactorshas at mostc.” m elements whenever i j, then i=∩ j = ≥ m ̸= i ∩ j ≤ ̸= sible for S to be uncountably infinite. Here is one such SProofis (at. All most) numbers countably in the infinite. sequence mπ(pk) mk pk. Either mk < pk 1 or mk pk 1. k = ≥ + ≥ + b(a 1) , k 1, 2, 3,... are distinct and of the form construction. For each real number α on the interval { + = } 1If mk < pk 1, and since an b. Thus c max(b, a 1) satisfies the condition ( 2 , 1), write+ the binary expansion of α in the form Proof+. Replace each= set Ai by+ the finite set Fi consisting pk mk,π(pk) π(mk)<π(pk 1), from which of the problem. α 0≤.1a2a3a4 ...≤, and associate+ with α the subse- of the m 1 smallest elements of Ai. Then if Ai Aj = mk + ̸= quenceπ(mk) A k, andof the positivem, so integersthat n mk consistingis an of we must have F F , because A and A can have at = α π(mk) = = i j i j n 4. (a) “What is the largest̸= integer N such that, if 1integer, 1a2, 1a for2a3 ,which1a2a3 a4,... mwhere. these are the binary most m common elements, by hypothesis. Hence there { π(n) =} 1 < k < N and k has no prime factor in common representations of integers. (For mk example,pk 1 is a one-to-one correspondence between the sets Ai If mk2 pk 1, then π(pk 1) k 1 > k m m+ , with N, then k is prime?” α ≥0.1010101+ ... +is = associated+ = with≥ the and the sets Fi. But the collection of all finite subsets of which= 3 = contradicts the choice of p as the largest prime Answer. N 30. Since sequences 1, 10, 101, 1010, 10101,...k of integers in the positive integers= is a countably infinite collection, { p } 30 2 3 5 p p p , the product of the binaryfor which representation, π(p) . . or 1 , 2 , 5 , 10, 21,... in deci- and a = fortiori× × the = collection1 × 2 × of3 subsets of m 1 ele- ≥ m { } first three primes, every k relatively prime to+ 30 mal notation.) For those real numbers with two repre- ments from the set of positive integers is a countably cannot be divisible by 2 or 3 or 5, and the smallest sentations, one “terminating” and one “repeating”, we infinite collection; so S A is (at most) a countably 2. “Every positive integer belongs to exactly one of the k > 1 divisible by none={ ofi} these and not prime is can use either representation; for specificity, let us use infinite collection. two sequences s n π(n) and p2 72 49, which is bigger than 30. the repeating representation.{ n}={ + (For} example, α 3 can 4 = = tn n pn 1 .” = 4 { }={ + − } beProof written. In the as landeither of 0.1100000... Primordia, orthe 0.101 sequence111111 ....p Foris Note the(b) similarity “What is the of thelargest solutions odd integer given N heresuch for that, Problem if 1 { n} theused former as a “tax representation table”, in the thesense sequence that the sales becomes tax and Problem1 < 3.k < N and k has no prime factor in common 1, 3, 6, 12, 24, 48,... ; for the latter representation, it { increases by one cent} at every term of the sequence with 2N, then k is prime?” p (and at no other values). Thus the sales tax on Answer. N 105. Since { n} = the price p is exactly k. More generally, the sales tax 105 3 5 7 p p p , the product of the k = × × = 2 × 3 × 4 on the price m is π(m), the number of terms of p first three odd primes, every odd k relatively prime { n} IEEE Informationnot Theoryexceeding Society m. Newsletter to 105 (i.e. every k relatively prime to December 2004 From this point of view, the “total price” (including 2 105 210) cannot be divisible by 2 or 3 or 5 or × = tax) on an item with a net price of n is n π(n). The 7, and the smallest k divisible by none of these + sequence n π(n) thus consists of all numbers and not prime is p2 112 121, which is bigger { + } 5 = = which can occur as “total prices”. What numbers than 105. cannot occur as “total prices”? As the net price increases through one of the terms of p , say from (Known results on the distribution of the primes prevent { n} p 1 to p , the total price increases from larger solutions than 30 and 105 to these problems.) n − n (p 1) (n 1) to p n, thus skipping the value n − + − n + p n 1. If m is not of the form p , then the total 5. “For what positive integers n is it true that n + − n price goes from (m 1) π(m 1) to m π(m), p n?” − + − + = increasing by only one cent, because in this case p π(n) ≤!

IEEEIEEE Information Information Theory Theory Society Society NewsletterNewsletter SeptemberJune 2004 2017

IEEE Information Theory Society Newsletter June 2017 237

Reprinted from Vol. 54, No. 3, September 2004 issue of Information Theory Newsletter continued Reprinted from Vol. 54, No. 3, September 2004 issue of Information Theory Newsletter continued 13

Answer. n 5, 17, 41, 77, 100 . For ”large” n, p > n, density of the sequence of prime numbers) to ={ } p π(n) contain only a small finite number of prime ≤! and n 100 is the last value for which equality holds. numbers, whereas our uniform bound B can be = 10100 chosen arbitrarily large (e.g. B 1010 ). Anyone (For a more detailed solution, see the Solution to Problem = who can exhibit a specific sequence a such that E3385 (American Math. Monthly) listed as Reference 5 at { k} all its translates provably contain no more than B the end of these Solutions.) primes each (for a specific B, however large) will 6. Let a < a < a < be an increasing, infinite earn a permanent place in the history of prime 1 2 3 ··· sequence of positive integers.” number theory.

(a) “Construct such a sequence a having the All the problems in this set are based on articles or prob- { k} property that, for every integer n (positive, lems which I published (over several decades) in either the negative, or zero) the sequence a n contains American Mathematical Monthly (AMM) or in { k + } only finitely many prime numbers.” Mathematics Magazine (Math. Mag.). Construction. Let a ((2k)!)3 for { k}= k 1, 2, 3,.... For A a n , if n 2 then all 1. “On the ratio of N to π(N)”, AMM, vol. 69, no. 1, Jan. = n ={" k + }# | |≥ terms of A with k n are divisible by n, and 1962, 36-37. n ≥ hence not prime. For n 0, A a is clearly = n ={ k} composite for all k 1. Finally, using 2. “The ‘Sales Tax’ Theorem”, Math. Mag., vol. 49, no. 4, ≥ x3 1 (x 1)(x2 x 1)and Sep.-Oct. 1976, 187-189. + = + − + x3 1 (x 1)(x2 x 1), the values of A for − = − + + n n 1 are composite for all k 2. 3. Problem E2725, AMM, vol. 85, no. 7, Aug.-Sep., 1978, =± ≥ (Many other constructions are possible.) p. 593. Solution, vol. 86, no. 9, November, 1979, p. 790. (b) “Is there such a sequence a and a constant B > 0 { k} such that, for every integer n (positive, negative, 4. Problem E3137, AMM, vol. 93, no. 3, March, 1986, p. or zero) the sequence A a n contains no 215. Solution, vol. 94, no. 9, November, 1987, p. 883. n ={ k + } more than B prime numbers?” 5. Problem E3385, AMM, vol. 97, no. 5, May, 1990, p. The answer to this is unknown. A “yes” answer 427. Solution, vol. 98, no. 9, November, 1991, pp. 858- would contradict the “prime k-tuples” conjecture, 859. which the late Paul Erdös was convinced had to be true. However, at least two other plausible 6. Problem 10208, AMM, vol. 99, no. 3, March, 1992, p. conjectures in prime number theory also 266. Solution, vol. 102, no. 4, April, 1995, pp. 361-362. contradict the “prime k-tuples” conjecture. On the model of the construction given in 6.(a) For further information about Problem 6.(b) and its relation to the “prime k-tuples conjecture”, see Paolo above, let a ((2k)!)3F(k) , where F(k) can be { k}={ } Ribenboim, The Little Book of Bigger Primes, Second an integer-valued function that grows Edition, Springer, New York, 2004, pp. 201-204, uncomputably fast. Then each translate sequence where the existence of a solution to 6.(b) is called An a n will be “expected” (by the thinning “Golomb’s Conjecture”. ={ k + }

June 2017 IEEE Information Theory Society Newsletter June 2017 IEEE Information Theory Society Newsletter

September 2004 IEEE Information Theory Society Newsletter 10

824

ReprintedParticipants from in Vol. the 54, 4th No. Asia-Europe 4, December 2004Workshop issue of in Information Lucca, Italy. Theory Newsletter

GOLOMB’S PUZZLE COLUMN™ CountableCountable oror UncountableUncountable Solutions Solutions Solomon W. Golomb

In all three problems, S A is a collection of infinite sub- becomes 1, 2, 5, 11, 23, 47,... . For the present con- ={ i} { } sets (or, infinite subsequences) Ai of the positive integers. struction, we can use either of these; and in fact the argument is strengthened if we use both.) Suppose 1. If A A whenever i j, then S is (at most) β α where β 0.1b b b b ... is the binary expan- i ∩ j =∅ ̸= ̸= = 2 3 4 5 countably infinite. sion of β. The sequence Aβ has only finitely many terms in common with the sequence Aα; because, since Proof. Let M be the collection of smallest elements of α β, there is a smallest t for which a b . Then not ̸= t ̸= t all the sets A . Since the sets A are pairwise disjoint, only 1a a a 1b b b , but all subsequent inte- i i 2 3 ··· t ̸= 2 3 ··· t each one contributes a different positive integer to M; gers in Aα and Aβ are different. Thus, Aα Aβ is a ∩ so M is (at most) countably infinite; but the elements finite set for every α β, and there is such a set Aα for ̸= of M are in one-to-one correspondence with the ele- every α in the uncountably infinite set of real numbers 1 ments Ai of S. in the interval ( 2 , 1).

2. If A A is finite (or empty) whenever i j, it is pos- 3. If A A has at most m elements whenever i j, then i ∩ j ̸= i ∩ j ̸= sible for S to be uncountably infinite. Here is one such S is (at most) countably infinite. construction. For each real number α on the interval 1 ( 2 , 1), write the binary expansion of α in the form Proof. Replace each set Ai by the finite set Fi consisting α 0.1a a a ..., and associate with α the subse- of the m 1 smallest elements of A . Then if A A = 2 3 4 + i i ̸= j quence Aα of the positive integers consisting of we must have F F , because A and A can have at i ̸= j i j 1, 1a , 1a a , 1a a a ,... where these are the binary most m common elements, by hypothesis. Hence there { 2 2 3 2 3 4 } representations of integers. (For example, is a one-to-one correspondence between the sets Ai α 2 0.1010101 ... is associated with the and the sets F . But the collection of all finite subsets of = 3 = i sequences 1, 10, 101, 1010, 10101,... of integers in the positive integers is a countably infinite collection, { } binary representation, or 1, 2, 5, 10, 21,... in deci- and a fortiori the collection of subsets of m 1 ele- { } + mal notation.) For those real numbers with two repre- ments from the set of positive integers is a countably sentations, one “terminating” and one “repeating”, we infinite collection; so S A is (at most) a countably ={ i} can use either representation; for specificity, let us use infinite collection. the repeating representation. (For example, α 3 can = 4 be written as either 0.1100000... or 0.101111111.... For Note the similarity of the solutions given here for Problem 1 the former representation the sequence becomes and Problem 3. 1, 3, 6, 12, 24, 48,... ; for the latter representation, it { }

IEEE Information Theory Society Newsletter December 2004

IEEE Information Theory Society Newsletter June 2017 itNL0305.qxd 3/3/05 1:42 PM Page 9

259 9 Reprinted from Vol. 55, No. 1, March 2005 issue of Information Theory Newsletter

GOLOMB’S PUZZLE COLUMN™ A Quadratic Sequence Solutions Solomon W. Golomb

The problems concern the sequence S s 2n2 2n 1 . Note that s 2n2 2n 1 n2 (n 1)2 ((2n 1)2 1)/2.In ={ n}={ + + } n = + + = + + = + + particular, each sn is a sum of two consecutive squares. The following facts are well-known from elementary number theory:

a. The primes which are sums of two squares are those of the form 4m 1, and 2. + b. If u and v are relatively prime, all prime factors of u2 v2 are primes which are sums of two squares. If further + u2 v2 is odd, its prime factors must all be of the form 4m 1. + + c. For primes p of the form 4m 1, the number 1 is a quadratic residue modulo p. That is, there is a + − number a such that a2 1(mod p); and then also b p a satisfies b2 1(mod p). ≡− = − ≡− Now for the solutions.

1. Since n and n 1 are relatively prime, and s n2 (n 1)2 is odd, all prime factors of s are primes of the form 4m 1, + n = + + n + for every sn.

2. and 3. Given any p 4m 1, a prime, we will find values of n such that p divides s ((2n 1)2 1)/2, i.e., such that = + n = + + ((2n 1)2 1)/2 0(mod p).Multiplying both sides by 2, this says (2n 1)2 1(mod p), and by fact c., there are num- + + ≡ + ≡− bers a and d p a with a2 d2 1(mod p). Since a d p, which is odd, one of a and d must be odd, say a . 0 0 = − 0 0 ≡ 0 ≡− 0 + 0 = 0 0 0 Then take a0 2n 1, so that n (a0 1)/2.For this value of n,and all other n congruent to it modulo p,sn is a multiple of p, i.e., 2 = + = − 2a 2a 1 is a multiple of p for all a a0(mod p); i.e., all a a0 kp for all integers k. Now take b0 p a0 1. For + + 2 ≡2 = 2 + 2 = − − n b0, sn 2b 2b0 1 2(p a0 1) 2(p a0 1) 1 2a 4a0 2 2a0 2 1 2a 2a0 1 0(mod p). = = 0 + + = − − + − − + ≡ 0 + + − − + ≡ 0 + +p 1≡ Thus p divides sn for n b0 and for all n b0(mod p). Note also that a0 b0, for otherwise a0 b0 − , leading to p 1 = ≡ ̸= = = 2 a2 b2 ( − )2 1(mod p), from which (p 1)2 4(mod p); but (p 1)2 ( 1)2 1 4(mod p), and 0 ≡ 0 ≡ 2 ≡− − ≡− − ≡ − ≡ ≡− 5 0(mod p), which happens only for p 5. However, looking at the actual sequence S, sn is not divisible by 5 when ≡ 5 1 = n 2 − , but when n a 1, and when n = b = 5 - 1 - 1 = 3. = = 2 = = 4. For s n2 (n 1)2 c2, we have a “Pythagorean triple” of the special form (n, n 1, c), such as (3, 4, 5) and (20, 21, n = + + = + 29). We find the general solution as follows: Suppose s ((2n 1)2 1)/2 y2 for some y. Set x 2n 1, so that n = + + = = + x2 1 2y2, x2 2y2 1.This is a case of “Pell’s equation”, which we now solve. The smallest solution has x y 1, + = − =− 0 = 0 = so that 12 2 12 1. We factor the Pell equation to get (x √2y)(x √2y) 1, and then − · =− + − =− (x √2y)n(x √2y)n ( 1)n. At n 2, we have (x √2y )2 (1 √2)2 3 2√2, for x 3, y 2, as in + − = − = 1 + 1 = + = + 2 = 2 = 32 2 22 ( 1)2 1. At n 3,(1 √2)3 7 5√2,corresponding to 72 2 52 ( 1)3 1.It is only for odd val- − · = − =+ = + = + − · = − =− ues of n that we get x2 2y2 1. Remember that x 2n 1, so that n (x 1)/2. From x 1, n 0, and n − n =− = + = − 1 = 1 = s 02 12 12. From x 7, n 3, and s 32 42 52 . At n 4, we have (1 √2)4 17 12√2, as in 0 = + = 3 = 3 = 3 = + = = + = + 172 2 122 ( 1)4 1. At n 5, we have (1 √2)5 41 29√2. From x 41, n 20, and − · = − =+ = + = + = = s 202 212 841 292 . Next, (1 √2)6 99 7√2, where 992 2 702 1; but (1 √2)7 239 169√2, where 20 = + = = + = + − · =+ + = + 2392 2 1692 1. With x 239, n 119, and s 1192 1202 28, 561 1692. − · =− = 5 = 119 = + = =

5. Thus, the sub-sequence C = {c1, c2, c3, c4,...} of square roots of the squares in sequence S begins C = {1, 5, 29, 169,...},

and can easily be generated recursively by: c0 1, cn 1 6cn cn 1 for n 1. (This can be proved inductively from = + = − − ≥ the Pell equation approach. It yields a much simpler way of obtaining the values of the cn’s.) Thus c = {1, 5, 29, 169, 985, 5741, 33461, 195025, 1136689, 6625109, 38613965, 225058681, 1311738121, 7645370045, 44560482149, 259717522849,... . The } 2 recursion cn 1 6cn cn 1 corresponds to the polynomial equation x 6x 1 0, with roots 3 2√2. Let + = − − − + = ± ρ 3 2√2 5.8284271247 .... Then c n 1 ρ c n for all n 0,and lim (cn 1/cn) ρ. To get the sn corresponding to = + = + =2 ⌊ ⌋ 2 ≥ n + = cn cn 2 →∞cn cn cn (not the same values of n) we have c , where 1 . It is remarkable how close the √2 + √2 = n √2 + = √2 (irrational!) values of ρn are to integers.!" (Actually,#$ !%ρn &$ρ n is an integer for" every# n %1, and& the powers of ρ 1 3 2√2 + − ≥ − = − go to 0 very rapidly with n.)

JuneIEEE 2017Information Theory Society Newsletter IEEEIEEE Information Theory SocietyMarch Newsletter 2005 itNL0305.qxd 3/3/05 1:42 PM Page 10

2610

10Reprinted from Vol. 55, No. 1, March 2005 issue of Information Theory Newsletter continued

6. If any s is a perfect even power it must be in the sequence C. Among the first 250 terms of s , the only power high- n { n} er than the second power is s 28, 561 134. I don’t know of any odd (perfect) powers in S, or other perfect even 119 = = powers, but they may well exist.

7. It is “very likely” that the sequence S s contains infinitely many primes. However, this has not yet been proved ={ n} for any quadratic expression in n. The “twin primes” result from removing, from the set of all odd integers > 0, two residue classes modulo every prime p > 2, and no one has yet been able to prove that there are infinitely many twin primes. In the quadratic sequence S, of a sparse subset of the odd integers > 0, we remove two residue classes modulo those primes of the form 4m 1 to see what (prime) values remain; so proving that S contains infinitely many prime + values seems at least as hard as (or harder than) the “twin prime” problem.

Vince Poor Wins IEEE Education Medal

H. Vincent Poor has been named recipient of the The Medal is presented annually for a career of 1995 IEEE James H. Mulligan Education Medal outstanding contributions to education in the with the citation: fields of interest of IEEE. It is presented only to an individual. “For leadership in electrical engineering education through inspired teaching, a classic In the evaluation process, the following crite- textbook, innovative curricular development, ria are considered: Excellence in teaching and research, and mentoring.” ability to inspire students, Leadership in electrical engineering education through publica- The IEEE James H. Mulligan Education Medal (for- tion of course materials and writings on engi- merly IEEE Education Medal) was established in 1956 neering education, Leadership in the develop- by the American Institute of Electrical Engineers, and ment of programs in curricula or teaching continued by the Board of Directors of the IEEE. methodology, Contributions to the engineering profession through research, engineering It is through this Medal that the Institute recognizes achievements, and technical papers, and the importance of the educator's contributions to Participating in the education activities of pro- the vitality, imagination, and leadership of the H. Vincent Poor fessional societies. members of the engineering profession.

IT Members Win Several IEEE Awards

Several IT Society Members have been awarded 2005 IEEE Medals. lic key cryptography.”

Eugene Wong won the 2005 IEEE Founders Medal with the citation: Neil Sloane's won the 2005 Hamming Medal with the citation:

“For leadership in national and international engineering research “For contributions to coding theory and its applications to com- and technology policy, for pioneering contributions in relational munications, computer science, mathematics and statistics.” databases.” H. Vincent Poor has been named recipient of the 2005 IEEE James Jim K. Omura won the 2005 IEEE Alexander Graham Bell Medal H. Mulligan Education Medal with the citation: with the citation: For leadership in electrical engineering education through inspired “For contributions to the theory of communication systems and teaching, a classic textbook, innovative curricular development, the commercial applications of spread spectrum radios and pub- research, and mentoring.

IEEE Information Theory Society Newsletter June 2017 IEEE Information Theory Society Newsletter June 2017

IEEE Information Theory Society Newsletter March 2005 2711

Reprinted from Vol. 55, No. 2, June 2005 issue of Information Theory Newsletter

June 2017 IEEE Information Theory Society Newsletter June 2017 IEEE Information Theory Society Newsletter 1228

Reprinted from Vol. 55, No. 2, June 2005 issue of Information Theory Newsletter continued

IEEE Information Theory Society Newsletter June 2017 itNL0905.qxd 10/4/05 10:46 AM Page 11

1329

Reprinted from Vol. 55, No. 3, September 2005 issue of Information Theory Newsletter 11

GOLOMB’S PUZZLE COLUMN™ SomeSome MatrixMatrix Questions Questions Solutions Solutions Solomon W. Golomb

11 01 10 10 1. The matrices A and B are similar, since P 1AP B with P and P 1 . = 00 = 01 − = = 11 − = 11 ! " ! " ! − " ! " 02 00 00 However, AB and BA are not similar, since clearly is similar only to itself. = 00 = 00 00 ! " ! " ! "

2. Theorem. If, for a given complex matrix M, there exists a unitary matrix U such that U 1MU !, where ! is a diagonal matrix, − = then M is normal.

Proof. From U 1MU !, we have ! !H (U 1MU)H UHMH(U 1)H U 1MHU. Now !! ! !, because if ! − = ∗ = = − = − = − ∗ = ∗ is the diagonal matrix with λ1,λ2,... ,λn as its diagonal elements, then !∗ is the diagonal matrix with λ1∗,λ2∗,... ,λ∗n as its diagonal elements, and both !! and ! ! are diagonal matrices with λ 2, λ 2,... , λ 2 as their diagonal elements. ∗ ∗ | 1| | 2| | n| Now !! (U 1MU)(U 1MHU) U 1(MMH)U,! ! (U 1MHU)(U 1MU) U 1(MHM)U , and since these are ∗ = − − = − ∗ = − − = − equal, MMH MHM, so M is normal. =

3. “If N and N are normal n n matrices then N N is normal” is false. For a counter-example, we can use the fact that every 1 2 × 1 2 12 (real) symmetric matrix S is normal, since SH ST S, from which SSH S2 SHS. Then with N and = = = = 1 = 23 ! " 21 49 47 49 N , we have P N N and PH PT NTNT N N . Now PPH 2 = 14 = 1 2 = 7 14 = = 2 1 = 2 1 = 9 14 = 7 14 × ! " ! " ! " ! " 47 97 154 47 49 65 134 but PHP , so that P N N is not normal. 9 14 = 154 245 = 9 14 7 14 = 134 277 = 1 2 ! " ! " ! "! " ! "

00 01 10 11 4. Let R , , , O, Z, I, J over GF(2). R forms a commutative group with respect to = 00 00 00 00 ={ } #! " ! " ! " ! "$ matrix addition modulo 2, and R is closed under matrix multiplication modulo 2. In this ring, both I and J are “left identities”,

10 ab ab 11 ab ab since and , but neither one is a “right identity”, since 00 00= 00 00 00= 00 ! "! " ! " ! "! " ! " 01 10 00 01 11 00 ZI 0, and ZJ O. = 00 00= 00= = 00 00= 00= ! "! " ! " ! "! " ! " 5. “If the n2 elements of an n n matrix A are integers chosen independently and at random, what is the probability that A , the × | | determinant of A, is odd?”

For all n 2, the answer is not one-half. Rather than worry about what sample space integers can be chosen from ”inde- ≥ pendently and at random”, we need only agree that each entry in A is equally likely to be even or odd, and independently of the other entries. Then our question is equivalent to: “What fraction of the n n matrices over GF(2) are non-sin- × gular?” (The even determinants all reduce to 0, and the odd determinants all reduce to 1, modulo 2.) To form a non-singular n n matrix over GF(2), we can pick the top row in 2n 1 ways (only the all-zeroes case is excluded), the second × − n n th n j 1 n j 1 row in 2 2 ways, and the j row in 2 2 − ways, for all j, 1 j n. This gives (2 2 − ) non-singular matrices, out − − ≤ ≤ j 1 n − (n2) = j 1 3 7 2n 1 of 2 matrices altogether. Then the probability is the ratio, which simplifies to% (1 2− ) 2 4 8 2−n . This j 1 − = · · ··· sequence of probabilities, 1 , 3 , 21 , 315 , , converges rather rapidly to%= a positive limiting value, 2 8 64 1024 ··· ∞ 1 1 0.2887878 .... Thus, the probability that a “large” n n matrix of “random” integers would have an odd 2 j j 1 − = × determinant%= & ' is a bit less than 29%. (At n 8, this probability is already down to 0.289919 ....) =

September 2005 IEEE Information Theory Society Newsletter JuneJune 20172017 IIEEEEEE InformationInformation TheoryTheory SocietySociety NewsletterNewsletter 10

[1] http://www.nsf.gov/div/index.jsp?org=OISE [4] David Goodman, Joan Borras, Narayan Mandayam, Roy Yates, "Infostations: A New System Model for Data and Messaging [2] http://www.nsf.gov/cise/geni/ Services", 1997 IEEE VTC, vol. 2, pp. 969-973

[3] Peter Freeman, “The GENI Initiative,” to appear in Computing [5] http://www.nsf.gov/cise/ccf/sod_pres_video.jsp Research News November 2005 [6] Michael Frayn, Sweet Dreams, Collins, 1973

1430

Reprinted from Vol. 55, No. 4, December 2005 issue of Information Theory Newsletter

GOLOMB’S PUZZLE COLUMN™ SomeSome Matrix QuestionsQuestions Solutions Solutions Solomon W. Golomb

1. It is sufficient to remove only one edge from K6 so that 40 of the 45 edges of K10 can be 2-colored without form- the fourteen remaining edges can be colored using two ing a solid-color triangle. (To convince yourself that no colors without forming a solid-color triangle: forbidden triangles are formed, adjoin the new “clone points” one at a time.) 1

3. The question told you that all the edges of K16 can be 3- colored without forming a solid-color triangle, but that 2 5 this is not true for K17. Hence, we can adjoin a clone P′ 6 for one of the points P of K16, and connect P′ to every original point Q of K16 except P, with the same color edge as the edge connecting P to Q, without forming a solid-color triangle. That is, we need remove only one 34 edge from K17 so that the remaining 135 edges can be 3- Only the edge connecting the points 1 and 6 is missing colored without forming a solid-color triangle. from K6. The clever way to view this example is to consider that we started with a triangle-free 2-coloring of 4. The same reasoning shows that if r r(c) is the smallest r = K5, on the points 1, 2, 3, 4, and 5, and then adjoined 6 as positive integer such that, if the 2 edges of Kr are colored a “clone” of point 1. That is, 6 is connected to each of 2, using c colors a solid-color triangle must be created, then ! " 3, 4, and 5 with the same colors of edges as those ema- it suffices to remove a single edge from Kr so that the nating from 1; so if a solid-color triangle involving 6 is remaining edges can be c-colored without forming a formed, there would already have been a solid-color tri- solid-color triangle. Specifically, we start with a c-coloring angle involving 1. of Kr 1 that has no solid-color triangle, and adjoin a new point− P as a clone of P, one of the original r 1 points of ′ − 2. In a similar way, we can adjoin clones for each of the five Kr 1, and proceed as in Solutions 1 and 3 above. original points 1, 2, 3, 4, 5. Call these new points 1’, 2’, 3’, − 4’, 5’. Then the only lines missing from K10 are the five lines connecting each original point to its clone. That is, Note. A key idea for this column was supplied by Herbert Taylor.

IEEE Information Theory Society Newsletter December 2005

IEEEIEEE InformationInformation TheoryTheory SocietySociety NewsletterNewsletter June 2017 10

itNL0306.qxd 3/13/06 11:06 AM Page 10

8 1531 10 ReprintedReprintedParticipants from from Vol. in Vol.56, the No. 54, 4th 1,No. Asia-EuropeMarch 4, December 2006 issue 2004Workshop of issueInformation of in Information Lucca, Theory Italy. NewsletterTheory Newsletter

GOLOMB’SGOLOMB’S PUZZLE PUZZLE COLUMN™ COLUMN™ SimpleCountable ProbabilitiesProbabilities or Uncountable Solution Solution Solutions SolomonSolomon W. W. Golomb Golomb

In all three problems, S A is a collection of infinite sub- becomes 1, 2, 5, 11, 23, 47,... . For the present con- ={ i} { } sets (or, infinite subsequences) A of the positive integers. struction, we can use either of these; and in fact the 1. After the first four cards dealt arei all seen to be hearts, there are six head faces, two of which belong to the argument is strengthened if we use both.) Suppose the deck still contains 9 hearts among 48 cards, so the crooked coin; so the probability that the unseen side is 1. If A A whenever i j, then S is (at most) β α where β 0.1b2b3b4b5 ... is the binary expan- probabilityi ∩ thatj = the∅ fifth card will̸= also be a heart is also heads̸= is 2 0=.333333 .... countably infinite. sion of β. The6 = sequence Aβ has only finitely many terms 9 3 0.1875. 48 = 16 = in common with the sequence Aα; because, since 7. Expected number of tosses of a pair of dice to see a total Proof. Let M be the collection of smallest elements of α β, there is a smallest t for which at bt. Then not 2. The number of ways to select 5 cards, all hearts, is of k, 2̸= k 12, on the two dice, is E(k), where̸= all the sets Ai. Since the sets Ai are pairwise disjoint, only≤ 1a≤2a3 at 1b2b3 bt, but all subsequent inte- 13 1287. The number of 5-card hands with at least ··· ̸= ··· 5 =each one contributes a different positive integer to M; gers in Aα and Aβ are different. Thus, Aα Aβ is a four hearts is the number of hands with 5 hearts plus the ∩ ! " so M is (at most) countably infinite; but the elements finite set for every α β, and there is such a set Aα for number of hands with 4 hearts and one non-heart, ̸= of M are in13 one-to-one13 39 correspondence with the ele- every α in the uncountably infinite set of real numbers which equals 1287 27, 885 29, 172. kE(k) kE1 (k) kE(k) ments Ai of5 S+. 4 1 = + = in the interval ( , 1). Thus the probability that a 5-card hand with at least 2 24.6051 52 5.8849 9 5.8849 ! " ! "! " four hearts actually contains five hearts is 3 12.1268 6 4.6355 10 7.9662 2. If Ai Aj is finite (or empty) whenever i j, it is pos- 3. If Ai Aj has at most m elements whenever i j, then 1287 3 ∩ 0.044117647 .... ̸= 4 7.9662∩ 7 3.8018 11 12.1268 ̸= 29,172 sible= 68 for= S to be uncountably infinite. Here is one such S is (at most) countably infinite. construction. For each real number α on the interval 8 4.6355 12 24.6051 1 3. (a) For( 2at, 1least), writeone of the six binary dice to expansionshow a 5, the of αprobabilityin the form Proof. Replace each set Ai by the finite set Fi consisting 6 , α 05.1a2a331a4 ...031 , and associate with α the subse- of the m 1 smallest elementsE(k )of Ai. Then if Ai Aj is 1 = 6 46,656 0.6651020233 ..., or nearly two- + 36 k 1 1 ̸= quence− =Aα of the= positive integers consisting of E(kwe) is mustthe solution have F to F , −because+ A and, and A canis given have by at thirds.# $ i ̸= j 36 =i 2 j 1, 1a , 1a a , 1a a a ,... where these are the binary most m common elements, by hypothesis. Hence there { 2 2 3 2 3 4 } E(k) log 2/(log 36 log# (36 $k 1)) for 2 k 7, and (b) Forr epresentationsexactly one of six dice of to integers.show a 5 , there (For is a choice example, is= a one-to-one correspondence− − + between≤ the≤ sets Ai 2 α 0.1010101 ... is associated5 with the E(7anda )the Esets(7 Fi.a )Butfor the 1 collectiona 5. of all finite subsets of of which= 3of= the six dice shows the , and the other dice + = − ≤ ≤ sequences 1, 10, 101, 1010, 10101,... of integers in the positive integers is a countably infinite collection, must avoid showing{ a 5; so the probability} is: binary representation, or 1, 2, 5, 10, 21,... in deci- and a fortiori the collection of subsets of m 1 ele- 6 55 55 3125 { } + ×6 5 0.40187757 .... 8. Although the grass always looks greener on the other 6 mal= notation.)6 = 7776 For= those real numbers with two repre- ments from the set of positive integers is a countably sentations, one “terminating” and one “repeating”, we sideinfinite of the collection;fence, it can't so S be trueA mathematicallyis (at most) a countably that no ={ i} 4. (a) Thecan probability use either that representation; all six dice forturn specificity, up the same let us is use matterinfinite which collection.of the two envelopes you picked, the other 6 1 3 6 the repeating0.000128601 representation..... (For example, α can one is 25% better (at least in expectation)! So where is 6 = 7776 = = 4 (b) Thebe probabilitywritten as either that all 0.1100000... six dice turnor 0.101 up111111 different.... For Notethe the fallacy? similarity The of problem the solutions stated given that herex, a forpositive Problem real 1 (i.e.the that former every representation number from the 1 to sequence 6 appears) becomes is andnumber, Problem was 3. picked “at random”, but it didn't specify 6! 5 6 1, 3, 6, 120,.01543209924, 48,...... ; for. the latter representation, it the distribution from which x was selected. The rea- 6 = 324 = { } soning tacitly assumed it was from the “uniform dis- 5. Originally, your chance of guessing right is one in four, tribution”, but there is no uniform distribution on the or 25%. After two wrong alternatives are removed, your positive real numbers. Your best strategy is to assume IEEE Informationoriginal choice Theory is still Society only Newsletter25% right, so by switching (best guess) what the mean of the unrevealedDecember distribu- 2004 you increase your winning probability to 75%. (The two tion is, accept y if it exceeds this mean, but switch if it remaining doors are not equally likely!) is below. Another defensible strategy is to accept y if you would be satisfied with that amount of money, but 6. Among the four honest coins plus one two-headed coin, reject y otherwise (the “minimum regret” approach).

JuneJuneIEEE 20172017 Information Theory Society Newsletter IIEEEEEE InformationInformation TheoryTheory SocietySociety NewsletterNewsletterJune 2017 IEEE Information Theory Society Newsletter March 2006 itNL0606.qxd 5/22/06 2:29 PM Page 25

REFERENCES:

[1] http://www.nsf.gov/pubs/2006/nsf06542/nsf06542.htm [6] http://www.knowledgebasednetworking.org/

[2] http://www.nsf.gov/cise/geni/ [7] http://www.nsf.gov/pubs/2006/nsf06525/nsf06525.htm

[3] http://www.geni.net [8] http://www.igert.org/programs.asp

[4] http://www.nsf.gov/pubs/2006/nsf06551/nsf06551.htm [9] http://eeweb.poly.edu/faculty/goodman/

32[5] http://csc-ballston.dmeid.org/darpa/registration/ [10] Kim Stanley Robinson, Forty Signs of Rain, Publisher: intro.asp?regCode=yujjuryE Spectra; Reprint edition July 26, 2005, ISBN: 0553585800 Reprinted from Vol. 56, No. 2, June 2006 issue of Information Theory Newsletter

GOLOMB’S PUZZLE COLUMN™ Mini-Sudoku Solution So l o m o n W. G o l o m b

1. There are 288 12 4! distinct Mini-Sudoku solutions. The factor 4! 24 corresponds to all per- = × = mutations of the four symbols. Here are the 12 cases that differ by more than permutation of the symbols.

1 2 34 1 2 34 1 2 34 1 2 34 1 2 34 1 2 34 2 2 1 3 4 2 1 1. 3 4 1 2. 3 4 1 2 3. 3 4 1 2 4. 3 4 1 5. 3 4 2 6. 214 3 234 1 412 3 432 1 214 3 431 2 423 1 421 3 243 1 241 3 413 2 241 3

1 2 34 1 2 34 1 2 34 1 2 34 1 2 34 1 2 34 4 3 2 1 4 3 2 1 4 3 2 1 4 3 2 1 4 3 1 2 4 3 1 2 7. 8. 9. 10. 11. 12. 214 3 241 3 314 2 341 2 214 3 342 1 314 2 341 2 214 3 241 3 324 1 241 3

2. Here is a Mini-Sudoku solution in which the four elements on each of the two diagonals are also distinct: (It is the seventh of the twelve cases shown above. The fourth case also has this property.)

1 2 34

4 3 2 1

231 4

314 2

3. At least four cells must be filled in to guarantee a unique Mini-Sudoku solution. Three distinct symbols must appear, since otherwise two unused symbols could be interchanged in the filled-in solution, destroying uniqueness. I tried all inequivalent ways of placing one each of 1, 2, and 3 in the 4 4 grid, and none of these led to a unique Mini-Sudoku solution. × There are many ways to place four symbols that will guarantee a unique solution. Here is one of them:

1 1 2 34 2 3 4 1 2

3 413 2 , which forces . 3 231 4

June 2006 IEEE Information Theory Society Newsletter

IEEE Information Theory Society Newsletter June 2017 1733

Reprinted from Vol. 56, No. 2, June 2006 issue of Information Theory Newsletter continued

June 2017 IEEE Information Theory Society Newsletter June 2017 IEEE Information Theory Society Newsletter 10

34188 Reprinted from Vol. 56, No. 3, September 2006 issue of Information Theory Newsletter ReprintedReprintedParticipants from from Vol. in Vol.56, the No. 54, 4th 3,No. Asia-EuropeSeptember 4, December 2006 2004Workshop issue issue of Information of in Information Lucca, Theory Italy. Theory Newsletter Newsletter

GOLOMB’S PUZZLE COLUMN™ Countable or Uncountable Solutions Solomon W. Golomb

IEEE Information Theory Society Newsletter December 2004

IEEE Information Theory Society Newsletter June 2017 IEEEIEEE Information Information Theory Theory Society Society Newsletter Newsletter JuneJune 2017 2017 10

1935 8 Reprinted from Vol. 56, No. 4, December 2006 issue of Information Theory Newsletter ReprintedParticipants from in Vol. the 54, 4th No. Asia-Europe 4, December 2004Workshop issue of in Information Lucca, Italy. Theory Newsletter

GOLOMB’S PUZZLE COLUMN™ Countable or Uncountable Solutions Solomon W. Golomb

IEEE Information Theory Society Newsletter December 2004

JuneJune 20172017 IIEEEEEE InformationInformation TheoryTheory SocietySociety NewsletterNewsletter IEEE Information Theory Society Newsletter June 2017 3620

Reprinted from Vol. 56, No. 4, December 2006 issue of Information Theory Newsletter continued

IEEE Information Theory Society Newsletter June 2017 10

8 37

Participants in the 4th Asia-Europe Workshop in Lucca, Italy. ReprintedReprinted from from Vol. Vol.57, No.54, No.1, March 4, December 2007 issue 2004 of issueInformation of Information Theory TheoryNewsletter Newsletter

GOLOMB’S PUZZLE COLUMN™ THECountable 3X+1PRO BorLEM Uncountable Solutions Solomon W. Golomb

IEEE Information Theory Society Newsletter December 2004

JuneIEEE 2017 Information Theory Society Newsletter IEEE Information Theory Society NewsletterJune 2017 itNL0607.qxd 7/16/07itNL0607.qxd 1:32 PM 7/16/07 Page 23 1:32 PM Page 23

3822 22 23 23 Reprinted from Vol. 57,Reprinted No. 2, June from 2007 Vol. issue57, No. of Information2, June 2007 Theory issue of Newsletter Information Theory Newsletter

Calculator Magic Solutions

IEEE Information TheoryIEEE Information Society Newsletter Theory Society Newsletter June 2017 June 2017

June 2007 June 2007 IEEE Information TheoryIEEE Information Society Newsletter Theory Society Newsletter NLNL MP MP

3923 23 34 34 Reprinted from Vol.Reprinted 57, No. 3, from September Vol. 57, 2007 No. issue 3, September of Information 2007 issue Theory of InformationNewsletter Theory Newsletter

GOLOMB’S PUZZLEGOLOMB’S COLUMN™ PUZZLE COLUMN™ CCONNECTONNECTCONNECT THETHE DOTS DO THETS DOTS Solomon W. GolombSolomon W. Golomb

1. The four 6-segment1. The circuits four 6-segment on the 4 circuits4 array onof dotsthe 4 are:4 array of dots are: × ×

2. Here are three inequivalent2. Here are three 8-segment inequivalent circuits 8-segment on the 5 circuits5 array onof dots.the 5 5 array of dots. × ×

3. Here are two 10-segment3. Here are circuits two 10-segment on the 6 circuits6 array onof dots.the 6 6 array of dots. × ×

The solution on theThe right solution stays withinon the rightthe convex stays withinhull of thethe convex6 6 array hull of dots.the 6 6 array of dots. × × 4. This 14-move queen's4. This tour14-move of the queen's chessboard tour ofwas the first chessboard published was by firstSam published Loyd. It is by included Sam Loyd. in Sam It is Loyd included and His in SamChess Loyd and His Chess Problems, compiledProblems by Alain, compiled C. White, by published Alain C. 1913White, by published Whitehead 1913 and by Miller; Whitehead Dover andreprint, Miller; 1962. Dover The reprint,queen's 1962.circuit The queen's circuit is: a1-h1-a8-a2-h2-b8-b4-f8-c8-g4-g8-b3-h3-h8-a1.is: a1-h1-a8-a2-h2-b8-b4-f8-c8-g4-g8-b3-h3-h8-a1.

5. Here is the unique5. Here 5-segment is the uniquecircuit on5-segment the 3 4circuitarray onof dots.the 3 4 array of dots. × ×

Reference. The definitiveReference. article The on definitive this subject article is byon S.W.this subjectGolomb is and by S.W.J.L. Selfridge, Golomb and“Unicursal J.L. Selfridge, Polygonal “Unicursal Paths and Polygonal Other Paths and Other Graphs on Point Lattices,”Graphs on Pi PointMu Epsilon Lattices,” Journal Pi Mu, Fall, Epsilon 1970. Journal, Fall, 1970.

IEEE Information TheoryIEEE Information Society Newsletter Theory Society Newsletter September 2007 September 2007 June 2017 June 2017 IIEEE Information ITheoryEEE Information Society Newsletter Theory Society Newsletter itNL1207.qxd 11/19/07 11:32 AM Page 56

19. Marc Fossorier presented the Awards Committee report. The of BoG candidates. Awards Committee recommends that the Best Paper Award be given to the following paper: H. Weingarten, Yossef Steinberg, The Board unanimously accepts the recommendation of the S. S. Shamai: The Capacity Region of the Gaussian Multiple- Nominations Committee. Input Multiple-Output Broadcast Channel. IEEE Transactions on Information Theory, vol. 52, No. 9, pp. 3936-3964, The decision to extend the list is left at the discretion of the September 2006. Nominations Committee.

The Board unanimously accepts the recommendation of the 22. David Neuhoff opened the discussion for nominations of new Awards Committee. Marc Fossorier raised the concern that the officers. number of nominations for the Joint Com-Soc/IT Paper Award is low. The Board discussed possible measures to increase the The Board unanimously approved the nominations of Frank number of nominations, including seeking stronger involve- Kschischang and Robert Calderbank for 2nd Vice-President. ment by the Associated Editors of the Transactions. 23. There was no new business. 20. Andrea Goldsmith explained the procedure to select the candidates for the Chapter of the Year Award. The winner this 24. The next Board meeting will be held at Allerton in September. year is the Seoul Chapter. 40 25. The meeting was adjourned at 18:58. 21. Steve McLaughlin reported on the process of forming the list Reprinted from Vol. 57, No. 4, December 2007 issue of Information Theory Newsletter

GOLOMB’S PUZZLE COLUMN™ EASYEASY PROBABILITIESPROBABILITIES SOLUTIONS SOLUTIONS Solomon W. Golomb

10 1. There are ( 5 ) = 252 ways to select 5 of the 10 decimal dig- win the middle match, and B is its. When these are arranged in ascending order as a < b < c weaker than A. Also, the sequence < d < e, the only way a + b + c > d + e can occur is when the BAB gives you only one chance to five selected numbers are {5, 6, 7, 8, 9}, so that 5+6+7 > 8+9. defeat strong player A, while ABA Thus, the probability of this occurring “at random” is only gives you two chances. 1 252 = 0.00396825 . . .. 5. (a) If at least one of the two children is a girl, there are three 2. To maximize the probability that a green marble will be equally likely cases (in “birth order”): BG, GB, GG, and the 1 selected, place a single green marble in one jar, with the probability that both are girls is 3 . remaining n - 1 green marbles and all n red marbles in the second jar. If the contestant chooses the first jar (with proba- (b) If the older child is a girl, the younger child is equally like- 1 bility 2 ), the selected marble will be green. If the second jar ly to be a boy or a girl; so in this case the probability that both 1 is chosen, the probability of a green marble being drawn is are girls is 2 . (n 1) (2n− 1) . Thus the probability of a green marble being selected, − 1 n 1 2 for this arrangement, is 2 (1 2n− 1 ) , which is 3 if n has the 6. The marble originally in the jar is either black (B1) or white + − 3 minimum value n = 2, but tends to the limiting value of 4 as (W1). A new white marble (W2) is inserted. The jar now con- n increases. tains either B1W2 or W1W2 (equally likely). When a marble is removed and observed to be white, there are now three 3. If North-South have all the hearts then East-West have none. equally likely cases: i) W2 out, B1 still in, ii) W2 out, W1 still Thus the probability that you and your partner (together) in ; or iii) W1 out, W2 still in. (Observing the withdrawn mar- have all the hearts is the same as the probability that the two ble to be white eliminated the case iv) B1 out, W2 still in.) 2 of you have none. Thus the probability that the unseen marble is white is 3 .

4. If A is a stronger player than B, your probability of winning Reference. Problem 1 is my own, and previously unpublished. two consecutive matches is better in the sequence ABA than Some version of each of the remaining problems (all oldies) in the sequence BAB. You can calculate this exactly if P1 is the can be found in Martin Gardner’s Colossal Book of Short probability that A will beat you and P2 is the probability that Puzzles and Problems, W. Norton & Co. 2006, which identifies B will beat you, with P1 > P2. Intuitively, the result follows Problem 6 as coming from Lewis Carroll’s book Pillow from: to win two matches (of the three) in a row, you must Problems.

IEEE Information Theory Society Newsletter December 2007

IEEE Information Theory Society Newsletter June 2017 41 In Memoriam: Mary Elizabeth (Betty) Moore Shannon Reprinted from the Boston Globe (with permission of the Shannon family).

Mary Elizabeth (Betty) Moore Shannon, 95, for- Maze-Solving Mouse, a pioneering experiment merly of Winchester, Massachusetts, died May in artificial intelligence, and during a memo- 1, 2017 at her home at Brookhaven in Lexington, rable trip to Las Vegas, helped test a device de- Massachusetts. She was born in New York City signed to beat the house at roulette, considered to Vilma Ujlaky Moore and James E. Moore. by many to be the first wearable computer.

Betty excelled academically in high school, Betty left Bell in 1951 to raise a family. She be- winning a full scholarship to New Jersey Col- came an avid weaver, an interest she pursued lege for Women (now Rutgers’ Douglass Col- for 40 years. She joined the Boston Weaver’s lege) where she graduated Phi Beta Kappa with Guild, served as Dean of the Guild from 1976- a degree in mathematics. Upon graduation, she 1978, and received the Guild’s Distinguished began working the next day at Bell Laboratories Achievement Award. She worked closely with in Manhattan as a “computer,” one of a group the Handweavers Guild of America for many of women whose job was to do the mathemati- years and received an honorary Life Member- cal calculations required by the engineers. She ship in 1996. She was a member of the Cross likened it to a secretarial pool for math majors. Country Weavers and the Wednesday Weav- She was promoted to Technical Assistant at ers. In the ‘70s, Betty was one of the first explor- Bell, and worked with John Pierce, inventor of ers of computerized hand weaving, though she the communications satellite, collaborating on found, in the end, that she preferred the less several projects including a Bell Labs Technical technological approach. Memorandum entitled “Composing Music by a Stochastic Process.” In her later years she developed an interest in genealogy and never lost her love for all things mathematical. At Bell Labs, Betty met Dr. Claude Shannon, the creator of Informa- tion Theory. They married in 1949 and were devoted to each other Betty is survived by her son Andrew, her daughter Peggy, Peggy’s until Claude’s death in 2001. She and Claude shared a playful sense partner Nina, and their daughters Nadja and Eva. Services will of humor, and Betty assisted Claude in building some of his most be private. Donations in her memory can be made to Associate famous inventions. She did much of the wiring of Theseus, the Alumnae of Douglass College.

Nominations Sought for 2018 and 2019 Leaders Volunteers needed to serve as corporate officers and committee chairs and members By Howard E. Michel, Chair 2017 IEEE Nominations and Appointments Committee

IEEE is governed by volunteer members and depends on them for many things, including editing IEEE publications, organizing conferences, coordinating regional and local activities, authoring and authorizing publication of standards, leading educational activities, and identifying individuals for IEEE recognitions and awards.

The Nominations and Appointments (N&A) Committee is responsible for developing recommendations to be sent to the Board of Directors and the IEEE Assembly on staffing many volunteer positions including candidates for president-elect and corporate officers. Accordingly, the N&A Committee is seeking nominees for the leadership positions. How to Nominate

For information about the positions, including qualifications and estimates of the time required by each position during the term of office, check the Guidelines for Nominating Candidates at www.ieee.org. To nominate a person for a position, complete the online IEEE Nominations & Appointments Committee Nomination Form.

June 2017 IEEE Information Theory Society Newsletter 42 Recent Publications

IEEE Transactions on Information Theory

MARCH 2017 VOLUME 63 NUMBER 3 IETTAW (ISSN 0018-9448) Table of content for volumes 63(3), 63(4), 63(5).

Vol. 63(3): Mar. 2017.

SHANNON THEORY T. J. Lim and M. Franceschetti Information Without Rolling Dice 1349 D. Shaviv, A. Özgür, and H. H. Permuter Capacity of Remotely Powered Communication 1364 O. Sabag, H. H. Permuter, and H. D. Pﬁster ASingle-LetterUpperBoundontheFeedbackCapacityofUniﬁlar 1392 Finite-State Channels U. Pereg and I. Tal Channel Upgradation for Non-Binary Input Alphabets and MACs 1410

CODING THEORY AND TECHNIQUES F. L . P iñero The Structure of Dual Schubert Union Codes 1425 L. Jin and C. Xing New MDS Self-Dual Codes From Generalized Reed–Solomon Codes 1434 F. Jardel and J. J. Boutros Edge Coloring and Stopping Sets Analysis in Product Codes With 1439 MDS Components G. Garrammone, D. Declercq, and M. P. C. Fossorier Weight Distributions of Non-Binary Multi-Edge Type LDPC Code 1463 Ensembles: Analysis and Efﬁcient Evaluation C. Rush, A. Greig, and R. Venkataramanan Capacity-Achieving Sparse Superposition Codes via Approximate Message 1476 Passing Decoding I. Tal On the Construction of Polar Codes for Channels With Moderate Input 1501 Alphabet Sizes K. Huang, U. Parampalli, and M. Xian On Secrecy Capacity of Minimum Storage Regenerating Codes 1510

SPARSE RECOVERY, SIGNAL PROCESSING, LEARNING, ESTIMATION T. Wadayama Nonadaptive Group Testing Based on Sparse Pooling Graphs 1525 R. M. Castro and E. Tánczos Adaptive Compressed Sensing for Support Recovery of Structured 1535 Sparse Sets C. Cartis and A. Thompson Quantitative Recovery Conditions for Tree-Based Compressed Sensing 1555 A. Montanari, D. Reichman, and O. Zeitouni On the Limitation of Spectral Methods:FromtheGaussianHiddenClique 1572 Problem to Rank One Perturbations of Gaussian Tensors A. Xu and M. Raginsky Information-Theoretic Lower Bounds on Bayes Risk in Decentralized 1580 Estimation M. Genzel High-Dimensional Estimation of Structured Signals From Non-Linear 1601 Observations With General Convex Loss Functions M. Kohler and A. Krzy˙zak Nonparametric Regression Based on Hierarchical Interaction Models 1620

COMMUNICATION NETWORKS N. Xie and S. Weber Delay on Broadcast Erasure Channels Under Random Linear Combinations 1631 S.-W. Jeon, S.-N. Hong, M. Ji, G. Caire, and Wireless Multihop Device-to-Device Caching Networks 1662 A. F. Molisch F. Yavuz , J. Zhao, O. Yagan, ˇ and V. Gligor k-Connectivity in Random K-Out Graphs Intersecting Erd˝os-Rényi Graphs 1677

GAUSSIAN NETWORKS J. M. Romero-Jerez and F. J. Lopez-Martinez ANewFrameworkforthePerformanceAnalysisofWirelessCommunica- 1693 tions Under Hoyt (Nakagami-q)Fading F. Parvaresh and H. Soltanizadeh Diversity-Multiplexing Trade-Off of Half-Duplex Single Relay Networks 1703 D. Castanheira, A. Silva, and A. Gameiro Retrospective Interference Alignment: Degrees of Freedom Scaling With 1721 Distributed Transmitters Z. Zheng, L. Wei, R. Speicher, R. R. Müller, Asymptotic Analysis of Rayleigh Product Channels: A Free Probability 1731 J. Hämäläinen, and J. Corander Approach L. V. Truong, S. L. Fong, and V. Y. F. Tan On Gaussian Channels With Feedback Under Expected Power Constraints 1746 and With Non-Vanishing Error Probabilities

SOURCE CODING L. Zhou, V. Y. F. Tan, and M. Motani Discrete Lossy Gray–Wyner Revisited: Second-Order Asymptotics, Large and 1766 Moderate Deviations

QUANTUM INFORMATION THEORY M. M. Wilde, M. Tomamichel, and M. Berta Converse Bounds for Private Communication Over Quantum Channels 1792 M. Fukuda and G. Gour Additive Bounds of Minimum Output Entropies for Unital Channels and an 1818 IEEE Information Theory Society Newsletter Exact Qubit Formula June 2017 W. Kumagai and M. Hayashi Second-Order Asymptotics of Conversions of Distributions and Entangled 1829 States Based on Rayleigh-Normal Probability Distributions

SECURE COMMUNICATION R. A. Chou, B. N. Vellambi, M. R. Bloch, and Coding Schemes for Achieving Strong Secrecy at Negligible Cost 1858 J. Kliewer N. Shlezinger, D. Zahavi, Y. Murin, and R. Dabora The Secrecy Capacity of Gaussian MIMO Channels With Finite Memory 1874 P. Mukherjee, J. Xie, and S. Ulukus Secure Degrees of Freedom of One-Hop Wireless Networks With No 1898 Eavesdropper CSIT M. Mirmohseni and P. Papadimitratos Secrecy Capacity Scaling in Large Cooperative Wireless Networks 1923 COMMUNICATION NETWORKS N. Xie and S. Weber Delay on Broadcast Erasure Channels Under Random Linear Combinations 1631 S.-W. Jeon, S.-N. Hong, M. Ji, G. Caire, and Wireless Multihop Device-to-Device Caching Networks 1662 A. F. Molisch F. Yavuz , J. Zhao, O. Yagan, ˇ and V. Gligor k-Connectivity in Random K-Out Graphs Intersecting Erd˝os-Rényi Graphs 1677

SOURCE CODING L. Zhou, V. Y. F. Tan, and M. Motani Discrete Lossy Gray–Wyner Revisited: Second-Order Asymptotics, Large and 1766 43 Moderate Deviations

QUANTUM INFORMATION THEORY M. M. Wilde, M. Tomamichel, and M. Berta Converse Bounds for Private Communication Over Quantum Channels 1792 M. Fukuda and G. Gour Additive Bounds of Minimum Output Entropies for Unital Channels and an 1818 Exact Qubit Formula W. Kumagai and M. Hayashi Second-Order Asymptotics of Conversions of Distributions and Entangled 1829 States Based on Rayleigh-Normal Probability Distributions

APRIL 2017 VOLUME 63 NUMBER 4 IETTAW (ISSN 0018-9448)

Vol. 63(4): Apr. 2017.

CODING THEORY AND TECHNIQUES T. Johnsen and H. Verdure Generalized Hamming WeightsforAlmostAffineCodes 1941 R. Saptharishi, A. Shpilka, and B. L. Volk Efficiently Decoding Reed–Muller Codes From Random Errors 1954 V. Gur us wami and C. Wang Deletion Codes in the High-Noise and High-Rate Regimes 1961 C. Schoeny, A. Wachter-Zeh, R. Gabrys, Codes Correcting a Burst of Deletions or Insertions 1971 and E. Yaakobi M. El-Khamy, H. Mahdavifar, G. Feygin, Relaxed Polar Codes 1986 J. Lee, and I. Kang M. Ye and A. Barg Explicit Constructions of High-Rate MDS Array Codes With Optimal Repair 2001 Bandwidth N. Raviv, N. Silberstein, and T. Etzion Constructions of High-Rate MinimumStorageRegeneratingCodesOver 2015 Small Fields H. Khodaiemehr and D. Kiani Construction and Encoding of QC-LDPC Codes Using Group Rings 2039 Z. Wang, H. M. Kiah, Y. Cassuto, and J. Bruck Switch Codes: Codes for Fully Parallel Reconstruction 2061 D. T. H. Kao, M. A. Maddah-Ali, Blind Index Coding 2076 and A. S. Avestimehr Y.-C. Huang Lattice Index Codes From Algebraic Number Fields 2098 S. Cai, M. Jahangoshahi, M. Bakshi, and S. Jaggi Efficient Algorithms for Noisy Group Testing 2113

SPARSE RECOVERY, SIGNAL PROCESSING, LEARNING, ESTIMATION M. Gavish and D. L. Donoho Optimal Shrinkage of Singular Values 2137 S. Said, L. Bombrun, Y. Berthoumieu, Riemannian Gaussian Distributions on the Space of Symmetric Positive 2153 and J. H. Manton Deﬁnite Matrices m G. Puy, M. E. Davies, and R. Gribonval Recipes for Stable Linear Embeddings From Hilbert Spaces to R 2171 A. Kumar On Bandlimited Field Estimation from Samples Recorded by a Location- 2188 Unaware Mobile Sensor C. Suh, V. Y. F. Tan, and R. Zhao Adversarial Top-K Ranking 2201 E. Chaumette, A. Renaux, and M. N. El Korso AClassofWeiss–WeinsteinBoundsandItsRelationshipWiththe 2226 Bobrovsky–Mayer-Wolf–Zakaï Bounds

SHANNON THEORY S. Hu and O. Shayevitz The ρ-Capacity of a Graph 2241 X. Wu, A. Özgür, and L.-L. Xie Improving on the Cut-Set Bound via Geometric Analysis of Typical Sets 2254 P. Moulin The Log-Volume of Optimal Codes for Memoryless Channels, 2278 Asymptotically Within a Few Nats A. Xu and M. Raginsky Information-Theoretic Lower Bounds for Distributed Function Computation 2314 A. Sharov and R. M. Roth On the Capacity of Generalized Ising Channels 2338 B. T. Ávila and R. M. Campello de Souza Meta-Fibonacci Codes:Efﬁcient Universal Coding of Natural Numbers 2357

GAUSSIAN CHANNELS H. Ghozlan and G. Kramer Models and Information Rates for Wiener Phase Noise Channels 2376 S. Yang and S. Shamai On the Multiplexing Gain of Discrete-Time MIMO Phase Noise Channels 2394 A. Ben-Yishai and O. Shayevitz Interactive Schemes for the AWGN Channel with Noisy Feedback 2409

June 2017 SEQUENCES IEEE Information Theory Society Newsletter F. Sala, R. Gabrys, C. Schoeny, and L. Dolecek Exact Reconstruction From Insertions in Synchronization Codes 2428 Y. Yang, Y. Zhang, and G. Ge New Lower Bounds for Secure Codes and Related Hash Families: 2446 AHypergraphTheoreticalApproach H. Hu, Q. Zhang, and S. Shao On the Dual of the Coulter-Matthews Bent Functions 2454

QUANTUM INFORMATION THEORY D. Bacon, S. T. Flammia, A. W. Harrow, and J. Shi Sparse Quantum Codes From Quantum Circuits 2464

COMPLEXITY AND CRYPTOGRAPHY M. Berta, O. Fawzi, and V. B. Scholz Quantum-Proof Randomness Extractors via Operator Space Theory 2480 W. Kumagai and M. Hayashi Random Number Conversion and LOCC Conversion via Restricted Storage 2504

SECURE COMMUNICATION N. Weinberger and N. Merhav ALargeDeviationsApproachtoSecureLossyCompression 2533 M. Mishra, B. K. Dey, V. M. Prabhakaran, Wiretapped Oblivious Transfer 2560 and S. N. Diggavi L. Yu, H. Li, and W. Li Source-Channel Secrecy for Shannon Cipher System 2596

COMMENTS AND CORRECTIONS N. Raviv and A. Wachter-Zeh ACorrectionto“SomeGabidulinCodesCannotbeListDecodedEfﬁciently 2623 at any Radius” N. Datta and F. Leditzky Corrections to “Second-Order Asymptotics for Source Coding, Dense Coding, 2625 and Pure-State Entanglement Conversions” A. Xu and M. Raginsky Information-Theoretic Lower Bounds for Distributed Function Computation 2314 A. Sharov and R. M. Roth On the Capacity of Generalized Ising Channels 2338 B. T. Ávila and R. M. Campello de Souza Meta-Fibonacci Codes:Efﬁcient Universal Coding of Natural Numbers 2357

GAUSSIAN CHANNELS H. Ghozlan and G. Kramer Models and Information Rates for Wiener Phase Noise Channels 2376 S. Yang and S. Shamai On the Multiplexing Gain of Discrete-Time MIMO Phase Noise Channels 2394 44 A. Ben-Yishai and O. Shayevitz Interactive Schemes for the AWGN Channel with Noisy Feedback 2409

SEQUENCES F. Sala, R. Gabrys, C. Schoeny, and L. Dolecek Exact Reconstruction From Insertions in Synchronization Codes 2428 Y. Yang, Y. Zhang, and G. Ge New Lower Bounds for Secure Codes and Related Hash Families: 2446 AHypergraphTheoreticalApproach H. Hu, Q. Zhang, and S. Shao On the Dual of the Coulter-Matthews Bent Functions 2454

QUANTUM INFORMATION THEORY D. Bacon, S. T. Flammia, A. W. Harrow, and J. Shi Sparse Quantum Codes From Quantum Circuits 2464

MAY 2017 VOLUME 63 NUMBER 5 IETTAW (ISSN 0018-9448)

Vol. 63(5): May. 2017.

SHANNON THEORY J. Liu, P. Cuff, and S. Verdú Eγ -Resolvability 2629 F. Haddadpour, M. H. Yassaee, S. Beigi, A. Gohari, Simulation of a Channel With Another Channel 2659 and M. R. Aref J. Zhu and M. Gastpar Gaussian Multiple Access via Compute-and-Forward 2678 N. Merhav Reliability of Universal Decoding Based on Vector-Quantized Codewords 2696 S. I. Bross The Discrete Memoryless Interference Channel With One-Sided Generalized 2710 Feedback and Secrecy S.-H. Lee, V. Y. F. Tan, and A. Khisti Exact Moderate Deviation Asymptotics in Streaming Data Transmission 2726 Y. Murin, Y. Kaspi, R. Dabora, and D. Gündüz Finite-Length Linear Schemes for Joint Source-Channel Coding Over 2737 Gaussian Broadcast Channels With Feedback

CODING THEORY AND TECHNIQUES I. Dumer and O. Kapralova Spherically Punctured Reed–Muller Codes 2773 C. G. Blake and F. R. Kschischang On the VLSI Energy Complexity of LDPC Decoder Circuits 2781 J. Gómez-Torrecillas, F. J. Lobillo, and G. Navarro Ideal Codes Over Separable Ring Extensions 2796 L. Wei, R.-A. Pitaval, J. Corander, and O. Tirkkonen From Random Matrix Theory to Coding Theory: Volume of a Metric Ball 2814 in Unitary Group K. Lee, N. B. Shah, L. Huang, and K. Ramchandran The MDS Queue: Analysing the Latency Performance of Erasure Codes 2822 L. Jin Construction of MDS Codes With Complementary Duals 2843 N. Adler and Y. Cassuto Burst-Erasure Correcting CodesWithOptimalAverageDelay 2848

SOURCE CODING X. Yang and A. R. Barron Minimax Compression and Large Alphabet Approximation Through 2866 Poissonization and Tilting A. V. Anisimov and I. O. Zavadskyi Variable-Length Preﬁx Codes With Multiple Delimiters 2885 L. Zhou, V. Y. F. Tan, and M. Motani Second-Order and Moderate Deviations Asymptotics for 2896 Successive Reﬁnement

SPARSE RECOVERY, SIGNAL PROCESSING, LEARNING, ESTIMATION X.-J. Liu, S.-T. Xia, and F.-W. Fu Reconstruction Guarantee Analysis of Basis Pursuit for Binary Measurement 2922 Matrices in Compressed Sensing IEEE Information Theory SocietyS. Jalali Newsletter and H. V. Poor Universal Compressed Sensing for Almost Lossless Recovery June2933 2017 F. Pourkamali-Anaraki and S. Becker Preconditioned Data Sparsiﬁcation for Big Data With Applications 2954 to PCA and K-Means MAY 2017 VOLUME 63 NUMBER 5 IETTAW (ISSN 0018-9448)

SPARSE RECOVERY, SIGNAL PROCESSING, LEARNING, ESTIMATION X.-J. Liu, S.-T. Xia, and F.-W. Fu Reconstruction Guarantee Analysis of Basis Pursuit for Binary Measurement 2922 Matrices in Compressed Sensing S. Jalali and H. V. Poor Universal Compressed Sensing for Almost Lossless Recovery 2933 F. Pourkamali-Anaraki and S. Becker Preconditioned Data Sparsiﬁcation for Big Data With Applications 2954 to PCA and K-Means Z. Zhu and M. B. Wakin On the Asymptotic Equivalence of Circulant and Toeplitz Matrices 2975 I. Waldspurger Phase Retrieval for Wavelet Transforms 2993 A. Ganesan, S. Jaggi, and V. Saligrama Learning Immune-Defectives Graph Through Group Tests 3010 P. Jain, A. Tewari, and I. S. Dhillon Partial Hard Thresholding 3029 B. K. Guépié, L. Fillatre, and I. Nikiforov Detecting a Suddenly Arriving Dynamic Proﬁle of Finite Duration 3039 R. Mazumder and P. Radchenko The Discrete Dantzig Selector: Estimating Sparse Linear Models via 3053 Mixed Integer Linear Optimization

COMMUNICATION NETWORKS D. Tian, J. Zhou, and Z. Sheng An Adaptive Fusion Strategy for Distributed Information Estimation 3076 Over Cooperative Multi-Agent Networks N. Naderializadeh, M. A. Maddah-Ali, Fundamental Limits of Cache-Aided Interference Management 3092 and A. S. Avestimehr J. Hachem, N. Karamchandani, and S. N. Diggavi Coded Caching for Multi-level Popularity and Access 3108 J. Zhang and P. Elia Fundamental Limits of Cache-Aided Wireless BC: Interplay 3142 of Coded-Caching and CSIT Feedback A. Papadopoulos and L. Georgiadis Broadcast Erasure Channel With Feedback and Message Side Information, 3161 and Related Index Coding Result N. Xie, J. M. Walsh, and S. Weber Properties of an Aloha-Like Stability Region 3181

GAUSSIAN NETWORKS Y. Liu and E. Erkip Completion Time in Two-User Channels: An Information-Theoretic 3209 Perspective L. Luzzi and R. Vehkalahti Almost Universal Codes Achieving Ergodic MIMO Capacity Within 3224 aConstantGap S. Haddad and O. Lévêque On the Broadcast Capacity Scaling of Large Wireless Networks at Low SNR 3242 Y. Wang and M. K. Varanasi Degrees of Freedom Region of the MIMO 2 2 Interference Network 3259 With General Message Sets ×

QUANTUM INFORMATION THEORY C. Portmann, C. Matt, U. Maurer, R. Renner, Causal Boxes: Quantum Information-Processing Systems Closed 3277 and B. Tackmann Under Composition M. Berta, H. Gharibyan, and M. Walter Entanglement-Assisted Capacities of Compound Quantum Channels 3306 N. A. Warsi and J. P. Coon Coding for Classical-Quantum Channels With Rate Limited Side Information 3322 at the Encoder: Information-Spectrum Approach

COMMENTS AND CORRECTIONS T. J. Lim and M. Franceschetti Corrections to “Information Without Rolling Dice” 3332

June 2017 IEEE Information Theory Society Newsletter 46

Problems of Information Transmission Volume 53, Issue 1, January 2017

Some “goodness” properties of LDA lattices S. Vatedka, N. Kashyap Pages 1–29

Bounds on the rate of separating codes I. V. Vorob’ev Pages 30–41

Optimal conflict-avoiding codes for 3, 4 and 5 active users T. Baicheva, S. Topalova Pages 42–50

Remark on balanced incomplete block designs, near-resolvable block designs, and q-ary constant-weight codes L. A. Bassalygo, V. A. Zinoviev Pages 51–54

Linear algorithm for minimal rearrangement of structures K. Yu. Gorbunov, V. A. Lyubetsky Pages 55–72

Model of a random geometric graph with attachment to the coverage area S. N. Khoroshenkikh, A. B. Dainiak Pages 73–83

Occurrence numbers for vectors in cycles of output sequences of binary combining generators O. V. Kamlovskii Pages 84–91

Number of curves in the generalized Edwards form with minimal even cofactor of the curve order A. V. Bessalov, O. V. Tsygankova Pages 92–101

IEEE Information Theory Society Newsletter June 2017 47

5th International Castle Meeting on Coding Theory and Applications PRELIMINARY CALL FOR PAPERS

This is the first announcement of the Fifth International Castle Meeting on Coding Theory and Applications (5ICMCTA), which will take place in Vihula Manor, Estonia, from Monday, August 28th, to Thursday, August 31st, 2017. Information about the 5ICMCTA can be found at http://www.castle-meeting-2017.ut.ee/ .

We solicit submissions of previously unpublished contributions related to coding theory, including but not limited to the following areas: Codes and combinatorial structures, Algebraic-geometric codes, Network coding, Codes for storage, Quantum codes, Convolutional codes, Codes on graphs, Iterative decoding, Coding applications to cryptography and security, Other applications of coding theory.

Organization:

General chair: Vitaly Skachek

Scientific Committee co-chairs: Ángela Barbero and Øyvind Ytrehus

Publicity: Yauhen Yakimenka

Scientific Committee

Alexander Barg • Irina Bocharova • Eimear Byrne • Joan-Josep Climent • Gerard Cohen • Olav Geil • Marcus Greferath • Tor Helleseth • Tom Høholdt • Camilla Hollanti • Kees S. Immink • Frank Kschischang • Boris Kudryashov • San Ling • Daniel Lucani • Gary McGuire • Sihem Mesnager • Muriel Médard • Diego Napp • Frederique Oggier • Patric Östergard • Raquel Pinto • Paula Rocha • Joachim Rosenthal • Eirik Rosnes • Moshe Schwartz • Vladimir Sidorenko • Patrick Sole • Leo Storme • Rüdiger Urbanke • Pascal Vontobel • Dejan Vukobratovic • Jos Weber • Gilles Zémor

Important dates: Paper submission: May 1, 2017 Notification of decision: June 12, 2017 Final version paper submission: July 3, 2017

June 2017 IEEE Information Theory Society Newsletter 48

The 10th International Workshop on Coding and Cryptography WCC 2017 Saint-Petersburg, Russia, September 18–22, 2017 http://wcc2017.suai.ru/

ANNOUNCEMENT AND CALL FOR PAPERS Organizing committee: Important dates (for extended abstracts):

• Pierre Loidreau (co-chair, DGA, U. Rennes 1, • Submission by April 6, 2017

France) • Notiﬁcation by May 24, 2017

• Evgeny Krouk (co-chair, SUAI, Russia) • Final version by June 26, 2017 Veronika Prokhorova (SUAI, Russia) • Program committee: Evgeny Bakin (SUAI, Russia) • Daniel Augot (co-chair, INRIA, France) Local organization: • • Delphine Boucher (U. Rennes 1, France) • Oksana Novikova • Lilya Budaghyan (U. Bergen, Norway) • Maria Shelest • Eimear Byrne (UC Dublin, Ireland)

Invited Speakers: • Pascale Charpin (INRIA, France)

• Alexander Barg (U. Maryland, USA) • Alain Couvreur (INRIA, France)

• Claude Carlet (U. Paris 8 and Paris 13, France) • Ilia Dumer (UC Riverside, USA)

• Camilla Hollanti (Aalto U., Finland) • Tuvi Etzion (CSD Technion, Israel)

• Grigory Kabatiansky (Skoltech and IITP RAS, • Markus Grassl (MPL Erlangen, Germany)

Moscow, Russia) • Tor Helleseth (U. Bergen, Norway)

• Patric Östergård (Aalto U., Finland) • Thomas Honold (U. Zhejiang, China) Thomas Johansson (Lund U., Sweden) This is the tenth in the series of biannual workshops • Gohar Khyureghan (U. Magdeburg, Germany) Coding and Cryptography. It is organized by IN- • Ivan Landjev (NBU Sofia, Bulgaria) RIA, SUAI and Skoltech and will be held in the main • Subhamoy Maitra (ISI Kolkata, India) building of SUAI (http://suai.ru/), Saint-Petersburg, • Ryutaroh Matsumoto (Tokyo Tech., Japan) Russia. • Gary McGuire (UC Dublin, Ireland) Conference Themes. Our aim is to bring together • • Sihem Mesnager (U. Paris 8 and Paris 13, France) researchers in all aspects of coding theory, cryptog- Marine Minier (INSA-Lyon, France) raphy and related areas, theoretical or applied. • • Kaisa Nyberg (AU Helsinki, Finland) Topics include, but are not limited to: Ayoub Otmani (U. Rouen-Normandie, France) coding theory: error-correcting codes, decoding al- • • Ferruh Ozbudak (METU Ankara, Turkey) gorithms, related combinatorial problems; • • Kevin Phelps (AU Auburn, USA) • algorithmic aspects of cryptology: symmetric cryp- • Alexander Pott (U. Magdeburg, Germany) tology, public-key cryptography, cryptanalysis; Josep Rifa (UA Barcelona, Spain) discrete mathematics and algorithmic tools related • • Palash Sarkar (ISI Kolkata, India) to these two areas, such as: Boolean functions, se- • • Natalia Shekhunova (SUAI St.Petersburg, Russia) quences, finite fields, related algebraic systems. Vladimir Sidorenko (TU Munich, Germany) Submissions. Those wishing to contribute a talk • • Faina I. Solov’eva (co-chair, IM Sobolev, Russia) are invited to submit a 6-10 page extended ab- Jean-Pierre Tillich (INRIA, France) stract, before April 6, 2017 (23:59 Greenwich). • • Alev Topuzoğlu (Sabanci U. Istanbul, Turkey) The submission server is now open, informa- Peter Trifonov (PU St.Petersburg, Russia) tion on the submission process is available at • • Michail Tsfasman (CNRS and IITP RAS, Mar- http://wcc2017.suai.ru/submission.html. seille, France) Full papers. After the conference, authors of ac- • Serge Vladuts (Aix-Marseille U. and IITP RAS, cepted abstracts will be invited to submit a full pa- France) per for the proceedings to appear as a special issue • Arne Winterhof (Austrian Acad. of Sc., Linz) of the journal "Designs Codes and Cryptography". • Gilles Zémor (U. Bordeaux, France ) Contributions will be thoroughly refereed. • Victor Zinoviev (IITP RAS, Moscow, Russia)

• Victor Zyablov (IITP RAS, Moscow, Russia)

IEEE Information Theory Society Newsletter June 2017 49

55th Allerton Conference Call for Papers: Due July 10, 2017 Manuscripts can be submitted during June 16-July 10, 2017 with the submission deadline of July 10th being firm. Please follow the instructions at allerton.csl.illinois.edu.

CONFERENCE CO-CHAIRS: Naira Hovakimyan & Negar Kiyavash PAPERS PRESENTING ORIGINAL RESEARCH ARE INFORMATION FOR AUTHORS: Regular papers suitable for presentation in SOLICITED IN THE AREAS OF: twenty minutes are solicited. Regular papers will be published in full (subject to a maximum length of eight 8.5” x 11” pages, in two column format) in the • Biological information systems • Coding techniques & applications Conference Proceedings. Only papers that are actually presented at the conference • Coding theory and uploaded as final manuscripts can be included in the proceedings, which will • Data storage be available after the conference on IEEE Xplore. For reviewing purposes of papers, • Information theory a title and a five to ten page extended abstract, including references and sufficient • Multiuser detection & estimation detail to permit careful reviewing, are required. • Network information theory • Sensor networks in communications • Wireless communication systems IMPORTANT DATES - 2017 • Intrusion/anomaly detection & diagnosis JULY 10 — Submission Deadline • Network coding • Network games & algorithms AUGUST 7 — Acceptance Date Authors will be notified of acceptance via e-mail • Performance analysis by August 7, 2017, at which time they will also be sent detailed instructions for the preparation of their papers for the Conference Proceedings. • Pricing & congestion control • Reliability, security & trust AFTER AUGUST 7 — Registration Opens • Decentralized control systems OCTOBER 3-6 — Conference Dates • Robust & nonlinear control OCTOBER 3 — Opening Tutorial Lectures • Adaptive control & automation Coordinated Science Lab, University of Illinois at Urbana-Champaign • Robotics • Distributed & large-scale systems OCTOBER 4-6 — Conference Sessions (Plenary Lecture October 6) • Complex networked systems University of Illinois Allerton Park & Retreat Center: The Allerton House • Optimization is located twenty-six miles southwest of the Urbana-Champaign campus of the • Dynamic games University of Illinois in a wooded area on the Sangamon River. It is part of the • Machine learning & learning theory fifteen-hundred acre Robert Allerton Park, a complex of natural and man-made • Signal models & representations beauty designated as a National natural landmark. Allerton Park has twenty miles • Signal acquisition, coding, & retrieval of well-maintained trails and a living gallery of formal gardens, studded with • Detection & estimation sculptures collected from around the world. • Learning & inference OCTOBER 8 — Final Paper Deadline Final versions of papers that are presented • Statistical signal processing at the conference must be submitted electronically in order to appear in the Confer- • Sensor networks ence Proceedings and IEEE Xplore. • Data analytics All speaker information will be added when confirmed.

WEBSITE | allerton.csl.illinois.edu EMAIL | [email protected] UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN

June 2017 IEEE Information Theory Society Newsletter 50

2017 IEEE Information Theory Workshop Kaohsiung Exhibition Center, Kaohsiung, Taiwan / November 6 10, 2017 2017 http://www.itw2017.org KAOHSIUNG

General Co-Chairs The 2017 IEEE Information Theory Workshop will take place in Kaohsiung, Taiwan, from November 6 to 10, 2017. Po-Ning Chen (NCTU, Taiwan) Based at the southern tip of the island and facing the Taiwan Strait, Kaohsiung is the second largest city in Gerhard Kramer (TUM, Germany) Taiwan and is one of the major seaports in Asia. Love-River cruises, night markets, delicious seafood, and day Chih-Peng Li (NSYSU, Taiwan) tours out into nature or to historic Tainan are some of the many attractions awaiting you in Southern Taiwan.

Situated directly by the waterfront, the Kaohsiung Exhibition Center (KEC) serves as workshop venue. It is a Technical Program Co-Chairs Hsiao-feng (Francis) Lu (NCTU, Taiwan) brand-new and multi-functional facility, designed by an international, pro-environment team of architects and Stefan M. Moser (ETHZ, Switzerland) built in the shape reminding of a billowing sail. The workshop participants will have an unforgettable experience Chih-Chun Wang (Purdue Univ., USA) visiting and enjoying some of the most dazzling attractions in Kaohsiung.

Finances Call for Papers Chung-Hsuan Wang (NCTU, Taiwan) Interested authors are encouraged to submit previously unpublished contributions in all areas of information Jwo-Yuh Wu (NCTU, Taiwan) theory with special emphasis on the following :

Publicity Osvaldo Simeone (NJIT, USA) • Information Theory for Content Distribution Hsaun-Jung Su (NTU, Taiwan) − Distributed data storage Publications − Peer-to-peer network coded broadcasting Stefano Rini (NCTU, Taiwan) − Coded caching for wireless and wireline transmissions I-Hsiang Wang (NTU, Taiwan) − Delay-constrained communications • Information Theory and Biology Local Arrangement Celeste Lee (NSYSU, Taiwan) − Information theory and intercellular communications Fan-Shuo Tseng (NSYSU, Taiwan) − Information theory and neuroscience Chao-Kai Wen (NSYSU, Taiwan) − Information-theoretical analysis of biologically-inspired communication systems Webmaster Yao-Win (Peter) Hong (NTHU, Taiwan) • Information Theory and Quantum Communications − Quantum information International Advisory Committee − Quantum computation Toru Fujiwara (Osaka Univ., Japan) − Quantum cryptography Shuo-Yen (Robert) Li (CUHK, China) Alon Orlitsky (UCSD, USA) • Information Theory and Coding for Memories A. J. Han Vinck (UDE, Germany) − New coding techniques for non-volatile memory channels Muriel Medard (MIT, USA) − Coding and signal processing for dense memory Andrea Goldsmith (Stanford, USA) − Multi-dimensional coding for storage channels Raymond W. Yeung (CUHK, China)

Sponsors Paper Submission IEEE Information Theory Society Paper submission guidelines will be posted on the workshop’s website : http://www.itw2017.org

Co-Sponsors Poster National Chiao-Tung University The technical program will feature a poster session. Details about poster submissions will be announced on the National Sun Yat-sen University workshop’s website by late July, 2017.

Important Dates Paper submission deadline : May 7, 2017 Acceptance notiﬁcation : July 21, 2017 Call For Papers

IEEE Information Theory Society Newsletter June 2017 51

Call for Papers 2018 International Zurich Seminar n nran and Communication

February 21 – 23, 2018

The 2018 International Zurich Seminar on Information and Communication will be held at the Hotel Zürichberg in Zurich, Switzerland, from Wednesday, February 21, through Friday, February 23, 2018. High-quality original contri butions of both applied and theoretical nature are solicited in the areas of:

Wireless Communication Optical Communication Information Theory Fundamental Hardware Issues Coding Theory and its Applications Network Algorithms and Protocols Detection and Estimation Network Information Theory and Coding ata torae Cryptography and Data Security

Invited speakers will account for roughly half the talks. In order to afford the opportunity to learn from and communi- cate with leading experts in areas beyond one’s own specialty, no parallel sessions are anticipated. All papers should be presented with a wide audience in mind. aer i be reieed on te bai of a manurit not eeedin ae of uffiient detai to ermit rea onable evaluation. Authors of accepted papers will be asked to produce a manuscript not exceeding pages in A4 double column format that will be published in the Proceedings. Authors will be allowed twenty minutes for presentation. The deadline for submission is September 17, 2017.

Additional information will be posted at

http://www.izs.ethz.ch/

We look forward to seeing you at IZS.

Amos Lapidoth and Stefan M. Moser, Co-Chairs.

June 2017 IEEE Information Theory Society Newsletter 52 Conference Calendar

DATE CONFERENCE LOCATION WEB PAGE DUE DATE

June 6–9, 2017 2017 North-American School of Atlanta, Georgia, USA http://www.itsoc.org/ Passed Information Theory conferences/schools/ 2017-north-american- school-of-information-theory

June 11–14, 2017 15th Canadian Workshop Quebec City, Quebec, http://cwit.ca/2017/ Passed on Information Theory Canada

June 25–30, 2017 IEEE International Symposium Aachen, Germany http://www.isit2017.org Passed on Information Theory (ISIT)

July 3, 2017 Munich Workshop on Munich, Germany http://www.lnt.ei.tum.de/ Passed Coding and Applications en/events/munich-workshop- on-coding-and-applications- 2017-mwca2017

July 3–6, 2017 The 18th IEEE International Sapporo, Japan http://www.spawc2017 Passed Workshop on Signal Processing .org/public.asp?page Advances in Wireless Communications =home.html

July 10–14, 2017 Croucher Summer Course Chinese University http://cscit.ie.cuhk.edu.hk/ Passed in Information Theory of Hong Kong

August 28–31, 2017 Fifth International Castle Meeting on Vihula Manor, Estonia http://www.castle-meeting- Passed Coding Theory and Applications 2017.ut.ee/ (5ICMCTA)

September 11–22, 2017 Quasi-Cyclic and Related Ankara, Turkey http://www.isit June 11 Algebraic Codes 2017.org

September 18–22, 2017 The Tenth International Workshop Saint-Petersburg, Russia http://wcc2017.suai.ru/ Passed on Coding and Cryptography 2017

October 3–6, 2017 55th Annual Allerton Conference University of Illinois at http://allerton.csl.illinois.edu/ July 10, 2017 on Communication, Control, Urbana-Champaign and Computing

October 15–17, 2017 58th Annual IEEE Symposium Berkeley, California, http://focs17.simons.ber/ Passed on Foundations of Computer USA. keley.edu Science (FOCS 2017)

November 6–10, 2017 IEEE Information Theory Workshop Kaohsiung, Taiwan http://www.itw2017.org/ Passed

December 4–8, 2017 IEEE GLOBECOM Singapore http://globecom2017.ieee- Passed globecom.org/

February 21–23, 2018 2018 International Zurich Seminar Zurich, Switzerland http://www.izs.ethz.ch/ September on Information and Communication 17, 2017

Major COMSOC conferences: http://www.comsoc.org/confs/index.html

IEEE Information Theory Society Newsletter June 2017