CODING and MODULATION for BANDWIDTH COMPRESSION of TELEVISION and FACSIMILE SIGNALS. a Thesis Submitted for the Degree of Doctor

CODING AND MODULATION FOR BANDWIDTH COMPRESSION OF TELEVISION AND FACSIMILE SIGNALS.

A thesis submitted for the degree of Doctor of Philosophy by

PIOTR WLADYSLAW JOZta BYLANSKI

Imperial College, University of London.

1966. 2.

ABSTRACT

Two aspects relevant to the coding of picture signals with a

view to compression of signal bandwidth are considered. Firstly: the problem of combining the information associated with the run length representation of the picture signal, for transmission over a practical channel, is studied. A solution is proposed in terms of combined amplitude and phase modulation and an analysis is presented which demonstrates the feasibility of the technique.

Secondly, several aspects of the digram representation of the picture signal are studied. The second order redundancy

is experimentally determined using a digital technique. The results obtained lead to the design of several simple and efficient digram -based codes for black and white facsimile pictures, resulting in useful reduction of signal bandwidth. These codes are implemented and assessed experimentally. An adaptive encoding technique is proposed whereby errors produced by a non-uniquely decodable code may be minimised in a controlled way by the use of predictive feedback at the encoder. 3. CONTENTS.

Item Page

ABSTRACT 2

CONTENTS 3

LIST OF SYMBOLS AND ABBREVIATIONS 6

ACKNOWLEDGEMENTS 9

CHAPTER. 1. INTRODUCTION 10

141 The Trend of Modern Communications. 10 1.2 Visual Redundancy. 11 1.3 Signal Compression. 13 1.4 Television and Facsimile Communication. 16

1,5 The Organisation of the Material of this Work. 18 1.6 Summaries of Individual Chapters. 19

CHAPTER 2. THE PROBLEM OF CHANNEL UTILISATION. 21 2.1 Bandwidth Compression. 21 2.2 The Run Length Coding System. 24

2.3 The Channel Utilisation Problem. 29 2.4 On Efficiency of Communication, 32 2.5 Possible Modulation and Coding Systems. 36

2.6 Digram Encoding, Error Rates and Channel Distortion. 45 2.7 Combined Amplitude and Phase Modulation. 50

2.8 Conclusions. 52

CHAPTER 3. ANALYSIS OF COMBINED AMPLITUDE AND PHASE

MODULATION OF THE PIMEt SIGNAL. 56

3.1 Objectives. 56 Page 3.2 On the Spectra of Angle Modulated Processes. 58 3.3 The Method of Analysis due to N. Abramson. 60 3.4 Spectral Computations. 64 3.5 Effects of Band Limiting. 70 3,6 Precoding of the Position Signal. 73 3,7 Compatibility with Known Requirements. 79 3.8 Summary and Conclusions. 81

CHAPTER 4, MEASUREMENT OF DIGRAM STATISTICS OF PICTURE SIGNALS. 83

4.1 Statistical Redundancy: 83 4.2 Calculation of Redundancy from Limited Statistics, 87 4.3 The Experiment. 92 4.4 Apparatus. 99 4.5 Results. 105 4.6 Conclusions and Discussion. 112

CHAPTER 5, INVESTIGATIONS INTO DIGRAM CODES. 116

5.1 Encoding of Black and White Signals. 116 5.2 Coding Theory Implications. 124 5.3 Experiments with Restriction of the Digram Alphabet. 130 5.4 A Restricted Alphabet Digram Code. 143

CHAPTER 6•. ADAPTIVE ENCODING. 145 6.1 A Specific Example. 145 6.2 An Adaptive Encoding Technique. 147 Page

6.3 Transmission Considerations. 151 6.4 A Proposed Adaptive Code. 159

SUMMARY OF CONCLUSIONS 3.65

REFERENCES 3.68

APPENDICES 172

Error Probability of Quantised Signal in Gaussian Noise. 172

The Decodability of Combined Amplitude and Phase Modulation. 171+ Autocorrelation Fianction of Combined Amplitude and Phase Modulation, 176

Table of Spectral Functions of Sec. 3.4. 180 Circuit Details of Digram Encoder. 181 6. LIST OF SYMBOLS AND ABBREVIATIONS

AN Amplitude modulation. B - Message bandwidth. Sanders' efficiency index. C - Channel capacity, bits or bits/sec. cps cycles per second. db decibels, d,c. direct current,

8 Quantal amplitude step. 8(x) Impulse function, 80 Quantal phase step. aT Sampling interval. D(t) = R(t) R(o)

G(N) Gaussian density function of variance N. Kc Kilocycles per second, L - Average code word length.

M - Number of message states or signal levels.. Mc Megacycles per second. N - Mean noise energy. nsec Nanosecond, 10-9 sec. P Mean signal energy. P Peak signal energy. pps Pulses per second, p(e) Probability of error.

PM Phase modulation.

PCM Pulse code modulation. P-signal Position signal. R(t) Autocorrelation function. r(t) Normalised autocorrelation function. r Data compression factor. Information rate bits/sec. R, First order redundancy, bits. R 2 Second order redundancy, bits. S/N Signal to noise ratio, general term. s(t) Channel signal. SSB Single sideband. a r.m.s. noise.

)( Characteristic function. VSB Vestigial sideband. Vsignal - Compressed video signal. W - Channel bandwidth. W(f) Power density spectrum. w(f) Normalised power density spectrum. Denotes convolution.

Notation used for ensembles

[X) - the set of symbols tx,,x2 xk..xm] P(X) the set of simple (first order) probabilities associated with the set X.

P(X/Y) - the set of conditional probabilities associated with the sets X and Y. H(X) the entropy measure associated with the set X; arbitrary units.

Hr(X) entropy expressed in units to the base r. H(XN) entropy associated with the N-th extension of X. E(X) expectation of X.

Summation convention: i=M :Ef(X) - f(xi) X i=1 and

i=M f()(4Y f(xi XY i=1

Boolean Logic Notation

Denoting the binary input symbols by a and b the notation is as follows: a + b = a OR b , signifying Boolean addition, a.b a AND b, signifying Boolean multiplication. a = NOT a , signifying Boolean negation.

In block diagrams logic gates are shown as triangles, the OR and AND functions being denoted by a plus sign and a dot respectively, in the centre of the triangle. A slanting line inside the triangle adjacent to any input denotes negation of that input. 9.

ACKNOWLEDGEMENTS

The author wishes to express his gratitude to his supervisor, Professor E. C. Cherry, for his guidance and encouragement and for the tedious task of ensuring funds for the research. Thanks are

due to the following bodies for providing personal financial support: the Department of Scientific and Industrial Research in particular, for providing a research scholarship for most of the duration of the work; the Institution of Electrical Engineers for granting an Oliver Lodge scholarship,and the Racal Co. Ltd., for

a bursary during the final difficult period. Help from the National Research Development Council in financing the project is

also appreciated. I am deeply indebted to Mr. R. E. C. White for undertaking the laborious task of reading, correcting and criticising the manuscript. Thanks are also due to Mr. A. M. Maciejowski for reading part of the manuscript. The cooperation of my colleagues is also appreciated; much useful information was acquired in informal discussions, especially with Messrs. D. J. Goodman, L. S. Moye, D. E. Pearson, A. H. Robinson, R. Thomas, B. J. Vieri, R. E. C.

White and A. A. White. Thanks are also due to Mrs. J. Dean for truly expert despatch of the manuscript and to Miss E. Krzysztoforska for much time devoted to this work. Finally I would like to thank my mother for constant encouragement.

10.

CHAPTER 1 INTRODUCTION

1.1 THE TREND OF MODERN COMMUNICATIONS

The development in recent years of the theory of communication based on statistical concepts has contributed to a changing approach in the treatment of communication systems. The pioneering work of

Shannon (42) on information transfer, and Wiener (48) on the optimum filtering of signals, have enabled a new overall understanding of existing systems and provided basic theoretical criteria for the synthesis of new ones. Hitherto the communications engineer has been absorbed in the pursuit of systems which are at the same time both efficient and simple. The statistical theory of communication has brought to light the basic fact that, in general, increased efficiency can only be bought at the price of increased complexity.

This fact emerges directly from the coding theorems of Information

Theory and is all the harder since the pursuit of efficiency hass to a great extent, been urged by the new appreciation of the basic inefficiency of some existing systems, made possible only by the tools provided by the very same theory.

The appreciation of the fundamental dicta of communication theory has led to significant changes in attitude to communication problems.

No longer is there an ingrained fear of growing complexity, a fear 11. previously arising from an unknown but suspected law of diminishing returns; the situation is now one where the returns are fully under- stood in the light of Information Theory. Thus, bold and ingenious systems have evolved; significant examples are pulse code modulation, digital transmission systems, colour television, pseudo-noise radar and many other systems of a more specialised nature.

1.2 VISUAL REDUNDANCY

The ability to define and measure information and the complementary quantity known as redundancy, have prompted the search for methods of optimising communication. This may be achieved by the exploitation of the natural redundancy which exists in many communication systems, especially those which have a human destination. In such cases the information transferred is either visual or auditory and a great deal of redundancy is involved. Present day conventional monochrome television systems use a channel capable 6 of transmitting about 50.10 bits of information per second. Measure.. ments on human subjects (Pierce and Karlin, 31) indicate a human information transfer capacity of only about 50 bits per second. The discrepancy or mismatch involved is enormous, and a great deal of redundant information is thus transmitted over a costly commodity - the communication channel. Efforts to decrease this visual redundancy by more efficient encoding seem, therefore, to be well 12.

justified.

Visual redundancy may be divided into two classes; perceptual

and statistical redundancy. The process of communicating a picture

to a human viewer is exceedingly complex and insufficiently under-

stood. Fig, 1.1(a) shows a basic model of a visual communication system. A visual real-life scene, U, is represented by a visual

form, X, which is amenable to transmission over the channel, In

conventional television X is the video signal produced by the normal

television scanning process. The acceptability of any given visual

communication system hinges on whether the visual form V, as displayed at the receiver after decoding from the transmitted video signal, is a sufficiently close approximation to one of the viewerts store of memorised visual impressions. This is so in the case of conventional monochrome television where the scanned nature of the picture is found to be quite acceptable to the viewer; similarly, in colour television, the viewer is found to be quite tolerant of the synthesis of a picture from two colour signals and a monochrome signal, as being a sufficient approximation to his impressions of the world in colour. Hence, whenever communication involves a human destination, the question of approximation within some subjective fidelity criterion is quite basic (Shannon, 40). The feasibility of such synthetic approximations in visual communication may be conveniently explained by the concept of perceptual redundancy. Thus, perceptual redundancy enables the representation of viii images by 13. means of various signal forms whilst satisfying the viewer's fidelity criteria. In particular, forms which are more efficient than the conventions] television signal are possible.

Statistical redundancy is concerned strictly with the signal used to convey the information and may be calculated directly from its statistical properties. It is therefore a less conceptual and more concrete quantity than perceptual redundancy. A basic distinction between the two forms is that the latter implies an irreversible operation of approximation allowed by some undefined inductive processes of the brain; statistical redundancy may, on the other hand, be directly exploited by statistical encoding techniques which are completely reversible.

1.3 SIGNAL COMPRESSION

Irrespective of the type involved, the sole purpose of exploiting redundancy is the reduction of channel capacity required to transmit the visual information. The capacity, C, of a continuous channel perturbed by additive Gaussian noise is given by Shannon's celebrated formula (Shannon, 42):

C = T.W.log(l ...(1.1) where, T time of duration of the message

W signal bandwidth (beyond which zero signal 14.

energy may be assumed).

P - mean signal energy

N - mean noise energy.

Alternatively, the capacity under unspecified conditions may be said

to be some function of the signal parameters T, W and P/N. Referring

to Fig. 1.1(a) the visual image, U, is processed in a conventional

television camera and scanning system producing a video signal X

which is normally transferred directly over a channel requiring some

standard capacity, C. This, in standard practice typically involves a bandwidth W of 3 Mc and a peak signal to r.m.s. noise ratio of 40 db. In order to reduce the required channel capacity, the signal

X must be processed in such a way as to reduce one or more of the

signal parameters T, W or P/N, resulting in a new signal X' for

which a capacity C' less than C is required. Alternatively, the signal X' satisfying these conditions may be produced directly by

some process not involving conventional scanning. In either case if the process by which X' is obtained is irreversible, then the signal Y at the output of the receiver decoder will not be identical to X even in the case of a noiseless channel, and the acceptability of the system depends on whether the perceptual fidelity criteria of the viewer are satisfied. Such a system depends on percept-on, redundancy. If the process of transformation from X to X' is one using statistical encoding allowed by the presence of statistical redundancy, then the system in the noiseless case is completely 15.

r~ .1CE CV-1 AN NE 1- ENct:).DE-R iLAL. X SOURCE

(c0 A Visual Communication System.

Sok, ES GE 44, Gt.4 AL. COMPRESSION -13.-- CHANNEL- Deco DER -1.`4CODER S-1"STEM .)0,54AL. X SOuRCE. VISUAL: isp46' (b) A Visual Signal Compression System.

FIG. 1.1.

reversible. The process of transforming the signal X into a more efficient signal Xf is carried out in what may be called a "Signal Compression System", (see Fig. 1.1(b) ). In the work that follows, such systems will be equivalently referred to as "information" or

"data compression" systems or simply, though rather loosely, as "Compression Systems". It is clear that signal compression in the broad sense, aimed at reduction of necessary channel capacity, may be attained by different menns depending on which signal parameter is reduced. Thus, 16. compression in the time of duration of a message may be achieved by any of the known statistical encoding techniques such as the Shannon-

Fano or the Huffman procedures (Reza, 34). Other schemes such as the systems of Kretzmer (23) and Roberts (36) achieve compression by exploiting specific forms of perceptual redundancy enabling a signal representation with fewer quantisation levels; this effectively reduces the signal power, P, required for a given signal to noise ratio.

The most desirable form of signal compression in practice is one which achieves compression by reduction of signal bandwidth; it is, at the same time, the most difficult to attain. The present study will be primarily concerned with several aspects of video signal bandwidth compression,

1.4 TELEVISION AND FACSIMILE COMMUNICATION

Communication of both television and facsimile material will be considered; certain basic differences exist between the two forms and it is felt that these should be outlined. The central difference lies in the type of visual information with which each system is required to cope. Television transmission involves predominantly real-life visual material which is rendered as a half- tone scanned picture. Facsimile material is, on the other hand, of a more synthetic nature. Here the source is already a two 17. dimensional reproduction, such as a drawing, a page of script, a pattern or a photograph. Most facsimile systems are intended for nominally black and white material, half-tone pictures usually requiring conversion of either the picture itself or the signal derived from it into digital form by various techniques. Basically, as in television, the human observer is still the destination, but the fidelity criteria involved are significantly different. The criterion underlying most facsimile applications is intelligibility rather than perfection. Perceptual redundancy is involved inasmuch as the copy must be acceptable to the viewer; however, it exists in a more limited form and may often be totally absent in the case of transmission of complex visual patterns, such as maps or radar displays, where the viewer may not be able to perform significant inferences based on his past experience. Thus, it may be considered that facsimile material is rather in the form of visual data and hence belongs partly to both the spheres of visual communication and data transmission. In contrast to television sources, small detailed areas of a picture such as numerals in a page of numbers may be very important. The viewer of the received copy thus performs an operation of pattern recognition rather than of visual image formation.

A further distinction is that the viewer at the receiver normally has unlimited time to study the copy, microscopically if he so wishes, and thus the question of the human rate of intake 18,

quoted earlier is not relevant. However, the unlimited inspection

time is offset by the higher cost of uncorrectable errors, since normally only one copy is generated at a single transmission unless a repeat is requested by the receiver.

1.5 THE ORGANISATION OF THE MATERIAL OF THIS WORK

The work that follows may be regarded as being in two parts

which, whilst self-sufficient have many aspects in common. However, it was thought unwise to attempt a strict division into two formal parts; it is hoped i;hat this and the following section will serve to clarify the presentation.

One part of the work, comprising chapters 2 and 3, is theoretical. It is a thorough study of the methods of solution of a problem common to a wide class of signal compression schemes. The problem emerges from the system of video signal bandwidth compression developed from the early proposals of Cherry and Gouriet

(11). Essentially, the problem is one of combining two signals which are produced by the system, so as to minimise the channel bandwidth occupancy whilst satisfying certain basic criteria.

The second part comprises the remaining chapters 4, 5 and 6 and has as its basic objective the measurement and exploitation by coding of second order statistics of picture signals with a view to signal bandwidth compression. 19.

The material of chapters 2 to 6 is so arranged as to make each chapter as nearly self explanatory as possible whilst preserving, it is hoped, a logically sequential whole. Partly due to this, critical reviews of past work are presented in each chapter rather than being combined into one separate review chapter.

1.6 SUMMARIES OF INDIVIDUAL CHAPTERS

Chapter 2. An outline is given of the bandwidth compression system based on a ruw-length signal representation. The resulting

"compressed" information requires specification by two parallel signals; the problem of combining these with a minimum channel bandwidth occupancy is studied. Several possible modulation and coding procedures are critically assessed in the light of basic requirements, and the conclusion is reached that combined amplitude and phase modulation is the most likely to afford a successful solution.

Chapter 3. A complete analysis of combined amplitude and phase modulation is presented. The spectrum of the modulated signal is computed using a series expansion technique due to Abramson (1),

Channel bandwidth, r.m.s. phase deviation and error rates are related, and the results are applied to provide recommended system parameters. Chapter 4. Second order statistics of video signals in terms 20.

of entropy and redundancy are measured. The measurement technique

involves conversion of the picture signal into digital form and

may be easily extended to the measurement of higher order statistics. Chapter 5. The systematic exploitation of second order redundancy by coding is considered for the case of black and white

picture material. Several coding schemes based on digrams of the

original signal are investigated, resulting in a simple code which

achieves a bandwidth compression by a factor of two. Chapter 6. A proposal is presented for a general technique whereby the errors resulting from the use of a non-uniquely de-

codable code which has certain otherwise desirable properties, may

be prevented or minimised by predictive feedback at the encoder.

Such an adaptive technique is implemented for a particular code

with a restricted digram alphabet.

A further proposal is made, though not implemented, of an adaptive procedure based on digram encoding, resulting in power and bandwidth saving and a code signal with zero mean level. 21.

CHAPTER 2 THE PROBLEM OF CHANNEL UTILISATION

2.1 BANDWIDTH COMPRESSION The basic redundancy of television transmission has, in recent years, invited considerable attention and research. Several compression systems which exploit some forms of picture redundancy have been proposed, the most significant being the systems of Cherry and Gouriet (11), Kretzmer (23), Roberts (36) and the system known as Run Length Coding. Very adequate reviews of these and other less basic systems appear in the theses of Prasada (33), and Kubba (25). As pointed out in Chapter 1, a successful compression system must encode the picture signal in such a way that a smaller amount of information is required to specify the signal, the quality of the processed or "reduced" picture remaining within the limits set by some subjective fidelity criteria. The end gain of any such system may be a reduction in the number of quantisation levels, in power requirements, in time of transmission or in bandwidth. Of these, the most desirable and at the same time the most tantalisingly difficult to achieve is the reduction in signal bandwidth; the realisation of bandwidth compression is the basic concern of the present work. None of the systems mentioned above, whilst achieving various types of information compression result in direct bandwidth compression.

At this stage a fairly strict definition of bandwidth compression 22.

will be proposed and this will be consistently adhered to in the rest

of this work.

A bandwidth compression is here defined as follows: given that

the original unprocessed information can be transmitted over a channel

with some specific frequency characteristic, and if as a result of

some process, the processed information can be transmitted using a

channel frequency characteristic identical in shape but compressed

on the frequency axis by a factor r, then a bandwidth compression by a factor r has been achieved.

In justification, it is felt that this definition, though by no means unique, is one which is realistic in the engineering sense.

Purposely, no mention is made of containing the spectrum completely within some band, thus avoiding the specific use of the troublesome rectangular frequency characteristic whilst not excluding it from the general definition. This feature will prove very useful in the context of facsimile transmission which is considered in later chapters.

It is now proposed to focus attention on a bandwidth compression system based on the process known as Run Length Coding and so introduce a problem inherent in the system. Possible solutions to this characteristic problem will form the material of this and the following chapter.

In effect, the principle of Run Length Coding was suggested by

Shannon (42) as an efficient noiseless encoding procedure for the case of a binary message in which one of the symbols predominates 23. in frequency of occurrence. Shannon suggested that the rare symbol should be represented by a special binary sequence, such as a series of zeroes, followed by a binary number representing the sequence or length of run of consecutive frequent digits. This approaches ideal coding as the probability of occurrence of the rare digit approaches zero. The situation is somewhat different in the case of television signals, in that no such assumption can be made about the probability distribution of amplitude levels, although long runs of constant brightness occur sufficiently frequently to render the technique quite applicable. Shannon's procedure was applied directly by Wyle et al. (50) in a pilot experiment using computer simulation with the aim of achieving a reduction in the time required for a facsimile transmission. As a first step the probability distribution of run lengths was obtained. Each run was then coded into a binary word using well known statistically weighted encoding methods for ensuring a minimum length code (Abramson, 2 ; Reza, 34,). This procedure produced a reported compression in message duration by a factor of approximately four for black and white material. A compression by a factor of approximately two was estimated for a picture quantised into eight levels. 24.

2.2 THE RUN LENGTH CODING SYSTEM

Investigations on a real-time run length coding system for television signals have formed the basis of recent research (Cherry et al., 12) in the Information Laboratory of Imperial College, of which the author is a member. As described above the technique, though achieving an information compression, does not yield a direct bandwidth compression. To achieve this objective further encoding is required, and a complete bandwidth compression system based on run length coding is shown in Fig. 2.1 as a simplified block diagram,

Only those points which are relevant to the present study will be described here; a complete description appears in the thesis of

Kubba (25).

The picture source is derived from a 35 mm transparency using a conventional flying-spot scanner operating on British television standards, (405 horizontal lines, 25 pictures per second, each picture consisting of two interlaced frames). A nominal bandwidth 6 of 3 Mc is assumed, requiring a Nyquist sampling rate of 6.10 clock pulses per second. These are provided by a stable clock source, from which are also derived the picture synchronising signals. Additional technical details are provided in section 4.4. The video signal is sampled at the Nyquist rate in regions containing picture detail; in regions of constant or near constant brightness samples are omitted. The decision whether the signal at any instant contains detail, and therefore requires sampling, or whether a constant run is in progress is provided by the Detail 25.

P IC.ru RE. (a) (6) DELAY SAMPLER —P--- vu R cE

DETAIL. D E-re.c.-rc)R O AND GATE

CLOCK PULSE. 500R CE.

I CHANNEL- I EL.AsTic. RECEIVER 4 SYSTEM I EN% CO De_R

Block Diagram.

Video Signal.

t Detail-gated Video Signal.

Compressed Video Signal.

t . Run Length (d), t I 52.21I 5 2.. I information.

FIG. 2.1. The Bandwidth Compression System. 26.

Detector. The output from this deyice takes the form of a binary

"sample" or "do not sample" command which steers the clock pulses

through the AND gate as required. Idealised waveforms at various

points in the system are shown in Fig. 2.1. The detail-gated clock

pulses are then used to sample the values of the original picture

signal in an amplitude sampling circuit. A delay is introduced

to compensate for the delay involved in the detail detection process.

The latter uses a statistical decision strategy in order to provide

the greatest possible discrimination between picture signal and

noise, (Kubba, 25). The noise referred to is predominantly source generated rather than channel generated, being caused by the granu-

larity of the photographic slide and the scanner tube phosphor. The

signal to noise ratio available from the system in terms of peak

signal to r.m.s. noise did not exceed 36 db.

The output of the sampler is thus a non-uniformly sampled

version of the picture signal, redundant samples having been re-

jected by the detail detection. Kubba (25) has shown that the

original signal can be closely reconstituted from these irregular

samples by means of zero order interpolation between successive

samples in which the value of a given sample is held until the

arrival of the next. Subjective comparison of the original picture

with the detail-sampled and zero order interpolated versions has shown that, for a full quality television signal, the number of samples retained could be reduced to approximately one third of the 27.

number of Nyquist samples otherwise required.

At this stage the system diverges from the proposals of Shannon

and Wyle et al. in that further encoding is provided to convert the information compression already achieved by omission of redundant

signal samples into a bandwidth compression. To achieve this the

picture samples must be fed into a buffer or queuing store and read out at its output at a uniform rate lower than the Nyquist rate.

Since such a uniform train of samples may be passed through a reduced bandwidth filter for transmission this constitutes a bandwidth compression. However, after the processing in the buffer store, information must be provided about the actual position of a given sample in real time so that the original spacing may be reconstituted by a decoder at the receiver. Thus two parallel signals are required, one giving picture amplitude information and the other decoding information about the queuing operations undergone by each picture sample. These will be referred to as the compressed video and the position signals respectively or the V-signal and P-signal, for brevity. A pilot investigation into such a buffer store or

Elastic Encoder was carried out by Pine (32) using electromagnetic coaxial delay lines and very limited storage. Pine showed the feasibility of such an encoder for the simplest pictures such as vertical bars. A project is at present nearing completion,

(Robinson, 37), having as its aim the digital instrumentation of the elastic encoder using shift register circuits. The increased 28. storage facility will allow the processing of real-life picture material. In this system, an analogue to digital converter is used to code the reduced-sample signal into a 6-bit binary code. The non-uniform digit spacing is equalised in a Digital Elastic Encoder resulting, at the output of the encoder, in a digit spacing equal ideally to r times the Nyquist interval. The compressed video and position information digits may then be reconverted to analogue form for subsequent transmission.

In connection with the decoding operation at the receiver, two proposals have been put forward. Firstly, the decoder could be a mirror image of the encoder, implying considerable complexity at the receiver. As the system is envisaged for point to point trunk transmission of television signals between communication centres, this is not an overriding disadvantage, A second proposal is to apply the compressed video information to the modulation grid of a cathode ray tube and the position information to a multi-velocity deflection circuit of the same tube. 29.

2.3 THE CHANNEL UTILISATION PROBLEM

It is possible to summarise the bandwidth compression system described in the last section from a black-box point of view by specifying the two output signals which emerge from the system. The nature of these signals will now be considered. The compressed video signal consists of regularly spaced samples occurring at a rate fN/r where fN is the Nyquist sampling rate of 6.106 p.p.s. and r is the sample compression ratio achieved. Let a conservative ratio of three be assumed. In the proposed digital development of the system, the compressed video signal is a

64-level, quantised signal. As the sample rate is fN/r a minimal filter of cut-off bandwidth fN/2r is required to transmit the information without ambiguity. As the original signal bandwidth is fN/2' (3 Mc), an apparent bandwidth compression by a factor r is achieved. The necessary presence of the position signal, however, com- plicates the issue considerably and explains the use of "apparent" in the above statement. The position signal specifies the length of run, in Nyquist intervals, of each compressed video sample, as indicated in Fig. 2.1. In theory any number of different run lengths, from one Nyquist interval to infinity, is possible. In the practical system, however, the number of different runs is restricted to a small permitted alphabet, any run lengths not belonging to the permitted set being broken down into combinations of these. At the time of writing current studies by Vieri (45) 30. indicate with certainty that the run length alphabet need not be larger than four. For instance, given permitted run lengths of

1, 2, 5 and 9 sampling intervals, a run of length 7 would be de- composed into a run of length 2 and a run of length 5, video samples being taken at the beginning of each. This does mean that the actual number of video and position samples is slightly increased and hence the compression in the number of samples is somewhat smaller than r; however, it has been estimated that the decrease is slight and hence, to avoid complication, the ratio r will be left unchanged.

Thus, the position signal can be regarded as a digital signal quantised into either three or four levels. As position information is required for each compressed video sample, the sample rates of the two signals are equal. Hence, the cut-off bandwidth required for the position signal is also fN/r. This means that, unless some further processing measures are taken, the V-signal and the P-• signal either require separate channels for transmission or, if frequency multiplexed on the same channel in an ideal manner, require a total channel bandwidth equal to twice their individual bandwidths, thus halving the overall bandwidth compression of the system to r/2.

If the achievable compression ratio r is only three, the result is an overall bandwidth compression of 1.5, which seems a poor return for considerable effort.

This distressing feature forms the essence of the problem referred to earlier; it is a problem which seems generic to all schemes in which an information compression in the time domain is 31.

converted into the frequency domain. Thus a study of possible

means of utilising available channel bandwidth for the transmission

of both signals with the lowest possible degradation in overall

bandwidth compression is essential.

The problem is one of finding a coding or modulation process,

or a combination of both so as to combine the two signals efficiently

whilst keeping error rates below agreed tolerance levels. The

maximum tolerable error rates for the two signals are clearly

different. An error in a compressed video sample would simply

appear at the receiver as a brightness error in the particular sample

or picture eTemert qffenf.ed; }his is the normal type of error to

which the television viewer is well accustomed and is easily toler-

ated. However, an error in the position signal will appear as a

spatial shift to left or right across a line of picture. A single

error may produce a shift equal to the length of the longest permitted run; an error burst may produce even larger shifts leading

to anomalous effects in the spatial form of the picture, to which

the human viewer is highly sensitive. The question of errors in

position information is treated in a current subjective investigation by Pearson (30), some of whose results will be referred to later. 3a.

2.4 ON EFFICIENCY OF COMMUNICATION

Procedures for processing the compressed video and position signals will be critically assessed in due course. Any such procedure must first and foremost satisfy the condition that both signals be perfectly recoverable from the combined signal given an ideal and noise-free situation. This is a sine qua non which, as will be seen, eliminates several seemingly feasible schemes.

Since the problem is one of the efficient utilisation of an information-bearing commodity, it may seem amenable to an Information

Theory form of attack. Unfortunately, information theory, though providing a powerful analytical tool for the assessment of communication systems, fails to provide concrete directives as to the means of synthesis. However, given a selection of alternative systems, information theoretic considerations can be fruitfully used to determine their relative efficiency; the possibility, therefore, of using such considerations as criteria for selection from several feasible alternatives warrants a brief digression.

A physically meaningful measure of the efficiency of a communication system has been proposed by Sanders (38) who defines an index of efficiency p as:

P W ...(2.1) = N R where P,N - signal and noise mean power respectively

W - channel cut-off bandwidth required 33.

R - rate of transfer of information from source to

destination. (This is Shannon's Mutual Information (Shannon0 42 Fano, 15) and is defined as the source entropy less the equivocation introduced by errors due to noise).

Now, Shannon's Noisy Coding Theorem, (42), states that an ideal encoding procedure, resulting in error-free reception, can be devised only if R is less than the channel capacity, C:

R 1; C. = W.log (1 + ;) ...(2.2)

For such an ideal system the rate of information transfer approaches the channel capacity:

R = W.log (1 + ...(2.3)

Combining equations 2.1 with 2.3, it is seen that an ideal system will have an ideal efficiency index, po, given by:

w Rw R o = R ( 2 - 1) ... (2.4)

An efficient system is clearly one which transfers as much information as possible in a given time for a given signal to noise ratio and given bandwidth; hence, referring to equation 2.1, t should be as small as possible, the absolute minimum value of p being Po. A system having an efficiency index represents the ultimate in efficiency in the information theoretic sense, and cannot be bettered. A plot of po against W/R is shown in Fig. 2.2. The index 3 has been computed by Sanders for various commonly used systems such as amplitude modulation and pulse code modulation, and some of his 34.

cRM— cszuAD RA7- uR N. v55.— vas-T-1 GI Ptl— StD-e--e, a-t.1 D. T dic).ct ivitex•vca secs. .s1 - over iy-ecy,e.tn ho..vtid 1/7 secs. AM

0 a dtr DEL NY DISTORTIo N

FIG. 2 . 2. Graphs Taken From the Results of R.W. Sanders, (top), and E.D. Sunde, (bottom). 35.

results are indicated in the same figure. A most interesting trend

is seen to emerge, in that some systems, in particular amplitude

modulation, tend to follow the ideal law indicated by the curve of p o as the ratio W/R is increased. Since a large W/R ratio is synonymous with a low information per digit, the attainment of the

lowest possible information per digit seems a prerequisite of an

efficient system. For a double sideband amplitude modulation

system, the channel bandwidth, W, is twice the message bandwidth,

B; the rate of information transfer, R, is equal to the mutual

information per digit multiplied by the number of digits per second

or the Nyquist sampling rate, 2B. Thus:

R = 2B.log2M ...(2.5)

where M is the number of quantisation levels. This assumes a signal

to noise ratio large enough for the equivocation to be negligible.

Hence, the largest W/R ratio obtainable with an amplitude modulated system occurs for the case of M = 2 and is clearly unity. This corresponds to the case of an amplitude modulated, non redundant binary signal as in Pulse Code Modulation. W/R can only be made to exceed unity, as required by the ideal trend of curve po, by the introduction of useful redundancy. Dupraz (14) shows that several redundant codes, such as the binary orthogonal codes, follow the trend of po, as indicated in Fig. 2.2.

In the context of the problem under consideration, therefore, it emerges that efficiency from the information theoretic viewpoint 36.

cannot unfortunately be a criterion for the choice of the required

system since the requirement for transmission of more information

per given bandwidth is in direct contradiction to the measures

indicated by information theory, Hence, the constraints within

which the problem must be solved doom any solution to information

theoretic inefficiency.

2.5 POSSIBLE MODULATION AND CODING SYSTEMS

Prior to further development it is instructive to mention that a problem similar in nature to the one under consideration was encountered and solved rather successfully in connection with transmission of Colour Television. In that context, a monochrome

"luminance" signal of full nominal bandwidth must be accompanied by two colour "chrominance" signals of bandwidth approximately 30% of the former. The three signals contain all the information necessary for complete colour rendering at the receiver. The problem of utilising a single channel normally available for a monochrome signal to transfer the complete set of three signals has been solved by the American NTSC (Fink, 16) and the French SECAM (17)systems. Both use a colour subcarrier whose frequency is made an odd half multiple of the line fundamental frequency. The spectrum of a still television picture is a line spectrum consisting of harmonics of picture and line frequencies. The subcarrier technique thus interleaves the subcarrier-based chrominance signal between luminance signal 37.

harmonics. The interleaving is perfect for a still picture; motion

causes slight spectral blurring, and some small overlap of the spectral

sidebands centred around the harmonics may occur. The optimum filter

for the mutual separation of the luminance and chrominance signals Pere, is quite simply a comb filter. Such a filter would eta complete

crosstalk - free recovery of all signals. As a comb filter of the

complexity required is unrealisable in practice, a seemingly startling

solution is adopted; quite simply no attempt at all is made to

recover the signals separately. The successful operation of the

system relies solely on the subjective acceptability of the resulting interference between the signals. Interleaving only serves to

minimise this interference, the optimum filter being unexploited.

This fact seems to be ignored in the literature on the subject. The

colour television systems are possibly the first attempts to match a communication system to the subjective requirements of the human

destination.

As already mentioned, the position signal in which errors can be ill-tolerated, must be completely separable from the compressed video signal, and any significant crosstalk is unacceptable. This requirement immediately and firmly precludes the application of the interleaving technique.

It is now proposed to consider several possible methods for the transmission of the compressed video and position signals.

The suitability of each will be critically assessed resulting in definite conclusions and certain proposals thought to be most 38.

capable of a satisfactory solution. For clarity, the following

basic points regarding the two signals are reiterated; firstly,

the V-signal may be regarded as either a continuous or a multi-

level signal (64-level). Secondly, the P-signal is digital and

quantised into either three or four levels. Thirdly, both can

be adequately contained in a frequency band of zero to B cycles per

second. It should be pointed out that at the time of this study

neither signal was yet actually available for experimental analysis.

Hence only the above basic properties could be taken as concrete

data; any further characteristics must be regarded as assumptions

needing justification.

In theory, the V-signal and the P-signal could be translated

in frequency to occupy adjacent bands of width B, by means of single

sideband modulation, in which case the resultant total bandwidth

would be 2B (see Fig. 2.3). This nominal bandwidth of 2B can serve

as a yardstick for comparison with systems to be considered.

2.5.1 Vestigial sideband modulation in adjacent bends

Single sideband modulation, though an obvious theoretical

possibility, cannot be applied in practice for several reasons.

The V-signal may be regarded as having generally similar low frequency characteristics to a normal television signal. Thus its spectrum extends down to frequencies lower than the fundamental frequency generated by the scanning process, i.e. the half-frame 39.

Compressed Video Signal (V signal) Frequency

Position Signal

(P signal) ' 0 B Freouency

Carriers Vestigial Sideband A B B Modulation in Adjacent V P Bands. Sec. 2.5.1. fel c2 Frequency Carriers As above, Psignal B B

Digram encoded, (P2). lS~ P2 a Sec. 2.6. 4 -B fc2 Freouency fcl f02 B/2 P and V signals combined B in Vestigial Sideband Channel. Sec.2.5.3. f -B f FrequenF el el

A f -B Quadrature Modulation or e Combined PM and AM or Coded DSB. AM. f -B Frequency Secs. 2.7,2,5.2,2.5.3. c

KEY : Ali- Amplitude Modulation PM-'Phase Modulation DSB- Double sideband. FIG.2.31 Diagrammatic Representation of Spectral Ocbupancy of Systems Discussed in Text. 1+0. frequency of 25 cps in the British system. The actual lowest frequency involved depends on the picture material which causes modulation sidebands around all the discrete half-frame and line frequency components. It must therefore be assumed that the V- signal spectrum extends to nearly zero frequency. Satisfactory single sideband operation, however, relies on a gap between the upper and lower sidebands produced by amplitude inodulating the signal, sufficient to enable the removal of one sideband by the use of a sharp cut-off sideband filter. As this separation does not exist in a signal having very low frequency components, a modified technique known as Vestigial Sideband Modulation must be adopted, in which a carefully controlled portion of the unwanted sideband is left. It is known (Black, 7 ; Wheeler,47 ; Ref. 4 ) that if the amplitude spectrum is made skew symmetric about the carrier, (see Fig. 2.3) and if the phase spectrum is linear, then, with the use of phase coherent product demodulation, distortionless detection of the modulating signal is possible. Conventional broadcast television systems use a vestigial frequency characteristic, the vestigial sideband being made to extend to about 25% of the full sideband width. However, envelope detection is used, the resultant distortion being subjectively tolerable.

Fig. 2.3 shows the spectrum of the resultant channel signal if vestigial sideband modulation is used to place the V-signal and the P-signal in adjacent frequency bands. It is felt that the more critical vestigial regions should abuttso that the low frequency 41.

spectral region is afforded some shelter from possible adjacent channel

crosstalk. Also, careful control of the frequency characteristic is then only necessary in one region of the total band.

The above vestigial sideband system suffers from several disadvantages. Firstly, a comparatively large total bandwidth equal to 2.5 times the message bandwidth, (W = 2.5 3), is required; the process thus limits the overall bandwidth compression to r/2.5, implying that the information compression ratio r, must be significantly greater than 2.5 for the scheme to be worthwhile. Secondly, real channels present spectral characteristics which are by no means ideally uniform in amplitude or linear in phase. This is especially true of wideband, long-haul coaxial systems. Maintenance of the correct vestigial characteristics by the use of phase-equalising networks proves difficult in practice, and the resultant distortion has to be tolerated. Though the distortion might be tolerable in the case of the V -signal, it is thought that the effect on the P signal could be very serious due to the coded nature of the signal, a point which will be elaborated in section 2.6.

A third objection is the necessity for coherent product demodulation, requiring knowledge of exact carrier phase at the receiver. As already discussed, both modulating signals contain very low frequency components, so that the transmitted carrier cannot be distinguished from them and extracted by selective filtering, a procedure used in single sideband transmission of speech. Hence the 42. carrier information must be sent in some other way. This is achieved in the NTSC colour television system by the provision of a so-called Colour Reference Burst (Fink, 16). This burst, consisting of a number of cycles of carrier, is sent during the line synchronising pulse, and provides a reference onto which a local oscillator is locked. This elegant technique inevitably suffers from certain practical imperfections (Richman, 35); it should be pointed out that most of the proposals to be made include the use of this method and that it is not unique to the above system.

It is felt, therefore, that the vestigial sideband system does not offer an adequate solution.

2.5.2 Quadrature modulation This technique offers another possible method for the transmission of the V and P signals. Quadrature modulation, (Black, 7 ), is a means of combining two independent messages around one carrier; this is achieved by amplitude modulating the two messages onto quadrature carriers of the same frequency. Thus if V(t) and P(t) are the message functions and we is the carrier frequency, then the modulated signal, s(t), is given by:

s(t) = V(t).cos w t P(t).sin w t c c ...(2.6)

As s(t) is the summation of two amplitude modulations, the bandwidth of the signal is twice the baseband width of the wider band message, i.e. 2B, (see Fig. 2.3). The system is outwardly simple, but suffers 43. from several shortcomings similar in nature to the vestigial sideband system considered previously. Detection of the two signals again entails coherent product demodulation using two local quadrature signals at the receiver. This again involves the reference carrier burst technique. More significantly, a phase error between transmitted and local carriers causes crosstalk between the two detected signals; specifically, an error of five degrees causes a signal to crosstalk ratio of 20 db, one degree causes 35 db. In order to achieve a ratio of 60 db, the phase error must be restricted to less than 0.06 degrees. Furthermore, phase error and consequent crosstalk are caused by any deviations of the channel phase spectrum from the ideal, as will be discussed in section 2.6.

2.5.3 Amplitude modulation with weighted encoding

In some applications envisaged for the compression system coarse quantisation of the video signal is quite acceptable. This situation occurs whenever simple black and white facsimile reproduction is not adequate, a somewhat larger number of shade gradations, four say, being required for a satisfactory reproduction.

These applications might typically include transmission of hand- written material, bank cheques, photographs and suchlike, for documentation or recognition purposes. In the compression system under consideration, this would typically involve a four level P signal and a four level V -signal; a total of four bits per digit.

Thus a 16-level signal could be used to specify both. In theory, 44.

therefore, the complete information could be sent with a 16-level

combined digital signal using vestigial sideband modulation, thus

achieving a total bandwidth of 1.25 B, (see Fig. 2.3). However,

the considerations of subsection 2,5.1 and section 2.6 which follows)

exclude this scheme as a real possibility. A more fruitful alternative would be the use of conventional

double sideband amplitude modulation for the transmission of the

combined 16-level signal. Though resulting in a bandwidth of 2B,

the system would suffer from none of the previously described faults.

In the process of combining the V and P signals into a single 16-

level signal, the more fragile position information could be re-

presented by the first two binary digits of a weighted binary, (PCM),

code, This would reduce the relative probability of error in the

P signal after reception, since the probability of error due to

Gaussian noise in the first, second, third n'th digits of a

weighted binary code may be shown to increase in the ratio 1,2,4 n-1 2 0 and so on, respectively. Bearing in mind that the proposed

digital development of the compression system will produce binary coded outputs, the implementation of such a system would be extremely simple. However, it will be seen in section 2.6 following, that a 16- level amplitude modulated signal is rather vulnerable to random noise and particularly channel distortion effects, so that it is felt that channels of a quality not normally available would be required. 45.

2.6 DIGRAM ENCODING, ERROR RATES AND CHANNEL DISTORTION

As the position signal is quantised into, at most, four levels,

it has been suggested (Prasada, 33) that it might be amenable to digram

encoding. This technique is a coding procedure whereby two con-

secutive symbols of the original message are represented by a single

symbol, or digram,of a new code (Abramson, 2). Specifically, given

an original message set X consisting for simplicity of binary symbols

x and x 1 2" the new digram code is a set Y consisting of four symbols, i,e.

Y Yl= x1x1' Y2= x1x2' Y3= x2x1' Y4= x2x2

In general, if the original message alphabet is M, the digram code

has an alphabet M2 corresponding to the number of group of two com-

binations. Clearly, since each digram specifies two original symbols,

the digram digit rate is half the original rate, and therefore the

digram encoded signal requires half the bandwidth of the original,!

Thus, with the P signal bandwidth reduced to B/2, the vestigial sideband system of subsection 2.5.1 would only require a bandwidth of approximately 1.9B, (see Fig. 2.3). The quadrature modulation system of subsection 2.5.2 would, of course, show no gain at all in this respect as the total bandwidth is determined by the broader band of the two modulating signals.

The penalty incurred in digram encoding is the squaring of the number of signal levels M, thus increasing the probability of error in the presence of noise. The situation is described by two 46.

generalised equations relating signal to noise ratio and error

probability in the presence of additive Gaussian noise:

S _ N • • • 2,7) and P(e) = g(M). Is exp(...x2/2).dx ...(248)

where:

- signal to r.m.s. noise voltage ratio

6 - quantal spacing between adjacent signal levels • r.m.s. noise M - number of signal quantisation levels P(e) - average probability of error

The functions f(M) and g(M) depend on whether the signal is specified as having a peak or average power limitation, on whether signal levels are balanced around zero level, and on the probability distribution of signal levels. Some elaboration is given in the Appendix to this section, and results of computations based on equations 2.7 and 2.8 are presented graphically in Fig. 2.4. The average probability of error, P(e), is plotted against the ratio of peak signal to r.m.s. noise for a balanced signal with equiprobable states in the presence of additive Gaussian noise. M is given as a paraw meter for the values 3,40 9 and 16 relevant to the digram encoding of the position signal.

A number of features emerge from the graphs. Firstly, to maintpir

47 .

_ —1 ., 1 , 1 I . W, 4111M1-rilli it 1 , P'11,1tii-1 111-11Jil 4111i 111J 10 , , ,, .•,.„,,, ,•,,, !!t .,•t- tT , 1 14 1 ,-.- ! , F , It 't ;7 ,1 ii-rt 1--titrifl 1 ,:r 11 .. 1fii 4:44- :::: . Hi:4i: •'111 : Ht ,-: t .! . :: : ];: .: i i !, :: ' • ' ThL Hi!i L .!. ... t i i , ; _': IT ii ' Y. '.: ii il!; iji iii: i1 i :: .. .. iii i : _i ,L, J i_i 1 . :::.!1 1 1:- ift 1 i-'1 if : . ,i. l',i j Th . ii...ii jiL i.:.i '-'irl ' - ,--1.' r•-', TIrj-,i.-- T1115:-4i -_,A -..--t,- I-4' , r ,• i'i :' i• •' ::• , 11 1 - ,11 i , 1 i i 1- , '1- 111' i 1 ,. 1 11 if4, ' 1 il If , 4 1 i ' 1111: il il , t -1- ' - ' , ' 1 ri -- f - 4 n; P "'"' 4 14 1 1 i c- : 1 i ' t i -, i-l' 4 1 1 I ti', ill } - 1- 1 1 i 1 11 r { —10 1-11t '1 1 ' ll I i I i P. ill 11', } 1 i hi. ,i f l I 1 1 I 4, 4 ', - r ii /.47 , f 4 r: i ,,,.-. . 10 i- r+ f*tt , -rF • it i H-i- 1-'41 + 4 H ,i,, • , • -f i •-

1_ I' ! ' i.if? ', ft;' ::1,,, I _ t7 ' J*4_ 1- -. ' ' , Ir 1 1 !

si 1-f- 1_L-T 4,141--_,1-',i if, 1 ' 1 I'i1: ' 0' 1 ttiii 11 , '.- f :1', • 1 ,43; ' + t ...I '.4-, _4-4 _ T ,. 1 ' , , :. --i,-. ' ' t -r- 4- -1-r-;4 1.- 1-,41,-- 1 rdinr• -, it ri t r 1 -ii ir-. ,i+t inal ;',-f fl a 1 1 11In n'i Nil —1 . 1 mita in 10 i----14- -1 , I.I: I 4- u i 1 h i-', 1' 1 1-1- -i- ! 1- T ' 0 ENT ttt IIMI ill-. 44-: i .4 r r r NEIMIIIMMISI I - t - ' ' ' tr',:_ _ t '-i -f',.. -71_,4: MEIN = -Li ,4 i l: ' N .: Mil Egginamm _, I r fl t,= ,, --iL, - ,_ ,L - ' '4,0 --H4 ' ' I i-:-T-- 4,-'' i,'-'i ,--t itf . 1 - \ Ili i _._,_ ,__ Ir" , 1 , -,_ 'F . , ,_, .4, - _,,,, :_,... ,_4,_ _-_,.., ,=.__ _ I.Y_,.. , , 1 In ,.. . Fri MAP N IT.i- 4 olii 1 m 0. I 11Hind 111111 1_, r Mal Inn KAI il I 111 4- 1111111 i 1111111111111 I 1 NM INIMIll HAI 111111111 Hi7 4 1-h -I -r , i- mom4. - man - 1r 1 I- -,:r. M No. of Amplitude Levels il __ _r4--fff_t_,: --4 __ ,,-_- 1-1, Additive Gaussian Noise, I-4- i 1 Peak Signal Limitation, 1 --- 4-4 n14 . ..7- ,_ Signal Levels Equiprobable. i IT ,,,+ •,___L-: 1 1 1_,, 4_, ,--k - * J#1,tticlipir 1,t' • +101,4-TrE--,+-TM ■ • RI: '1"lti'' -i - I. .1ii rin. I il Nli 11W .• 1 -1-7 1 11-117 ill AI N. i1, 1 IL' T al ii=i11n111 1111 III I- t us 1_ I MI 1111 l —0.1 I i 1 I " , -14 ir-. „r „i 10 So

* FIG. 2.4.. Probability of Error, p(e), vs. Signal to Noise Ratio. 48.

a constant error probability before and after digram encoding, an

increase in signal to noise ratio of 12 db is required for the case

of digram encoding from 3 to 9 levels, (case 1 for brevity), and 14 db

for the case of digram encoding from 4 to 16 levels, (case 2). Secondlyi

since the maximum achievable signal to noise ratio is a fixed quantity

for any given practical system, it is more realistic to note the in-

crease in error probability of the digram encoded signal for a given

specified signal to noise ratio. Thus, for a 30 db channel, which

would be considered fairly poor but acceptable by present standards,

the probability of error increases by about 50 orders of magnitude

for case 1, and by about 25 orders for case 2. For a channel with a 40 db signal to noise ratio, the corresponding increases are about

500 and 100 orders of magnitude for case 1 and case 2 respectively.

These figures are sufficiently astronomic so as to sow doubts about their relevance in any physical situation. In the transmission of digital data over telephone systems present experience indicates that it is difficult to achieve a digit error probability below about

10-6. It must be concluded, therefore, that given an adequate signal to noise ratio, say 30 db, Gaussian noise is not the main contributing factor in determining error rates; other effects obviously predominate, in particular intersymbol interference and impulse noise. The former is the mutual overlap of neighbouring digits when an imperfect channel, such as one of insufficient bandwidth, or one having delay distortion, (non-uniform delay with 49.

frequency), is used to transfer the digital signal. After trans-

mission over such a channel, the signal, when resampled at the

receiver, contains intermediate states in addition to the discrete

set permitted at the transmitter. This effect is thus similar to

that caused by finer quantisation and may be counteracted by

provision of extra signal power such that the smallest spacing

between received signal levels is maintained equal to the original

spacing 8, assumed previously. This increase in signal to noise

ratio necessary to offset channel distortion effects, termed the

"transmission impairment", is computed by Sunde (44) for rather

general forms of delay distortion common in practice for several

modulation methods. Sunde's results, (shown in abbreviated form

in Fig. 2.2), suggest that under all conditions the transmission

impairment for double sideband amplitude modulation is considerably

smaller (by at least 10 db) than for vestigial and quadrature mod-

ulation systems. For a number of signal levels in excess of three,

the impairment for the latter two systems, even for small amounts of

delay distortion, becomes rather catastrophic, being more than 20 db.

This substantiates remarks about distortion effects made earlier in subsections 2.5.1 and 2.5.2.

It must therefore be concluded that digram encoding prior to either vestigial sideband or quadrature modulation is completely unacceptable. It may be said that the two systems are too "fragile" to cope with the extra degree of complexity involved in digram 50.

encoding. However, the technique will be considered later in

connection with a more "robust" form of modulation. Moreoever,

it is felt that, even for the case of an uncoded 4 level original P-signal, the vestigial sideband and quadrature modulation systems

may be unworkable, unless channel distortion is stringently con- i~olle d.

2.7 COMBINED AMPLITUDE AND PHASE MODULATION

A system which is less prone to the effects which render the

vestigial sideband and quadrature modulation systems unsuitable will now be proposed. It is'suggested that the V-signal should be amplitude modulated and the P-signal phase modulated onto the same carrier.

As a complete analysis forms the basis of the next chapter, only qualitative remarks as to the choice of this scheme will be made in this section.

The main issue in the problem is the achievement of a compromise between two conflicting requirements; on the one hand a high degree of error immunity for the P-signal, and on the other the conservation of channel bandwidth; Any final choice can at best offer a tolerable balance between these two objectives. It is felt that the system proposed in this section may come close towards achieving the optimum balance.

In general, angle modulation, (either phase or frequency), offers 51. greater noise immunity than amplitude modulation. However, wideband channels are usually regarded as prerequisite for such systems.

(This emerges qualitatively from signal space considerations,

(Shannon, 41), where a wideband signal generates a longer locus in multidimensional signal space, thus increasing the distance between signal points and so decreasing the probability of error

(Kharkievich1 21 ). However, wideband operation diverges from the criteria of the present problem and hence the proposal must be inherently limited to narrow deviation conditions whose properties are only vaguely and incompletely known. Thus an analysis of such properties forms the bulk of Chapter 3.

The decodability of signals combined in the manner proposed is also established in Chapter 3. Digram encoding of the P-signal prior to phase modulation is a possibility; the implications of such a procedure are not, however, as clear as for the case of amplitude modulation, nd will be again left to Chapter 30 52.

2.8 CONCLUSIONS

Choice of system is basically determined by four criteria,

listed below in what is considered the order of importance:

1) Total channel bandwidth required.

2) Immunity of the position signal to channel distortion.

3) Immunity of the position signal to Gaussian noise.

4) Ease of implementation of the detection system.

It should be pointed out that the order is not strict; it is considered that the first three points are rather more significant than the last. Purposely, the immunity of the compressed video signal is not included since, as discussed earlier, errors in the

V--signal have considerably less effect and, in any case, every system described is capable of transmitting the signal with immunity comparable to that normally attained in television practice.

The last point merits inclusion due to the fact that in some systems unavoidable difficulties in the detection process introduce distinct limitations to system performance. An example is the possibility of phase error between local and received carriers in the quadrature modulation system due to an imperfect detector.

Furthermore, the requirement of some systems for coherent demodulation is a definite drawback which should be included in an assessment.

The results of the discussion of the present chapter may be conveniently presented in a table, (table 2.1) with a qualitative scale of goodness for each system. A three point grading scale is used, each point having the following significance: 53.

+ - best possible or nearly best

0 - average

- - worst possible or nearly worst.

Due to the flexibility of the three point grading, the scale may be

regarded either as having significance in the absolute sense, or as

being relative only to the six systems considered.

The table may serve to clarify the conclusions to be drawn.

Firstly, and most significantly, it is concluded that it is impossible

to achieve a practically feasible system requiring less than 2B

total channel bandwidth. This limits the overall bandwidth

compression achievable by the compression system to one half the

information compression, r, discussed in section 2.1. As the ratio r is considerably larger for coarsely quantised pictures than for

full-quality television, it seems that the system is inherently

more suited to the former type of material. However, recent

experiments by Vieri (45) have shown that, even for full quality or half-tone material, a compression, r, of up to five is possible if the low frequency components of the original signal, (up to 0.1 B), are transmitted uncoded by a band-splitting technique due to

Kretzmer (23). A resultant bandwidth compression of 2.5 would thus be achieved, which is a very significant gain. However, it is felt that the future of the bandwidth compression system in the light of the foregoing discussion and present day needs lies in the field of coarsely quantised pictures and black and white facsimile. Compression 54.

TABLE 2.1

Channel Channel Gaussian Detection System Bandwidth Distortion Noise System Immunity Immunity

VSB, Adjacent Bands, Uncoded P signal 2.5 B 0 0 0 Sec. 2.5.1. VSB, Adjacent Bands, Digram encoded P signal 1.9 B - - 0 Sec. 2.6. Quadrature Modulation 2 B 0 0 - Sec. 2.5.2. DSB AM Sec. 2.5.3. , P and V signals combined, 2 B + - + V signal coarsely quantised. VSB Sec. 2.5.3. P and V signals combined, 1.25B - - 0 V signal coarsely quantised, Combined Phase and

Amplitude Modulation, 2 B + + -4- Sec. 2.7. 55.

ratios, r, of typically 10 are expected for these applications, resulting in overall bandwidth compressions of approximately 5, after combination of the position and video signals in the manner

proposed.

Secondly, it is felt that, in a problem to which any particular

solution is essentially a compromise, the system of combined phase

and amplitude modulation offers the greatest chance of success.

The detection system need not, as will be discussed in Chapter 3,

rely on coherent demodulation, a further point in favour of the

system.

Lastly, it might be remarked that transmission of the two

signals on separate baseband channels each of bandwidth B is a

possibility, given that the characteristics of the two lines could

be strictly matched to prevent adverse differential distortion

effects. However, due to the limited number of trunk channel grades, it is unlikely that lower grade lines could be utilised,

This would necessitate the use of two standard video lines thus defeating the object of the undertaking.

56.

CHAPTER 3 ANALYSIS OF COMBINED AMPLITUDE AND PHASE

MODULATION OF THE PICTURE SIGNAL

3.1 OBJECTIVES

Several relevant qualitative considerations presented in the last chapter led to the choice of combined amplitude and phase modulation as the most feasible method for the transmission of the compressed video and position signals. To justify this choice quantitative results are required and hence the primary objective of this chapter is to present as complete an analysis as possible and to relate its general findings to the specific case under consideration.

At the outset two points require elaboration; firstly the decodability of the two modulating messages from the combined amplitude and phase modulated signal, and secondly, the effect of the simultaneous modulation of the two signal parameters on the modulated signal spectrum. Although the first point is heuristically obvious, its importance justifies a rigorous proof; this is presented in the Appendix to this section. The second point is by no means so straightforward and required a complete analytical investigation;, this is also given in the same Appendix. The results of the latter may be summarised as follows; the presence of simultaneous amplitude modulation of the phase modulated carrier 57.

has the effect of spreading the spectrum of the combined process

beyond that of either individual process, by superimposing a spectral

term which is the convolution of the two individual spectra (see

equation 16 of the same Appendix). However, this term is weighted

by a factor m 2, where m is the index of amplitude modulation,

and thus has only a second order significance for small values of

m (less than 30%, say),

It was considered that these results contained no aspects

which might invalidate the case for the combined amplitude and

phase modulation process, and hence an analysis was undertaken

with the following specific objectives:

a) To obtain the spectrum of the phase modulated position signal.

b) To relate the signal spectral occupancy to signal phase deviation and thus,

c) to determine error probabilities.

d) To investigate the possibility of encoding the position signal prior to modulation.

e) To relate these results to experimental results deter— mined by Pearson (30) for the subjective effect of errors in the position signal.

It should be stressed that since the amplitude modulation contributes only a second order effect to the combined spectrum, the analysis centres around the derivation of the spectrum of the 58.

phase modulated position signal.

3.2 ON THE SPECTRA OF ANGLE MODULATED PROCESSES

The analysis of the spectra of phase or frequency modulated

processes has received much effort, the results of which have

hitherto been characterised by a general lack of uniformity and

the absence of general conclusions. This is not surprising as

the problem is one of the most challenging in communication theory.

It is felt, therefore, that a brief comment on existing methods of

analysis will not be out of place: it will also serve to justify

the method finally adopted.

The most obvious approach is the classical method of analysis based on the assumption of a sinusoidal modulating signal. This analysis is well known (Cuccia, 13; Black, 7) and exact; however it becomes unmanageable for more complicated modulating waveforms.

The application of statistical techniques led to a more general characterisation of the modulating process, the Gaussian random model being the obvious choice (Middleton, 29). Such a model is much more general and realistic than any based on non- statistical considerations. The problem is thus the derivation of the power density spectrum of the modulated process. Since, by the Wiener-Khinchin theorem (Lee, 26), this is given by the Fourier

Transform of the autocorrelation function of the process, the 59.

analysis rests on the derivation of an expression for this function.

As will be seen, this is possible, the resulting function being

quite simple in form; however, no general functional form has as

yet been found for the Fourier Transform of this function. It is

this feature that has been the stumbling block in the analysis of

angle modulated spectra. Consequently, only approximate results

for various limiting cases have been obtained by several workers.

In partici/la. two limiting special cases were found to yield

definite conclusions; firstly, the case of a small mean square

value of modulating signal and secondly, the case of large but

finite value. For the former, the spectrum of the modulated signal is found to approximate closely to that obtained by amplitude

modulation of the same message. For large values the spectrum approaches the first order (simple) probability distribution of the modulating message. These results were independently obtained between 1948 and 1957 by several workers, the most prominent contributors being Middleton (29), Woodward (49), Blachman (8) and

Stewart (43). In addition to their lack of generality, these results were also difficult to apply in specific problems, pre— senting a maze of approximation terms of varying complexity. This unsatisfactory situation was transformed by an elegant paper in

1963 by N. Abramson (1) in which expressions for the spectra of angle modulated processes applicable in all practical conditions were presented. Thus the gap between the cases of small and 60.

large deviation was effectively bridged. Abramson pursued the

results obtained earlier by the workers mentioned above and succeeded in noting certain striking general features hitherto unnoticed. The computations which follow will use this new method of analysis and hence it is proposed to outline it in the following section.

3.3 THE METHOD OF ANALYSIS DUE TO N. ABRAMSON Let X(t) be a stationary Gaussian random process which modulates the phase of a carrier of frequency wc; then the modulated signal s(t) may be written as:

s(t) = cos pct X(t) ...(3.1) where 0 is a random phase angle. It is required to find an expression for the autocorrelation function of s(t) which is related to the power spectral density by a Fourier Transform. The following nomenclature will be used: R(t) - the autocorrelation function r(t) - normalised autocorrelation function R(t) i.e. 5 r(t).dt = 1 w(f) normalised power density spectrum

P mean square value of X(t) = Rx(o).

Subscripts refer to the specific process involved.

61.

If P is finite, it may be shown (Appendix to sec. 3.1) that:

-R(o) Rx(t) RRs(t) (t) =12 e - . e t cos wc

This may be conveniently normalised to:

-R (o) R (0).r (t) rs(t) 2.e x . e x x .cos wt

As the cosine term simply determines the position of the spectrum in the frequency domain, it is preferable to simplify the discussion by dealing with an "equivalent baseband" autocorrelation function r (t) : so

- r so(t) x = e P. Pire (t) ...(3.4) This may be expanded as a series:

P2 2 r so(t) = e + P.r(t)x + 21 . rx (t) +

r (t) +++ 31 . x3 ...(3.5)

To obtain the baseband power density spectrum, wso(f)' the Fourier Transform may be taken term by term: w (f) = e71) 8(f) + P.w (f) + w (f) m so x 21 • x x(f)

P3 4. . x(f) x(f) + + ...(3,6) where x(f) x(f) signifies a double convolution of x(f) with itself, i.e. wx(f) m wx(f) m wx(f). Further terms involve progressively higher order multiple convolutions. 62.

The expansion 3.6 was first obtained by Middleton (29) and then Stewart (43), but both failed to note certain important properties which render it applicable for quite general use. These were recognised by Abramson (1) and are listed as follows. a) Normalisation properties. As the spectrum x(f) is normalised, every convolution term such as x(f) m x(f) is also normalised. The sum wso(f) is likewise normalised. The total power, given by integrating is thus wso(f)' 2 -P v (f). df = e (1 + P + +++ ) = 1 50 2

If, therefore, only a finite number of terms, N, is used in the

expansion, the fraction of power lost by truncation of the series is simply: 2 N 1 (1 ) - e P + 2: + + + NiP ' •..(3.7) This fraction depends only on the mean square modulation, P, and

can thus be easily computed for any modulating process. b) Property of multiple convolutions. For many forms of modulating spectra, wx(f), met in practice, multiple convolution tends towards a Gaussian form, Abramson gives the example of a rectangular spectrum for which the second operation of convolution 3 (i.e. x(f) m x(f) ) produces a function which is already negligibly different from a Gaussian function of the same variance except in the tail regions.

c) The variance of the n-th spectral convolution is equal to

63.

n times the variance of the original spectral function wx(f).

Exploiting the latter two properties, the method thus consists

of replacing higher order terms of the expansion 3.6 by Gaussian

approximations of the same variance. In practice it is often

possible to derive explicit expressions for x(f) and

x(f) wx(f)' Replacing higher terms by Gaussian functions m : the desired approximate expansion for the spectrum becomes:

wso(f) = e-P L 8(f) P•wx(f) x(f) m x(f)

P3 + G(3) + 41 • G(4) ...(3.8)

where G(N) denotes a normalised Gaussian function of variance ? N. 2. a is the variance of the spectral function x(f) and is w given by:

2 00 2 cr w (f)• f w • df (3.

Expression 3.8 will form the basis of forthcoming computations. 64.

3.4 SPECTRAL COMPUTATIONS

It is now proposed to assume a realistic model for the power density spectrum of the modulating position signal and to compute the modulated spectrum using 3.8. As was pointed out in chapter

2, no experimental information as to the spectral characteristics of the position signal was available and hence the choice of some specific spectral model had to be made from a position of informed ignorance. It was decided that, in the absence of information about the correlative properties of the position signal, the worst case of a random noise spectrum should be used; however it was felt that the spectrum would be very unlikely to be white within the frequency band involved. Hence the spectral model finally adopted was that of random noise passed through an RC network.

It was felt that this model is more realistic than that of a rectangular spectrum.

The power density spectrum of the RC noise assumed above is given by (Middleton, 29):

A ...(3.10) (co) 2 1 + aw where A is the ordinate at the origin and a is the time constant

RC. The spectrum is band limited between - 12 and +ft at which points it will be assumed that the expression falls to half its value at the origin; this condition is given by an.= I.

If W (w) is normalised to w RC RC(w) (i.e. wRC (w).dw = 1),

65.

a final expression directly usable in computation emerges: 2 1 wRC(w) - nn, 2 for lw I

= 0 for lwl> fl ...(5.11)

This function is plotted in Fig. 3.1, In order to compute the next term in the expansion 3.8, the convolution of 3.10 with

itself is required. It was not possible to derive an expression for this. It was therefore decided to use a piecewise linear approximation to wRc(w) which has a straightforward convolution. The linear approximation to the RC spectrum is given by:

(1 - w I W (w) = B t2 1/ ) for Iw t <

= 0 for Iw, >

When normalised to wL(w), the expression becomes:

2 wL(w) = 3s1 167 ) for 10)1 (rt.

= 0 for lw I > rl ...(3.13)

The approximation is a close one as evidenced by Fig. 3,1 in which wRc and WL are plotted side by side. Moreover, the variance

for the RC noise spectrum is found to be 0,273512, compared to 0.2781/2 for the linear approximation. 0* The convolution wL(w) x w (w) is given by( wL(w).wL(x-w).dx L 00 66

o.16 FIG. 3.1. Spectral Functions Involved in

, 4-1 Computation of the Spectrum of Phase 0') 0.14 Modulation for P2.

.0:12 0

pi fg 0.10

0.0,8

0 . 0.04

0.04

,0.02

2s1 Frequency an

67. and may be evaluated in two parts:

w B2 (1 x +—w 2 ). (1 ).clx o

+B2 la (1 - x 2 ).(1 - x-°)2 )• dx ...(3.14) w where x is a variable of integration. It is then found that 2 2 3 w (w) x w (w) = 2 (A) + 4.(A)

for o < w < .4"/ and 2 3 2( SI-a2 ) (Si

for 2 n > co > n and

= 0 for I w I > 2n. ...(3.15)

The function is symmetrical and hence only positive frequencies are considered; it is plotted in Fig. 3.1 together with a Gaussian function of eqlml variance, G(2). The close similarity bears out remarks about successive Gaussian approximations made in section 3.3. From this stage the application of equation 3.8 is straight forward, entailing simply the tabulation of Gaussian functions of increasing variance. These are presented in the Appendix to this section. The number of terms required for a given accuracy 68. of computation may be deduced from equation 3.7. Thus, if an accuracy of 5% in terms of total power is desired, it is found that, for values of P equal to 0.5, 1, 2 and 4 the number of terms necessary in the expansion is 3, 4, 5 and 7 respectively. The spectrum is then computed point by point by weighting each spectral function according to the value of P chosen and summing as Indicated by the expansion 3.8.

The power density spectra for values of P equal to 0.5, 1, 2 and 3 radians squared are shown plotted in Fig. 3.2. The con- stituent functions of the expansion for the case of P = 2 are shown in Fig. 3.1 as an illustrative example.

It is noticeable that as P increases the spectrum approaches a Gaussian shape, the discontinuity of the modulating signal spectrum then contributing proportionally less to the summation,

Also, it is seen that power is removed from the discrete carrier component and progressively redistributed over a wider band.

FurthermoreF as P is decreased, corresponding to narrow deviation conditions, the spectrum is seen to approach that of the original modulation. 69

x 0.9

0.8

;>a 0

0 ct$ Normalised Power Density F-I o.6 0 Spectra of Phase Modulation with Mean 0 04 Square Angle_Deviation, P, as Parameter..: S-1 O 0. 0 111

0 (7.4' P 0 0 .

0.2

0.1

2XL Frequency 70.

3.5 EFFECTS OF BAND LIMITING It is now proposed to analyse the effect of band limiting the modulated signal spectrum. Two cases are of particular relevance in the present context. Firstly, the case of an uncoded position signal which is phase modulated directly onto the carrier; here the message bandwidth, B, is equal to the modulating signal band,S1 since message and modulating signal are identical. Secondly, the case of a position signal precoded into digrams prior to modulation; here the modulating signal bandwidth is half the original message bandwidth, i.e. ft = B/2. Let the channel be strictly band limited to a band Si; by the terms of reference described in chapter 2, the channel bandwidth is restricted to twice the original message bandwidth of the position signal, or synonymously, S2 = B. Thus, for the case of an uncoded position signal, (uncoded case,, for brevity), II = ft ; for the case of the digram encoded position signal prior to modulation, (precoded case, for brevity), = 211.

The effect of band limiting the spectrum may be computed for the two cases by considering each spectral term of the expansion

3.8. Clearly the resulting power loss depends on the mean square deviation, P; it is the object of the present section to derive this dependence in the form of curves for the two cases cited above.

This will allow a direct comparison in the next section on the basis of error probabilities. 71.

The fraction of power left after band limiting the modulated spectrum to a band to +SA may be expressed as:

-P 2 P3 e 1/41 + a P + a2.-- + a +++ 1 21 3 31 . (3.3_6 )

where the coefficients a k are the fractions of power left in the individual terms of the expansion 3.8 after band limiting, i.e.: SI X . dw al S and in general n w n w . dw ak 1_, x x ...(3.17) For the uncoded case, = and the carrier term and the w fl, x term are unaffected (i.e. a1 = 1). a2 is obtained by integration of equation 3.15 between the limits - 41 and + fl ) and successive coefficients a k are then error functions, k(n), of variance 2 kcrw :

1 SI2/2ku? ak = ak( a ) - jr e w . dw o'141/2 n k -1/ f ..(3.18)

The values of the coefficients ak for the above case are given in Table 3.1. As for this case only small values of P will be

relevant (P 1), the number of terms taken in the expansion is sufficient for an accuracy of 0.1%.

72. Table 3.1

Uncoded Case

a a3 a5 al 2 a4 a6 1 0.801 0.726 0.654 0.602 0.480

Precoded Case , ft = 211

a a5 al 2 a3 a4 a6 1 1 0,971 0.942 0.910 0.878

a7 a9 a8 a10 all alt 0.848 0.822 0.794 0.767 0.746 0.726

The percentage of power lost by band limiting the signal to within a frequency 1~ about the carrier frequency is then computed for various values of P up to 1, from equation 3.16 and Table 3,1. An identical procedure applies to the case of a precoded signal where n. = 2.11.; here the constant term: the wx term and the w m w term are unaffected by the truncation (i.e. a x x 1'a2 = 1); thus, only the error functions k(21) beyond k = 3 have to be computed. In this case 2S1 2 . 1 e-w /21ca ak = s k(2 f1) w . dw w 2ic k -211 ...(3.19)

The coefficients ak for this case are also presented in Table 3.1.

The large number of terms is necessary in order to ensure an accuracy 73.

of 0.3% for P = 4. Progressively higher accuracy holds for smaller values of P. The results of the above computations are shown in Fig. 3.3 in which the percentage power lost as a result of band-limiting to within frequencies Si and 217L on either side of the carrier frequency is plotted against the mean square value of the modulating

signal .

It is seen that for a given percentage power lost, the allow-

able mean square angle modulation is considerably larger for the case when the modulating position signal is digram encoded prior to modulation; the increase is by a factor of 5.5 for a 1% power loss and 4.2 for a 5% loss. However, before a decision can be made as to which procedure is to be preferred, the mean square deviation,

P, must be related to the probability of error, as will be shown in the next section.

3.6 PRECODING OF THE POSITION SIGNAL

A signal phase modulated by a message having M discrete states is given by:

s(t) = A.cos(coct + 0i) ...(3..20) where Oi = 1,50, is the phase deviation corresponding to the i-th message; 80 is the incremental phase between adjacent message points and A is a constant. The mean square modulation, P, is

t r t,14-1144 ne.ffsee.:•1 T'l J1V- 44 " 1:il ,tli 111

FIG. 3.3. Power Truncated by Band Limiting PM Spectrum at .S2 and 2.52 as a Function of Mean Square Angle Deviation, P,(sec. 3.5).

f: 111111.:11nil III .F r — . .11'911E Moil. c' WE NE _,

111111111 11111111110 1111111111 111111111 11 1111111111

sminummium::::: wimmusimummammx .1:: mmommimmimmis mirnarmwsm sl. = ja mommenum OMMEMOMMOM WI an

1111111111 11111111ffawl Mill MOM SS 1104.01611 41....firdiar

1111111111111111111 11111.111 I111111111111NI1111I11111111111111111 = 11111 IIIIIIIIIIIIIIIIIIIIII 111111w 11 111111111111119 w 2 EOM APR O a,

::IMEMEMEMNMMOUMMINFill R. MIN

1 2 3 4 radians s•uared. 75. given by:

...(3.21)

where P. is the probability of the i-th message. Equation 3.20 assumes that the carrier phase is known at the receiver, implying coherent reception.

As the incremental phase 60 determines the distance in signal

space between any two signal points, the larger 80, the smaller the probability of error due to random channel perturbations. Expressions, both exact and approximate, for the probability of error due to random

additive Gaussian noise for the cases of digital amplitude, phase and frequency modulation have been derived by several workers (Cahn, 10; Arthurs and Dym, 3); the results of Arthurs and Dym„ which are particularly instructive, show that the average probability of error, p(e), for a digital phase modulated signal has bounds given by:

< P(e) < 2 ...(3.22)

1 where 'VETE r14r -1 .sin 80/2

If the phase modulation is balanced around zero carrier phase and if all levels are equiprobable, then equation 3.21 becomes: (M+1)/2

P = .2. 80 2 for M odd 76. and M/2 2 2 . 1 2 for M even ...(3.23) M . Bsd

Since the values of P corresponding to a given percentage signal power loss have been computed in the last section for both the case of an uncoded modulating message and a digram-encoded message prior to modulation, it is now possible to compute the incremental phases 50 and 80 respectively, for the two cases, o 1 using equation 3.23. As an example, from Fig. 3.3 for a 5% power loss P cannot exceed 0.85 radians squared for the uncoded case, and 3.5 radians squared in the precoded case. The angle 60 is tabulated for this condition against the number of signal levels in Table 3.2.

Table 3. •

Phase deviation 80 for uncoded and precoded cases with

5% power truncation due to band limiting.

Number of 50 , rads, rads, Increase in S/N for o 8.01' Levels, M precoded case for uncoded case precoded case equal p(e)„ in db. (See eq. 3.25). 2 1.84 3 1.13 4 0.83 1.68 Coding from 9 0.73 M to M2levels 16 0.41 2 to 4 0.6 3 to 9 3,6 4 to 16 6.0 77. It is immediately seeri that 80 is smaller throughout in the case of

the precoded signal. This signifies that the average probability of error is larger and hence it may be concluded that the procedure of digram encoding prior to modulation is of no advantage. In fact, for equal error probabilities in the two cases it is seen from equation 3.22 that:

1 o o 1 80 8° . sin N sin 2 2 ...(3.24) P0 N1 where the subscripts refer to the two cases; the increase in signal to noise ratio, in decibels, required for the precoded case to maintain equal error probability is thus given by: sin 800/2 20 log 10 sin 801/2 ...(3.25)

A plot of this quantity for the cases of a 2,. 3 and 4 level uncoded message, and for several values of percentage power truncation due to band-limiting is shown in Fig. 3.4. It is seen that in all but one case there is a finite loss incurred in the precoded case and that this loss increases as the number of message levels involved increases.

It may be concluded, therefore, that digram encoding of the position signal prior to phase modulation is of no advantage. The incremental phase angle 8 is tabulated in table 3.3 for M=3 and 4 and for $, 5% and 10% spectral power tbuncation due to band limiting, for the relevant uncoded case. 78

FIG. 3.4.1 Deterioration in S/N Ratio 1 due to Precoding of Position Signal Prior to Modulation. (Eq. 3.25.)

.. . • •; • I ., ! ILI, • • •

-I L

• 79. Table 3.3.

Incremental phase angle 80 for several values of

percentage power truncation due to band limiting.

No. of 80, degrees.

levels, M 1% 5% 10%

3 38.4 64.8 78.8 4 28.1 47.2 57.4

.3.7 COMPATIBILITY WITH KNOWN REQUIREMENTS In a recent series of experiments Pearson (30) studied the

effects of errors in the position information. In the experiment,

position errors were simulated by shifting the picture at random

across a line by a predetermined number of picture points. Subjects

were then asked to compare such a picture with a normal picture

to which Gaussian noise had been added. It was thus found that

for a fidelity equivalent, for instance, to a normal 40 db picture

to noise ratio, 150 errors per second could be tolerated, if the

picture error was confined to a shift of one picture point. For

a shift of 16 picture points, the worst case considered, the tolerable

error rate was found to be about 0.5 errors per second. This worst

case, corresponding to a probability of error of about 2.5.10-7

for the sampling rate involved, may thus be taken as a target

requirement. 80.

It remains to demonstrate that this requirement can, in fact,

be met by the system proposed above. Taking the case of M=4 and using Table 3.3 and equation 3.22 it is found that with a 30 db signal to noise ratio in the channel, an error probability of -16 1.9,10 is obtained for a spectral power truncation of 1%. If a 5% truncation can be tolerated then a probability of error of 3.5,10 _10is obtainable with a signal to noise channel ratio of only 20 db. These values are well in excess of the target value mentioned above.

Conclusions as to the above results should, however, be tempered by the realisation that these results apply only to the theoretical case of additive Gaussian noise. Telephone system experience over trunk connections indicates that such theoretical results may be optimistic by about 10 db in terms of signal to noise ratio. Even with this proviso it is clear that the system described is thoroughly capable of meeting the stipulated re— quirements. 81.

3.8 SUMMARY AND CONCLUSIONS

The system of combined modulation proposed in chapter 2, whereby the compressed video signal is amplitude modulated and the position signal phase modulated onto a common carrier has been analysed and found to be feasible.

It was shown (Appendix) that the spectrum of the combined modulated process is largely determined by the spectrum of the phase modulation.

A realistic spectral model for the position signal was assumed and the method of analysis due to N. Abramson used for spectral computation.

It was found that for a given percentage power truncation as a result of channel band limitation, no advantage was to be gained in precoding the position signal by digram encoding prior to modulation; in fact a loss is incurred by this procedure (Table 3.2).

The results obtained facilitate the choice of system parameters.

However, these were not found to be very critical; for a 1% power truncation due to channel band limiting and a 4-level signal, an incremental phase deviation of 28° may be used, and for a 10% truncation this may be increased to 570 (Table 3.3). An amplitude modulation index of up to 30% may be used for the compressed video signal.

It should be pointed out that the effect of spectral truncation 82. as a result of band limiting the modulated signal is not known and

was therefore treated as an unknown parameter. This problem is a fundamental one in cases of spectra of processes whose locus in signal space is non linear and which extend well beyond the modulating spectrum; it is thought that this topic deserves separate basic research* The primary object of such research would be to establish, if possible, relevant criteria which deter— mine the effect of band limiting of infinite spectra* 83.

CHAPTER 4 ] MEASUREMENT OF DIGRAM STATISTICS OF PICitairi SIGNALS

4,1 STATISTICAL REDUNDANCY

Visual information sources, whatever their nature, normally contain both perceptual and statistical redundancy, The compression scheme described in Chapter 2 is based on the exploitation of the former type. However, some visual data may not convey any meMories to a human observer. In such a case, the communication between

source and observer does not involve perceptual redundancy; at the same time the source may contain quite finite statistical redundancy; In the light of information theory this may be exploited to achieve

more efficient communication* The actual improvement in efficiency can only be predicted if the source redundancy is known. Implemen- cation of an actual system based on this prediction is then a matter of sufficient ingenuity, In many cases, the method is clear, but the complexity involved is often overwhelming. Nevertheless, given

data about the redundancy of a source, an absolute reference for the upper limit in improvement is immediately established* This chapter is concerned with outlining a new technique for the measurement of statistical redundancy of picture sources. The data so obtained will be used in the implementation of a more efficient visual data transmission system described in subsequent chapters.

Hitherto, the term "statistical redundancy" has been used rather 84.

vaguely. Different degrees of redundancy may be defined, each associated with some chosen source statistic. In a discrete source,

redundancy in its simplest possible form, known as first order re-

dundancy, exists if the probabilities of occurrence of different symbols are unequal, whilst the symbols are assumed to be generated

independently of one another. Dependence between adjacent symbols

only constitutes the next rank, (second order), in type of redund.- ancy. As dependence between longer blocks of symbols is taken into account, higher orders of redundancy are brought to light; computation of such higher order redundancy, hoWever; requires in- creasingly higher orders of statistical data. For instance, a statistic in the form of the set of probabilities of all the possible different pictures which a television system is capable of transmitting, would yield a measure of very high order redundancy. The redundancy associated with such statistics could be regarded as rea-

sonably complete. Due to the astronomical number of pictures in-, volved a much less ambitious measure of redundancy must suffice in practice.

At this stage a clarification of the nomenclature to be used and the general context of the situation will be given. The picture in a television system is assumed to be completely representable j'y a version quantised in both amplitude and time. Thus, it may be considered to have a mosaic structure of picture elements. In most television standards the number of such elements is of the order of 850

200,000, assuming equal horizontal and vertical resolution. The number of quantisation levels deemed necessary for a satisfactory rendering of tonal gradation is usually given as 64, (6 bits). A. particular amplitude level in any given picture element, denoted by xi, belongs to the M-level set X composed of symbols[xi,x2,...xi...xml. The set of simple probabilities P(X), associated with the set X, forms the first order statistics of the picture. Leta particular level in an adjacent element be denoted by yi; then the two-dimensional

set of conditional probabilities P(Y/X), having elements such as

P(Yibti), together with the set P(X)„ form the second order statistics

of the pictures The sequence of two symbols x, y,3 say, is referred to as a digram belonging to the digram set XY. It is the redundancy calculated from these second order pieture statistics which it is proposed to consider here. Though the exact definition of the term is left to section 4.2, the reasons for the choice of redundancy as the relevant measure describing the source characteristics should be pointed out; firstly, being an average measure of a whole set of statistics, it provides a concise end result; secondly it plays a central part in coding theory, as will be indicated in Chapter 5. The measurement of the statistics of television signals has been tackled by several workers; it is considered that the most significant contributors are Kretzmer (24) and Schreiber (39), whose methods will be briefly reviewed* 86.

Kretzmer measured the simple probability distribution of signal

levels, P(X), the second order distribution P(Y/X) in the form of

the distribution of the differences in amplitude between adjacent

picture elements, and finally the spatial autocorrelation function

of the picture. The first two sets of measurements were obtained

by means of what Kretzmer named the "probabiloscopet', a device which

made use of electronic, optical and photographic techniques to

produce a direct record of the probability density function on

photographic film. It is felt that this ingenious technique is$

however, prone to certain inaccuracies. Firstly production of

en optical wedge with a logarithmic scale extending over six decades

must involve some inaccuracy. Secondly, stray light is mentioned

as a limit to accuracy. Kretzmer himself does not quote any specific

accuracy of result.

Schreiber's experiments involved the measurement of second and

third order statistics. A full-standard television signal generated from a transparency was processed in a specially constructed beam- switching tube which produced an output pulse whenever the signal attained a specific level, the level being set by a reference voltage applied to an electrode. A cycling circuit stepped the reference voltage by one quantum level every frames (1/30 sec.), and the readout proceeded automatically, the data being processed by a computer to give the end result in terms of relevant entropies.

A third order measurement of one picture took two and a half hours. 87.

The apparatus entailed very considerable electronic complexity,

in view of which it is remarkable that a reported accuracy of 3%

was achieved.

Both Kretzmer's and Schreiber's results will be subsequently

Compared with those obtained by the author. It is felt that the

technique of measurement to be described in section 4,3, is markedly

simpler than either of the above, and, coupled with the theory

developed in section 4,2, offers great scope for future measure—

ments of higher order statistics of signals, whilst still retaining

reasonable simplicity.

4,2 CALCULATION OF REDUNDANCY FROM LIMITED STATISTICS

This section is intended to demonstrate the connection between

the information theory measures of entropy and redundancy and rele—

vant statistics. It will be shown that the technique of digram

encoding provides a strong method of estimation of these quantities.

The entropy of a discrete source X, whose symbols are assumed independent is given by:

H(X1) = p(x).log p(x) ...(4.1) X

(The superscript 1 is inserted for clarity as will become clear later; for the summation convention see the Glossary of Symbols.) 88,

If the source symbols are statistically interdependent, a smaller

amettm4 of information is conveyed by the source since the occurrence

of preceding symbols provides some measure of prediction about success-

ive elements. Thus a truer measure of the actual entropy may be

obtained by considering a set S composed of a sequence of N symbols.

If it is assumed that now all sequences S are independent, the en-

tropy associated with the set S is similarly given by:

H(s) = :,>, p(s), log p(s) S

The set S„ each symbol of which consists of N consecutive symbols of th the set X, is known as the N extension of the source X (Abramson, 2).

Thus, the entropy of the source per symbol, calculated on the basis

of sequences S of length N, or the entropy of the Nth extension of X, is denoted by H(XN) and defined by:

H(XN) = 1-N ° H(S) ...(4.3)

As the length N is increased, dependence between sequences S dim-

inishes, and H(XN) decreases to a limiting value H(X), which can then

be considered as the true or absolute entropy of the source X, and is defined by:

H(X) = Lim H(XN) N -3 co

This implies that, on an average, H(X) so defined is the absolute minimum amount of information, in bits say, required to specify the source X completely and uniquely (Fano,15). However, for finite N, th it may be that some higher than N order statistical dependence exists, so that the conditional probability P(X/S) P(X), in which

case it is necessary to consider the conditional entropy H(X/S). This is the information required to specify a symbol of X when the preceding sequence S is known, and is given by:

H(X/S) = - p(xls).log p(x/s) XS

For the sake of clarity, a non-rigorous notation is introduced by writing H(X/S) as H(X/N), to denote conditional entropy based on

sequences N symbols long. It may be shown, (Shannor?.142), that: N-1 H(X/N) = N.H(XN) (N 1).H(X ) ...(4.6)

Thus, the conditional entropy H(X/N) provides a better approxu•

imation to H(X) for a given N, than does H(XN). It is known as th the N order entropy of the source, since it is, in fact, the th closest approximation to the actual entropy, given N order statistics. Equation 4.6 indicates that to compute the N-th order entropy the entropies H(XN) and H(XN-kl) must be known. This requires the N measurement of the set of joint probabilities, P(X )associated with the N-th extension of the source. The set of probabilities P(XN -1) can then be derived from the set P(XN)„ thus enabling the calculation of H(X/N) from equation 4.6. It is suggested that the N-th extension of X may be conveniently obtained by a process of successive digram encoding. Thus, 90.

to reach the N-th extension, m repeated digram encoding operations are necessary where N = 2m. After only three such operations, for instance, 8-th order statistics are available and computation of the 8-th order entropy is possible.

The various relevant entropy measures involved in such an iterative procedure are illustrated in Fig. 4.1. Successive stages of measurement and computation are indicated by the ringed numbers.

FIG. 4.1. Iterative Entropy Estimation.

It is now proposed to consider the simplest case of a binary source and its digram encoded extension since this is basic to the subsequent experiments. The estimate of entropy based on a knowledge of digram statistics is given by equation 4.6: 91.

H(X/2) = 2 H(X2) H(X1)

Denoting the two binary states by 0 and 1 the above results become: H(X1) = - p(x).log p(x) X

p(0).log p(0) + p(1).log p(1) ...(4.7)

(Logarithms to the base 2 unless otherwise stated). The set X2 is composed of digram elements

00, 01, 10, 11 and thus, from equations 4.2 and 4.3:

H(X2) = [ p(00).log p(00) p(01).log p(01)

+p(10).1013 p(10) + p(11).log p(1l)] .A.(4.8)

The second order entropy H(X/2)is then:

H(X/2) = [p(00).log p(00) + p(01).log p(01)

+p(10).10g p(10) + p(11).log p(11)

-p(0).log p(0) p(1).log p(1)] ...(4.9)

This is the best estimate of entropy available with second order statistics. As the maximum information per digit is one bit, the second order redundancy, R2, is defined by:

R2 = 1 - H(X/2) ...(4.10) 92.

This measure will be referred to as the total second order redundancy, as it includes both the redundancy due to actual second order interdependence between symbols and the redundancy contributed by a non-uniform first order (simple) probability distribution. It may, however, be of interest to separate the two effects, in which case the first order redundancy, Pti, is given by Ri 1 - 11(x') ...(4.11) and hence the redundancy due to second order effects only is given by the difference

Ri - R2 = H(X1) H(X/2) ...(4.12)

4.3 THE EXPERIMENT Two logical steps are involved in the experiment; firstly, the conversion of the picture signal into digital form as a number of parallel binary outputs; secondly, the measurement of the digram statistics of each binary output enabling the computation of the second order redundancy as indicated in section 4.2. The redundancy of the picture signal is then the summation of the redundancies of the individual binary outputs. The functional block diagram for the experiment is shown in Fig. 4.2. A description of each item of apparatus appears in section 4.4, technical details being left to the Appendix. At this stage only experimental procedure will 93.

ANALOGUE. 0 To DIGITAL VI 0 S_O NVS. t..12 S IGNAI--,

Vsird a 3 Ai- S DIGITS

MO h! ITC* R

DtG RANI DIG rrs

FIG. 4.2. Functional Block Diagram of the Experiment. 94.

be considered.

The video signal, with line and frame blanking but without

synchronisation pulses, is fed into an analogue to digital converter

as shown. The converter quantises the signal into 32 levels and

provides the five resultant binary digits as parallel outputs. Each

output is Sampled at the Nyquist rate of 6 Mc. Monitor displays of the five converter outputs are shown in Fig. 4.3. In what

follows) presence of a sampling pulse is denoted by 1, absence of

a pulse by 0.

The digram probabilities of each binary output from the

analogue to digital converter are measured by means of a digram

encoder. This device accepts a binary input signal, 0 or 1, and

provides four parallel outputs indicating which of the four possible

digrams, 00, 01, 10 or 11, has occurred., As the digrams form a

mutually exclusive set of events, only one output at a time is active,

the occurrence of a particular digram being indicated by a pulse.

The digram readout rate is half the Nyquist sampling rate, since each

digram represents the signal over two past Nyquist samples or picture

elements.

The experimental procedure consisted of taking complete

measurements for each output .git of the converter in turn.

Probability measurements were made by counting, on a high-speed

pulse counter, (Hewlett-Packard model 524 C 10 Mc counter), the number of pulses occurring in a fixed time. A counting time of

10 seconds was chosen, which corresponds to approximately 250 picture 95.

Original, Picture 4. 1-st Digit

2-nd Digit 3-rd Digit

4-th Digit 5-th Digit

!FIG. 4.3.1 binary Outputs of Analogue to Digital Converter. 96. presentations. In this way it was hoped that any random errors would tend to average out. An experiment was started by connecting the counter to the first digit output of the analogue to digital converter, and recording the number of pulses in 10 seconds, thus measuring the probability p(l), since the total number of pulses is 6 known to be 6.10 per second. The counter was then connected in turn to the four outputs of the digram encoder, and pulse counts were made of the numbers of 00, 01, 10 and 11 digrams occurring in

10 seconds. These measurements enabled the computation of the second order entropy for the first converter digit from equation 4.9. The procedure was repeated for all five digits of the analogue to digital converter in turn, a single picture requiring 25 pulse 2 counting measurements in all. This compares favourably to the 32 measurements required by Schreiber's method. The pictures selected for measurement are shown in Fig. 4.4. They represent a fair cross-section of picture material, ranging from the fairly simple portraits, the reasonably detailed aeroplane and rockets, to the complexity of the roof-top view and the synthetic test card C. Statistics for the first digit were also measured for two black and white pictures representing an electric circuit and a map of Europe; these are shown in Figs, 5.3 and 4.4. Prior to each measurement the video signal was adjusted so as to fill as nearly as possible the full amplitude range of the converter. At the same time the level of blanking pulses was set at 97.

Picture 1 .:?-1cture- 2

POLA ND.

•

FRANc. ke::-..xusTRIA-ojrIGAR-i:— e• • U RV MAN IP L t A

GAR IA

Picture 3 Map

ffilOMP4010411014,M1REIEL, r 0_4111ELLAA01 Picture 5 Picture 6

[pia, 4.4.1 Test Pictures. 98„

approximately black level by means of a clipping circuit. Thus the

video signal as measured included the standard duration blanking

period, during which time the signal level was held at black.

The proper functioning of the whole experimental apparatus was

checked in two ways. Firstly, the four digram probabilities were measured independently, so that their summation provided an immediate and valuable check during measurement. Secondly, as an overall check, a high quality noise source, (General Radio type 1390 B), was used as a test signal. In theory, such a source should be truly random and have a symmetrical distribution; under such assumptions there should be no measurable redundancy, a condition which provides an absolute check. Quantitative assessment of the results of both these checks is given in the next section.

As the above checking procedures provided a large measure of control and confidence in the experiment, no elaborate attempts were made to measure the repeatability of results under strictly controlled conditions. However, separate measurements taken for the same pictures on different occasions showed that the results were constant to about

7%. It is stressed that no attempt was made to control the conditions to any extent apart from the signal level setting-up procedure described earlier. Thus, such factors as the accurate positioning of picture slide relative to scanning raster were not taken into account.

It is felt that such precautions would have been meaningless in view of the arbitrary nature of any particular choice of standard conditions.

The resultant repeatability to within 7% is, in the circumstances, quite 99: interesting in itself.

4.4 APPARATUS It is here proposed to describe only the essentials of the apparatus used; some further circuit and design details are left to the Appendix to this section. A photograph of the experimental layout is shown in Fig. 4.11.

4.4.1 The picture generating equipment

The picture signal is derived from a•35 mm transparency by means of a flying spot scanner in a conventional equipment which provides a central service for the whole laboratory. The picture and synchronisation signals are distributed to various points on 75 sikenv coaxial cables, as in normal television studio practice. Synchron- isation signals are derived from a basic clock source. The clock pulses are generated at the Nyquist rate of 6.106 p.p.s. corresponding to a nominal 3 Mc video signal bandwidth, timing accuracy being controlled by a 6 Mc crystal oscillator. The clock repetition rate is divided by a factor of 297 resulting in synchronisation standards very close to the British Television standards; frame frequency is not, however, locked to the mains frequency. The divider was constructed by the author and uses nine transistor bistable circuits 100,

connected as binary dividers; the required ratio of 297 is obtained

by suitable logic feedback loops. The output of the divider provides

pulses at a rate of 20,202.02 p.p.s. recurring, which constitutes

the twice line frequency of the system. These are in turn used to

drive a synchronising pulse generator which supplies all the re-

quired frame and line synchronising signals and trigger pulses. The

resulting frame frequency differs from the mains frequency by a small

fraction of a cycle per second. The central picture generating

equipment has been in existence in basic form for a number of years

and has been used and improved by successive workers including the

author. A detailed description appears in the theses of Kubba (25)

and Prasada (33). Consequently matters such as synchronisation

standards had been settled earlier and were not easily changeable,

as the work of several colleagues in the laboratory would have been

affected. Thus, points vital to some projects, such as coherence

of the synchronisation signals with the clock pulses, were quite irrelevant to the author's work. The above requirement for coherence resulting in a frame frequency which was "unlocked" from the mains frequency, was a source of constant difficulty, producing interaction of mains hum with the synchronising signals; objectionable hum bar patters on monitor displays were thus difficult to avoid. 101.

4.4.2 The analogue to digital converter

This equipment was constructed by a colleague, D. Brown, and

forms the subject of an M.Sc. thesis, (Brown, 9). Use is made

throughout of high-speed current-switching transistor circuits, the

signal level being compared sequentially with reference currents

in a series of identical cascaded stages. Each stage provides a

binary indication of the result of the comparison, these indications

constituting the output digits of the device. A parallel readout

of all binary output digits is achieved within one Nyquist interval,

(167 nsec)1 the signal level being sampled and held for this dura-

tion to permit the sequential logic process. A binary reflected

Gray code (Hurley,20) with a linear quantisation law is produced.

The equipment when it became available to the author, was

not sufficiently developed for reliable laboratory use. In particular, problems were encountered with the five 10V supplies of the converter; failure of any one supply due to some malfunction led on several occasions to calamitous failure of the whole equipment, at one stage destroying some twelve of nearly seventy transistors and diodes. It was therefore imperative to construct a reliable protection system. This entailed the conversion of all five series regulated power units to shunt regulation. An interprotection system was then devised such that a sudden short circuit in any one supply causes effective short circuiting of all others by intentionally saturating the shunt regulating transistors. 102.

4.4.3 The digram encoder

The digram encoder was constructed by the author and a functional block diagram is shown in Fig. 4.5. Oscilloscope traces of various relevant waveforms are shown in Fig. 4.6. The device accepts either a continuous input signal or a binary sampled signal.

In the former case the video input signal is shaped in a two level quantising or slicing circuit to remove any unwanted fluctuations and sampled with shaped clock pulses. The equipment as shown in the diagram is arranged for a continuous signal input; if a sampled signal is directly available, as from the analogue to digital converter, the sampler is by-passed, the signal samples being applied directly to the monostable circuit. The latter stretches the sharp signal samples to about 100 nsec. in order to facilitate subsequent gating operations. At this stage a delay of one sampling interval, 167 nsec., is inserted by means of a coaxial delay line, (Hackethal type HH 1500), thus producing a delayed signal b, in addition to the undelayed signal, a. As both signals a and b are binary, four possible conditions exist, being indicated by four gates. These perform the gating logic functions a AND b, a AND NOT b, b AND NOT a, and NOT a AND NOT b, corresponding to the Boolean operations (Hurley,20) ab, ab, ;.-b, and ab. These logic outputs thus correspond to the digrams 11, 10, 01 and 00 respectively.

The outputs of the four gates contain stray switching transients resulting from the gating operations and thus require resampling

103.

0 0 0 0 Id 41 0 9 o

Id D .6 c9 z

0 1 z 71, a It 2 a

0 0 0

; 1- _a z et Z 4- o N. 41 o < S g u _a Z Z 5 A 6 el 7 it L. d d < it W

0 o a

0 z

FIG. 4.5. Block Diagram of Digram Encoder. 104.

144,41444,4,,,a4„:. Input Pattern 1 LL: 11 Digram Output

Input Pattern

• 01 Digram Output

Input Pattern

.mailismos.romomismowa ' 10 Digram Output

Time Scale : Closest spacing between pulses = one sampling interval = 167 nsec.

:Input Pattern

11 Digram Output

Input Pattern '01 Digram Output

Input Pattern

:10 Digram Output

FIG. 4.6. Output Waveforms from Digram Encoder for Two Input Patterns. 105.

with pulses properly phased with respect to the clock pulses. The

resampling pulses are required at the digram readout rate, or half

the clock rate, and are derived from a binary bistable circuit which

is itself switched by a clock pulse input and which provides a gating

waveform to suppress alternate delayed clock pulses. The sharp

output pulses of the four resampling gates form the final digram

outputs. Circuits and design details are given in the Appendix

to section 4.4. The operation of the digram encoder is illustrated

by the waveforms shown in Fig. 4.6, The 11, 01 and 10 digram output

pulses are displayed for the same test signal pattern at the input

to the encoder.

4.5 RESULTS

The results of the measurements described above are presented in Figs. 4.7 to 4.9. Several relevant quantities are included, and some comments about these will now be made.

The total second order entropy estimate is computed by summing the individual entropies H(X/2) for each binary digit over all five converter digits. Denoting this by H2, we thus have

digit 5 H H(X2) 2 digit 1

The total percentage redundancy, R2, is then given by io6.

.oio• . - -TEST CARD C.

51•9 ...6 RoOFT0P VIEW

CiD 62.. 6 --Co 4-5 Plc .2.- Pc FerR 'NIT 2.

064- 5 8•7 P114..4- Ro CK75

0 65.5 9.8 P to. 3 - AsRo

T —0 43 P 1o. 1 - P.e)R-rizr‘i-r

®797 r-4 Nut-% ti-F —0 540 B [-ACK AP

c1.4- P t-Ac.K e...4 \NA rrs RCU 1-r

0 Io 4o So go 7c. 6.0 100 Re-ciu NDAN c-Y , oh TOTAL 5ScoNr) oIZpER — 1_11,4s. FIRST czWoE.R. — 5.01-To rsdi Li

FIG. 4.7. Total Redundancy of Various Picture Material. 107.

-4-

1 NoiSM 1 -- Numbers Refer to Test Pictures.

O• 2. 3 4 5 NuMBER of 131NikRsf CoriveRSIoN1 DiGrrs.

PIG. 4.8. Total Second. Order Entropy Estimate, H(X12). 108.

I 0 t's 1.4.01 5 0.9

• Numbers Refer to Test Pictures. 0.

CL 0• Iv / ci.)" Cc0

,,-,,, •i /. id -- , 4 5•'. a 0• 0 3.------__

Z 2" ,/' 0 0.1 ---1/--- - U Ul

DIGIT DIGIT DIGIT DIGIT D IG 11- 2— 3 4 5

FIG. 4.9. Second Order Per-digit Entropy Estimate. 109.

5 - H2 ) R .100 2 5

and is presented in Fig. 4.7 for the test pictures used. The first

order redundancy, R1, is presented in the same figure for comparison.

Of considerable interest is the manner in which the total

entropy or redundancy vary with the number of conversion digits,

(or equivalently signal quantisation levels), used. A graph of second order entropy, given by the sum of the first n individual digit n digit entropies H(X/2), i.e. H(X/2), is shown in Fig. digit 1 4.8 plotted against the number of conversion digits used, n.

Also shown is the plot of second order entropy, 11(X/2) for each particular binary digit, Fig. 4.9. This could be termed the per-digit entropy.

A typical data sheet involved in a set of measurements on picture 4, (the rockets), may serve to clarify both the experimental procedure and the computations involved. Several points need elaboration; firstly, the probability of the 0 digit at the output of the analogue to digital converter was taken as the complement of the probability of the 1 digit, the total number of digits per 6 second being equal to 6.10 . Secondly, the unequal numbers of the transition digrams 01 and 10 is obviously a result of experimental error, the likely cause being jitter between sampling pulses and the video waveform, due to stray effects such as noise and hum. This is an ever present problem in all time quantised or sampled 110. systems, some uncertainty about the discrete position in time of an edge existing in the sampled version. A first measure of experimental accuracy was obtained by com— puting the r.m.s. deviation of the sum of the four digram pulse counts in any one measurement from the true value of 3.106 in one second. This was found to be 9.1 . 103 p.p.s. for the data sheet shown. The overall r.m.s. deviation for all seven half-tone pictures used in the experiment, comprising 35 samples, was found to be 11.9 103 p.p.s. or 0.4% of the true value.

The results of the more indicative noise check are shown in Fig. 4.7, in terms of total percentage redundancy. The finite amount, 5.1%, of first order redundancy is purely due to the slightly asymmetrical distribution of the noise source, For a truly random noise source in which there is no second order statistical dependence the difference between the first and second order redundancy should be zero. It is seen that this difference in the actual experiment was 1.8%. The causes of this finite excess are contributed by two possible factors; firstly the actual inaccuracy of the experiment, and secondly, the divergence of the noise source characteristics from the ideal, this being difficult to ascertain. However, it may be concluded with some confidence that the experimental results are accurate to better than 2%.

V . to tA 0 r 0 03 r M I' Q.., . ro t o u o o o

cc tn tr) r r — In t 02 W 1 0 ko r r4 t-- er- 0 0 tO • cl rn r4 - 10 _ z ___ _ , ______, _ til- _ 1,44 „sri r ...oD t.0t- o 0 i4) ± t71) r- k) '6 ... - r" k.o rn v 0 + tp ,..1 r4.: z - o r^• LA• ro ti.l U1. 00 1/4'51 4. 0 ' • < — 0 0 f 6 a o 0 — O 0 o 0 d t Ni in \ 0 M W 0 p r4 0 r- o-- 9 '1/4.1) i5 °I 0 t.ri y 90 r 0 M 0 ? ri 0 ,- 0 N ‘9 0 n3 • N - La — 6 O o 6 O o 6 a o o

L7 co 2 `E. ci t cl-, S 00_ : o.p __ 0 - 9 o r . t 9 0 1.. 0 - I- \9 0 1,1 IP 3 .., te o h 6. o g U- 0 0 : o 6 - 07„v ,, r1 ' 0 I v 0 (1- 0 - t E el 17, 171 (3 if 00 ,4. 0 to 5 to 0 t9 J 1 O. 0 a ,1 ? t r _ r T . .D, . . . . 0 0 0 0 o 0 - o 0 "'"2i" A a , Z ot I 0 x rn I- in x rIl o a` x tn x r 0 01 tr cl. ''a" x rsi 0 CP — vo9 o r4 0 o ci D 0 r 0 r 0 tr* 0 Co 1: 0 0 o o 4 ° 0 0 0 vi 0 q) 0 o 9

2 00 t , tr r trt a- 00 in r--„, cP 7D- 17\ DO U V — 10 10 trfl (4 \.;) c°,,- V) r.11 1.r. ) ki ..o rfrn 0 ri; • T • , 0 . , 0 N. In a o 0 rn 0 0 -c71 0 0 M 0 0 - 0 o Y. , 0- — LA )1. 7 $ 'INI 7 1.1 %(N1 v) 0 xt to i) 01 7L. NI (‘.1 t_ „ki al %II p4) ri re, 11i,0' tr. 0 0 t- • -' o ' co ..sr tO o o N O a M 0 p ;71 D b $ r:0 ri)0

a. a. d6 td- - 0_ tin- - pm. 07WI" CL a- w• , p_ V`7 ' 0 2, CD n 0 2 el _1 0 CII o v 1d ..1 - D D 0 -.S1,r , . ' c8 r 5I 3 40 D o ce D 0 ce a• %.1 g D 0 0 -2a p_ co) _ 0- u Oaf. 0- U 0- a_ 0- u 0- a- o- a.

FIG. 410. Experimental Data Sheet for Picture No. 4. 112.

4.6 CONCLUSIONS AND DISCUSSION A remarkable feature of the results is the uniformity of the second order entropy and redundancy for seemingly very different

picture material. From Fig. 4,7 it is seen that four out of the six half-tone pictures measured have a total second order redundancy in the small range of 62 to 69%. This uniformity is also evident in Fig. 4.8. All real-life pictures, as opposed to synthetic material, were found to have a redundancy in excess of 50%. This consistency for quite diverse picture subjects augurs well for compression schemes exploiting statistical redundancy, since this also implies consistency of achievable compression. The number of quantisation levels used, 32, may seem to be rather small and some justification is called for, The 40 db peak signal to r.m.s. noise ratio available corresponds to a ratio 8/c', (see Appendix to section 2.6.), of 3. This results in a probability of error in the presence of additive Gaussian noise of 0.134. With 64 quantisation levels, 8/o. becomes 1.5, and the resulting error probability 0.464. Thus, it seems that if 64 levels were used the entropy estimate for the finest digit, (the 6th),

would to a large extent have been contributed by the noise. Inter- pretation of results would seem in such a case to be a matter of some ambiguity? It is therefore felt that the use of more than 32 levels in the measurement of signal statistics is not logically

justified for picture signals in which the noise level is not sig-

nificantly less than in normal television studio standards (40 db). 113.

The results of the experiments seem to be in general agreement with those obtained by other workers. The measurements of both

Kretzmer and Schreiber indicate an average second order redundancy of approximately 56% for varied picture material quantised into 64 levels. The results described show an average redundancy for real- life pictures of 62.490 with 32 quantisation levels. For purposes of closer comparison the 6th digit entropy contribution may be estimated from Fig. 4.8 by extrapolation to amount to about 0.8 bits, in which case an estimated 55% redundancy is indicated for 64 levels. The special case of black and white pictures is of particular interest as it forms the subject of subsequent chapters: The redundancy is large, as might be expected from the simple nature of the material, the average for the two pictures shown in Figs. 4.4 and 5.3 being 80.4%. The inherent simplicity of the binary signal and the large amount of redundancy therein, create a situation which seems very amenable to exploitation by means of some relatively simple encoding. The search for such a code is pursued in Chapter 5.

An original feature of the experiment are the results obtained for intermediate stages of quantisation. This has great relevance in some applications such as the transmission of pictures for recognition purposes only, where quality is of no great importance.

Picture material might here typically include portraits in the case of police work or signatures and documents in connection with bank accounting and office work. For these applications quantisation into only eight or even four levels may be sufficient. For these two cases average second order redundancies of 78.3% and 84.5%,

respectively, are indicated. These values are sufficiently large

to render exploitation of the redundancy by encoding very attractive

It is considered that for applications such as mentioned above, in

which coarse quantisation is allowable, the run length coding system

developed at Imperial College, (see section 2.1), is highly suitable.

It is thought that the results of the measurements described

may be associated with the results obtained for the run length

coding compression system; measurements on the latter system for

picture material of the type shown in pictures 1 to 4: have in-

dicated a mean redundancy of 66 to 75% for acceptable subjective

rendering. The second order statistical redundancy as measured in

the present digram experiments, for the same four subjects, was seen

to be in the range 62 to 69%. The two results, obtained by quite

independent and very different means are sufficiently close to

warrant some thought. A hypothesis may be ventured, therefore,

that the results of measurements in the two cases are mutually

predictable to within 10%. In addition, Schreiberts results show

that the third order redundancy is typically only 6% greater than the second order redundancy. The combination of these results

seems to imply that the run length coding compression system is a

near-ideal code implementation inasmuch as it exploits all the redundancy revealed in separate statistical experiments. It should be remembered that even the statistical measurements described must imply perceptual fidelity criteria as a result of quantisation. Hence 115. comparisons and conclusions of the type discussed above are thought allowable.

In the light of the above, the compression achievable by the run length coding system for pictures quantised into four and eight levels may be predicted to be about seven and five respectively.

As a final remark, it is felt that the experimental technique used offers the greatest versatility of any proposed hitherto.

The experiment described constitutes only the first stage of a general method of successive digram encoding, as described in section 4.2. Further stages would offer no practical difficulty and would lead to the measurement of entropy estimates hitherto unattained. The measurement of tetragram statistics, which is the next step, would require a second digram encoder to operate on the outputs of the first encoder, the measurement technique remaining basically unchanged. It is thought that the technique is very amenable to implementation by computer, use of which would allow the measurement of considerably higher order statistics of various sources, not necessarily visual. FIG. 4.11. Experimental Layout. Centre rack: the analogue to digital converter. Right-hand rack: the digram encoder. 116.

CHAPTER 5 INVESTIGATIONS INTO DIGRAM CODES

5.1 ENCODING OF BLACK AND WHITE SIGNALS Hitherto, the case of half-tone television signals has been considered. It is now proposed to limit the field of discussion to black and white facsimile applications and to develop the results of the last chapter in this connection using Coding Theory considerations.

Primarily the work which follows has still as its main goal the reduction of bandwidth necessary for transmission, in the sense defined in section 2.1. The preoccupation with bandwidth compression arises from cost considerations. Facsimile material is normally transmitted using the existing line facilities provided by national and inter- national network organisations; the actual channel may take the form of open wire, coaxial cable, radio or microwave links. The hire of such channels is sufficiently expensive to make the matter of efficient utilisation during hired time, of extreme importance. The choice of line grade for hire is limited to four or five categories, graded accorang to bandwidth. The voice frequency band of 300 cps to 3Kc is usually taken as a standard, in terms of which the channel grades normally available have bands of 1/12, 1, 12, 60 and 600 times the voice frequency band, the latter three being the so-called group, supergroup and mastergroup telephone services. In this context of necessarily fixed bandwidth facilities it may) at first sight) seem 117. irrelevant to consider reduction of signal bandwidth by arbitrary amounts different from the ratios of the bandwidths of the existing facilities. However, because of the nature of the facsimile signal, despite these constraints signal bandwidth compression is desirable for two reasons. Firstly, more signals may be transmitted over the same channel and, secondly, the time required for transmission of a single message may be reduced. The latter possibility applies directly to data and facsimile signals whose parameters may be controlled at will since the picture or data elements may be read out at any required speed, the bandwidth being thus proportionally ad- justable. By this token, a signal amenable to bandwidth compression is equivalently amenable to reduction in required time of transmission.

Transmission methods may be generally classified as either wholly analogue, where signal parameters are continuous in time, or wholly digital, where signal parameters are discrete and quantised in time. Existing facsimile systems, even those intended for nominally black and white material, are predominantly analogue; with the increasing use of data transmission, however, a strong trend towards digital picture systems is developing. The work which follows will be limited exclusively to the latter type.

Black and white picture signals may be regarded as visual data, and as such form a sub-class of the general class of digital data signals. As mentioned earlier in chapter 1, the classification arises from very significant differences in the properties of the two types 118.

of signal. Digital data, such as encountered in computers, has the

rather unique and unfortunate property in the field of communications,

in that its statistics have to be assumed as unknown and the signal

must be regarded as "useful noise". This is hardly ever the case

with picture signals because of the two-dimensional nature of the

picture, generally resulting in significant spatial correlation

which can usually be exploited by the human observer. Thus a

facsimile page of numerals containing some errors after reception,

may still be correctly interpreted by the viewer, who, in essence,

performs a pattern recognition process based on his store of known

two dimensional impressions. On the other hand, the same data may

be transmitted using a computer binary code, in which case any error

cannot be correctly decoded without auxiliary error correcting

facilities. Thus, though black and white picture signals may be

regarded as binary visual data, it must be remembered that their

ultimate decodability is a perceptual subjective matter.

Clearly, therefore, any successful signal compression scheme

which is applicable to digital data is equally applicable to picture

material, though the reverse need not be true. The two cases must

satisfy different requirements; a digital data system must be

equally usable for all signal conditions, whilst a picture trans-

mission system need only be subjectively acceptable in most con-

ditions. In other words, compression systems for digital data are

doomed to achieve compression of some signal parameter, such as bandwidth, only as a direct trade-off against some other, such as signal 119

power, in a deterministic and reversible fashion. Any system

proposed for picture signals is subject basically to certain fidelity

criteria, difficult to define quantitatively, but which eliminate the

need for strict determinacy.

A reversible bandwidth compression procedure which is theore-

tically viable is the technique of higher extension encoding. This

consists of encoding an M level signal into a higher N-th extension,

as discussed in section 4.2. The digits of the coded signal occur

at a rate 1/N the original and hence have to specify a total number,

MN, of possible signal combinations or states. Thus a bandwidth

reduction of N times is theoretically achieved, the method being

obviously unworkable in practice beyond amall.Values of M and N. If,

for instance M = 64 and N = 2, corresponding to the digram encoding

of a 64 level signal, the resulting number of signal states is 642, which is in practice unthinkable; for such a signal, in the presence

of additive Gaussian noise and otherwise ideal conditions, a peak

signal to noise ratio of 70 db is required to achieve a probability

of error of 0.5, and 86 db for a probability of error of 0.01. Such

a case was, however, seriously proposed by Billings (6 ). The

procedure does seem feasible for the case of a binary signal and a limited code extension of up to the 4-th, say, resulting in 16 signal states; given a distortionless channel and a noise resisting modulation method such as phse modulation, this application has practical possibilities.

A bandwidth compression scheme claimed to be applicable to data 120. transmission has been proposed by Lender (27) and called the duobinary technique. Recognising the random nature of the data signal, intentional correlation is introduced by the following coding procedure. The original binary signal, consisting of the levels zero and one, is encoded into a ternary code such that a zero remains unchanged; the digit one, however, is encoded into either plus one or minus one depending on whether the number of consecutive preceding zeroes is even or odd. If even, the polarity of the one remains unchanged; if odd, polarity is reversed. Thus, the chequerboard sequence 10101, is encoded into +1 0 -1 0 +1, and a sequence 11011, say, into +1 +1 0 -1 -1. It is seen that the code is basically taillary since all three signal states, plus one, zero and minus one are allowed; however, certain combinations such as plus one followed by minus one and vice versa are impossible, hence the name duobinary„ implying a quasi binary alphabet. Nevertheless, Lender's firm claim that the signal is not ternarys is, in the author's opinion, quite unjustified in the context of the accepted definition of signal alphabet as the number of signal states. In terms of bandwidth, it is seen that if a shaping filter having a specific impulse response, such as a raised cosine (Fig. 5.1.c), is used for a chequerboard input, then the period of the resultant sinusoidal output waveform is twice the digit interval, 2T say, for the original signal, and 4T for the coded signal. On this basis a bandwidth compression by a factor of two is claimed. However, this does not hold for all 121.

Fir sooRcs. SIGNAL_

DIC0DED 1\it\. 1\1/ I Si .1',1A1.-

DUoSIN ARY t SIGN As- LaA

Sou Rc_E.. SIGN A

P1 GRAM oil , I .1 I p , 1.1 1 p 0,1 0.1 , oro I I I I a RE-ADOUT 0,0 1,1 . 4 1.1 , 4 11 o 4 a cooE. E, SYMBOLS

CODE, 11, SIGN AL

/AO APTE-D 11 0.1 t i t l 0.1 I 1 . t o 0 1 1a o 1 1 It Co DE. 13 A A A

,ADAPI-eD Cope_ SIGN At_ (b)

o --AT o AT -.T -LIT o 4.1' A 3p -r 0 1- 2.44-r 2. z. 2. RAISED CoS (Iva RE..54PoNSE_ 1 R E_SPor-45e. To (C) ‘MPt.n..SE... RES% tmsE_ Racl- . ruLSE. STEP

FIG. 5.1. Raised Cosine Filter Responses for Different Codes. 122.

signal conditions, such as the sequence 1001001, which is encoded

without dhange according to the duobinary code book, and which cer-

tainly cannot strictly be passed through a half-bandwidth filter

without deterioration. This deterioration arises from the fact

that the filter time response is too slow to allow the output to

reach the level one, thus creating intermediate output signal states

known as partial responses(Kretzmer, 22 ). It may, of course, be

argued that the quoted sequence is very rare; it is nevertheless

a permissible combination of data.

It is therefore felt that Lender's claim of a bandwidth

compression by a factor of two is not wholly justified and that

the system cannot be applied to digital data without some further

assumptions. Recognising this fact, it is considered that the duo-

binary technique is a highly ingenious method which may be applied

with advantage to some specific cases, such as possibly telegraphy

and picture transmission.

The pursuit of signal bandwidth compression should not wholly

divert attention from what may be termed ancillary properties

associated with a code, the most significant being power saving,

ease of synchronisation and elimination of a d.c. component. The latter property is highly desirable as it allows the use of capacitive

coupling without subsequent d.c. restoration. A merit of the duo-

binary code is this property.

In this context some known encoding procedures for binary 123. signals should be described; although in themselves they do not achieve bandwidth compression, they may serve as comparisons for subsequent code proposals.

A logical and simple procedure which results in power saving is the representation of the black and white signal by a binary code whose active digit specifies the transition from black to white or vice versa. Decoding at the receiver is very simply achieved by using the coded transition digits to switch a bistable circuit.

Thus, with a probability of either polarity transitions of typically

0.01, a mean power saving by a factor of about 50 is so attained, compared to an original signal with equiprobable digits. In some applications, such as transmission from spacecraft, such a saving is very desirable. However: any errors in transmission anomalously reverse the polarity of the decoded sequence, the effect being cumulative.

The next logical step in encoding eliminates this difficulty by transmitting information about the actual polarity of the transitions using a ternary code consisting of plus one, zero and minus one signal levels. This technique, referred to in some sources as dicoding, (Bell, 4 ), is illustrated in Fig. 5.1(a) and has the added advantage of eliminating the d.c. component, the number of positive and negative transitions being equal. It is to be noted that neither the transition coded or the dicoded sequences can be passed through a reduced bandwidth filter, as indicated in Fig. 5.1(a).

Following the results of the last chapter, it is proposed to 124. study the feasibility of achieving an efficient code for black and white picture signals based on the digram representation of the signal with a view to bandwidth compression.

5.2 CODING THEORY IMPLICATIONS

In chapter 4 the second order entropy and redundancy of typical

picture signals were measured. In this section the significance of

these results in the light of coding theory will be investigated with the object of synthesizing an efficient code, In this connection it is first necessary to state Shannon's Noiseless Coding

Theorem (42), as follows: Consider a source X having an entropy H(X); given statistical

dependence between symbols of the source X, H(X) must be considered the entropy of X computed on the basis of very long blocks of symbols, in the sense described in section 4.2. Let each symbol xk of X be

encoded into a code word consisting of a sequence of r-ary digits,

(belonging to an alphabet r), the length of the word being 1(xk)r-ary

digits. Let the average code word length be L, defined by

L = p(x).1(x). X

Shannon's theorem then states that the minimum average code

word length for any uniquely decodable code, cannot be less than the

entropy of the source X expressed in units to the base r. 125.

Thus: L Hr(X) where Hr(X) = p(x).logrp(x).

With sufficiently involved encoding L can be made to approach Hr(X).

For a full treatise the reader is referred to Abramson (2). In the situation under consideration a second order entropy for black and white picture signals of about 0.2 bits has been revealed. higher order estimates being expected to be somewhat, though not significantly, smaller as discussed in chapter 4. In the light of Shannon's theorem, this means that it should be possible, with sufficiently refined encoding to construct a binary code whose average code word length would be 0.2 binary digits. In other words, the message duration may ideally be reduced to a fraction 0.2 of the original. Such a theoretically ideal encoding procedure would consist of deriving some sufficiently high extension of the n source, say of order no and encoding each of the 2 symbols into binary code words using a statistically weighted procedure. Given n were large enough the resultant reduction in the duration of a given message would then approach the fraction 0.2. However, there exist no means of predicting the order of extension, n, required. The object here is to devise a useful coding procedure based on the second extension or digram rtpresentation of the source signal. The two black and white picture signals considered in chapter 4 126.

had the following sets of digram probabilities:

[0.015 0.015, 0.641, 0.329] and [0.019, 0.019, 0,672, 0.282]

the entropies, H(X2), associated with these sets are 1.117 and 1.12

bits per digram digit respectively. The value of 1.12 will be used as typical. This result implies that if the original binary signal were represented by its digrams, then further encoding of the digrams into binary code words would, if the encoding were ideal, result in a message duration reduction of 1.12/2 = 0.56. The factor of two arises from the fact that one digram digit occupies two original digit intervals. Using Huffman's minimum redundancy encoding

procedure, (see Abramson, 2 ; Reza, 34)$ which ensures a minimum length code given the set of symbol probabilities, the digrams may be coded as follows:

the 00 digram into code word 0 the 11 digram into code word 10

the 01 digram into code word 110 the 10 digram into code word 111 resulting in average code word lengths of 1.35 and 1.39 digits for the two pictures, as compared to 2 digits for the digram source without further encoding. Thus, using the above code an immediate reduction in message duration to a fraction 0.7 may be obtained, compared to the ideal 0.56.

The ultimate aim, however, is signal bandwidth compression.

As pointed out in chapter 1, reduction in message duration is 127.

reciprocally equivalent to compression of bandwidth only in theory;

whether the latter is achievable given the feasibility of the former

remains basically a question of practical ingenuity. Thus, in

terms of bandwidth compression, it is not certain that even the

fractional saving of 0.7 could be attained without considerable

involvement in queuing stores. Thus encoding of the digram set

into binary code does not seem to be a very fruitful procedure.

At this stage the reader may wonder why the signal should not

be simply specified by its digram representation resulting in a

four symbol code alphabet and an immediate bandwidth reduction by

a factor of two. This seemingly obvious course has several dis-

advantages both of a theoretical and practical nature. Firstly, such a code would be very inefficient since a signal capable of

specifying two bits of information per digit would be used to convey

only 1.12 bits on average. This feature is obviously unsatisfactory in the present context of efficient encoding. Secondly, in practice a four level signal is rather undesirable compared to either a binary or lirnary signal; in the latter type particularly, only two active signal states are present making for great simplicity of decoding and good immunity to noise and distortion. Thus the use of a code with an alphabet smaller than four is very desirable.

The next logical avenue of investigation, therefore, is the encoding of the digrams into ternary code. For this case the entropy to the base 3 is required, i.e. the quantity H3(X2), which is simply

H (X2)/log 2 23, and is equal to 1.12/1.585 = 0.71ternary information 128.

units. Thus, from Shannon's theorem, the minimum achievable average

code word length using a ternary code is 0.71 ternary digits, or the maximum possible reduction in message duration is 0.71/2 = 0.355 of the original. An actual Huffman encoding procedure for this case results in the following code:

the 00 digram is encoded into code word 0

the 11 digram is encoded into code word +1 the 01 digram is encoded into code word -1 0 the 10 digram is encoded into code word -1 +1

The average code word length for either picture is 1.03 ternary digits instead of the two digits required without statistical encoding of

the digrams. A reduction in message duration of 0.515 can thus be directly achieved by this procedure. This result is encouraging although the problem of converting it into an actual bandwidth compression still remains. The above ternary code, as it stands, requires the same filter characteristic for transmission as the original binary sequence, since the minimum digit spacing remains the same. Hitherto the coding operations described have been completely reversible inasmuch as any one could be applied to any form of binary signal, whether random data or picture material, with unique one to one decodability. (The values for reduction of message duration quoted above apply, of course, only to the particular picture material considered). It is now proposed to take a logical step by resorting to an irreversible 129.

procedure by means of which a bandwidth compression is attained, and

which exploits the perceptual aspect of visual communication. The

procedure simply consists of restricting the set of four digrams to

an alphabet of only three symbols resulting in a new restricted

digram representation of the original signal. The restriction may

be imposed by representing any two digram symbols by a single symbol

of the new ternary alphabet. Thus the restricted digram source may

be coded into a ternary code as follows:

the 00 digram into code word 0

the 11 digram into code word +1

the digrams 01 or 10 into code word -.1

There is thus a one to one correspondence between symbols of the

restricted source and digits of the ternary code. The ternary code

digits occur regularly at the digram rate and can thus be passed

through a half-bandwidth filter, thus constituting a bandwidth reduction by a factor of two, However, it is clear that this

procedure introduces ambiguity, since, on reception of the code symbol -1, it is not known whether an 01 or a 10 digram had been sent. The above procedure of source restriction may therefore seem rather extreme, and it is thus the object of the next section to produce justification, both argumentative and experimental, for such an irreversible encoding. 130,,

5.3 EXPERIMENTS WITH RESTRICTION OF THE DIGRAM ALPHABET As a consequence of the above considerations it was decided to perform experiments to investigate the effect of restriction of the digram alphabet as a means of attaining a desirable code. The result of any given reduction of the digram source, though completely predictable in form, was, however, an unknown quantity in terms of visual effect. It was therefore nenessary to perform the decoding operation required at the receiver for any particular restricted alphabet code. This operation was implemented by means of a decoding unit, the function of which warrants some remarks,

Essentially the decoding unit consists of a bistable circuit with provision for switching from the various digram output pulses provided by the digram encoder described in section 4.4.3. The steering logic to the bistable circuit is such as to reproduce the original quantised binary picture sequence at its output from the complete set of digrams at its input. The logic required emerges from the truth table shown in Fig. 5.2(a). Since any digram specifies the signal over two sampling intervals, the switching operations must be considered both at an initial-state time, to, and the time t o + /It, where At is one sampling interval. The situation is thus completely described by indicating whether or not the bistable circuit is required to switch at these two instants.

This is indicated by a YES or NO in the truth table. Inspection of the resulting truth table at the initial time to immediately 131.

13s S -rAlli-E RE qv I RED 1 1•4 ITI AI- INPUT To SW ITCH, YES oR No OUTPUT PIGRAM tyr TIMES STATE . -1. 0 .Lc,i-At o 01 No -(E.s I 01 YE-5 yEs o 1 c. `{ES YES I 1 0 No YES o Oo No No I 00 YES No 0 I I YES Into I I I No No

TRUTH 'T-A6LE. FoR LOGIC DESIGN ot= 11i STPN31.-E.

DEC-0 PtNG Ut.,1

00 5w6 to I /P SW1 Sw3 01 SISTA:81-E. DELAY At CIRCUIT SW2. 5W4- T o VP I Ito $W PI GRAM INPUTS IM PLEMENTATioN oSTaV-RING LOGIC PROM

TRUTH -TAI .LE

FIG. 5.2. Logic Design of Decoding Unit. 132, indicates that the pairs of digrams (01 and 00) and (10 and 11) have identical switching requirements. Thus, using logic notation, (01 OR 00) must form one input to the bistable circuit and (10 OR 11) the other. It is also seen that at a time Lt later, the bistable circuit must be switched again, given the occurrence of either the

01 or the 10 digrams at time to. This function is implemented by producing the signal (01 OR 10), delaying it by Lt and inserting it as an input signal common to both inputs of the bistable. as in a binary counter connection. The complete steering logic required is shown in Fig. 5.2(b), the actual circuits involved being shown in the Appendix to this section (Fig. A.5.3). Various desired restrictions of the digram set can be introduced by switching appropriate digrams out of circuit by means of toggle switches shown in Fig, 5.2(b), or by rearranging the steering logic connections. Several such experiments will now be described. The results are presented in the form of monitor displays of the outputs of the decoding unit for various restrictions. These should be compared with the picture obtained by decoding from a complete set of digrams, Fig. 543(b). The original slide is shown in Fig. 5.3(a) and the output of the signal shaping circuits, (the output of the monostable circuit of Fig. 4.5), is shown in Fig. 5.4a. The latter is the result of amplitude and time quantisation, and shows noticeable quantisation effects. It should therefore be stressed that these phenomena, which are consequently visible in the digram 133.

—12V

FLIP-Ft-OP W rrm TRIGGER AMPLIFIER

x x 2..5x TNYQ

(a) Original

(b) Decoded from. Complete Set of Ligrams

[FIG. 5,3. I , 'sqoajj7 uo-p..esTqu-ent) •'r • DMj

°Gan4oTa et-PI-Po-M. y (q)

VII1111141%*".

8414 GA VOA 21 "VS

eangod pooT;u2C0 (e)

-pa 135.

processed picture, are not connected at all with the actual process

of digram encoding and decoding. Hence, to assess the effect of

different codes, any subsequent comparisons should be made with reference to the picture of Fig. 5.3 obtained by complete decoding, and not to the original slide shown in the same figure. The causes

of quantal deterioration are twofold and interdependent; firstly,

the picture signal produced by a flying-spot scanner from nominally black and white material is by no means binary. This will be appreciated by inspection of a typical waveform of such a signal as shown in Fig. 5.7, representing a portion of a line scanned across the word AMPLIFIER of the test slide. The actual slide was pro- ducedby photographing a specially prepared drawing using Micro -Neg

Pan Ilford Film, which had the highest contrast of any commercially available film; hence the photographic quality of the test slide is thought to be the best practically attainable. In spite of the high quality of the original, it was impossible to find a setting of the slicer circuit such that the truly two level signal produced was visually identical in fine detail to the original unquantised signal.

Secondly, the rate used for sampling the signal was the theoretical Nyquist rate. The Nyquist sampling theorem implies that the signal is completely representable by such Nyquist samples weighted with a sin x/x interpolation function. The output of the decoding unit, however, is not of this form, being the result of a sample and hold representation (constant interpolation). In practice, time 136.

quantised facsimile systems normally rely on this form of representa-

tion, since the operations of resampling and sin x/x interpolation

indicated by theory are considerably more difficult to implement.

However, to reduce the consequent time quantisation effects a sampling

rate higher than the Nyquist rate is normally used. In the present

investigation it was decided to retain the theoretical Nyquist

rate in conjunction with constant sample-to-sample interpolation

in order to avoid introducing an indeterminate parameter into the

experiments as would have been the case with the choice of any

other sampling rate. Furthermore, the imperfections of any par-

ticular code are the more evident the slower the sampling rate..

The question of the sampling rate required in relation to the re-

solution of the picture material for adequate subjective rendering

has, to the author's knowledge, received no direct research.

In the original drawing the width of vertical lines was carefully controlled so as not to exceed the resolution indicated by the sampling rate. Care was thus taken to ensure that the smallest width of line corresponded to not less than one Nyquist sample in the signal after quantisation and sampling.

An actual facsimile reproduction of similar picture material,

(Fig. 5.4), might prove instructive._ The copy shown was obtained by means of a commernially available facsimile system using continuous amplitude modulation. The transmitter and receiver were at the same location and hence the transmission suffered neither 137.

from distortion nor channel noise. The resolution for the picture

shown was double that indicated by Nyquist rate considerations. In spite of this, it is seen that the copy is by no means perfect, though quite adequate. In passing, attention should be drawn to the thickening of the vertical lines in the extreme left of the monitor displays, This is due to an unfortunate non-linearity in the line scan of the

picture display.

5.3.1 Experiment A

In this experiment the transition digrams 01 and 10 were switched out of circuit by means of switches SW1 and SW2 shown in Fig, 5.2(b). In this way, no separate symbols were provided in the code alphabet for the transition digrams, the occurrence of a 10 being interpreted as a 11 and that of an 01 being interpreted as an 00. This situation is therefore synonymous to the restriction of the digram source to binary form consisting of the symbols (00 +

01) and (11 + 10) in the logic sense. The result of this restriction is shown by the monittr display of Fig. 5.5(a). Compared to Fig. 5.3(b) deterioration is rather evident. Patterns containing gaps one sampling interval wide (minimal notches), such as vertical lines of maximum resolution, cannot be specified by the restricted code alphabet and hence such minimal transitions are completely missed. In

138.

(a)Code A.

27 470 470

4N 705 FLIP- FLOP w T RIGGER Auv1Pt.triER eilzmairm I I I 111111 1Z

(b)Code B.

FIG. 5.5.1 Decoded Pictures. 139, addition to the above "miss" errors which are the more serious, "shift" errors are also possible, Thus, those black to white steps which are specified by an 01 digram are displaced to the right by one sampling interval, (about 0.02 inches on a 17 inch monitor screen); white to black steps specified by a 10 digram are also shifted to the right. A less obvious "miss" error is possible even with a vertical line width of two sampling intervals, if the line happens to be specified by the digrams 10 followed by an 01. This error is evident in Fig. 5.5(a) in the collector end base connections of the right hand transistor. The general visual effect thus shows an intolerable degradation.

This is not totally unexpected considering the extreme restriction imposed on the alphabet,

5.3,2 Experiment B Here the transition digrams were combined to form one symbol,

(01 + 10) in logic notation, as described in section 5.2. This was achieved by opening switches SW3 and SW4 in Fig. 5.2(b), thus leaving only the common (01 + 10) symbol which will be referred to hence-• forth as the "delta" symbol. The restricted code alphabet is thus ternary consisting of the symbols 11, 00 and (01 + 10), In this situation the occurrence of a delta symbol signifies a command for the decoding circuit to change state after one sampling 140. interval. Thus steps from white to black or vice versa are decoded correctly. The only type of error occurs whenever a minimal black- white -black notch is specified by a 10 digram or a white-black-white notch is specified by an 01 digram. As the contingency of either event is random, only 50% of such patterns result in errors. The form of the error is again a shift of the notch to the right by one sampling interval. The actual visual effect is shown in Fig. 5.5(b). It is difficult to note any deterioration in the picture except in regions containing the finest lines such as the legend FLIP-FLOP. Although it was expected that degradation would be quite small, its nearly complete absence was found quite surprising6

5.3.3 Experiment C The experiments which follow are meant to show some interesting features of the digram representation of a black and white picture signal. One such feature which may be rather unexpected is the fact that the signal cannot be completely specified by the transition digrams 01 and 10 alone. This is so due to the fact that any one digram specifies the original signal for two sampling intervals, at the end of which the signal can assume either a zero or a one state. Hence constant interpolation between transition digrams cannot be used. Moreoever, a picture transition need not necessarily be specified by a transition digram since an 00 digram followed by a 11 digram or vice versa also constitutes a transition. The actmft 141. effect of eliminating the 00 and 11 digrams by means of switches SW5 and SW6 is shown in Fig. 5.6(c). It is seen that the resulting picture is completely chaotic because of the errors described above which are, moreoever, cumulative. This feature serves to emphasise the essential difference between the transition digrams and the simple differential of the signal from which the original may be completely reconstituted by constant interpolation.

5.3.4 Experiment D Here the occurrence of the transition digrams 01 and 10 is separately displayed. The occurrence of each digram is indicated on the monitor by a white "dot" one sampling interval wide as shown in Fig. 5.6(a) and (b). This was achieved by appropriate steering of the decoding unit. The exposures were made using a single-shot technique in order to obtain a single complete picture scan of 1/25th of a second. For this purpose an external gating signal derived from the Tektronix 545 oscilloscope was used to control the sampling circuit at the input of the digram encoder (Fig. A.4.4).

On the basis of several such single shot photographs taken at intervals of a few minutes, it was found that the pattern of occurrence of the transition digrams remained significantly stable. This was expected in theory since under perfect conditions the spatial picture pattern always generates the same signal pattern, specified by a particular sequence of digrams. The stability of the display in 142.

FIG. 5.6.(a) Display of 01 Digrarns

FIG. 5.6.(b) Display of 10 Digrams .

FIG. 5.6.(c) Exoeriment C. 143. practice indicates the absence of significant jitter between sampling pulses and the picture roster; such jitter was feared to be likely due to any hum on the scanning circuits which generated the roster.

5.4 A RESTRICTED ALPHABET DIGRAM CODE The results of the experiments described in the last section suggest rather clearly that the code described in experiment B should be chosen as the most acceptable of restricted alphabet digram codes, To reiterate, the code is ternary comprising the set of symbols 00, 11 and (01 + 10) in the logic sense. The amount of visnal deterioration from a subjective viewpoint is very small and may be judgtd by comparison of Fig. 5.3(b) with Fig. 5.5(b).

The code, referred to henceforth as code B, seems therefore to be quite applicable to facsimile transmission of black and white material. Simplicity of implementation at both encoder and decoder is an attractive feature of the scheme.

As the code symbols are read out for transmission at the digram rate, or half the original sampling rate, a bandwidth compression by a factor of two is directly achieved.

For transmission over a channel it is proposed that the code symbols be sent as the following signal levels: the 11 digram as a positive excursion, the 00 digram as zero level and the delta 144.

(or combined transition digram) symbol as a negative excursion.

This is indicated in Fig. 5.1(b) for a typical message sequence; the envelope of the resultant code sequence filtered with a raised cosine filter impulse response being also shown. Some practical filter experiments in connection with the above code B were also performed, their description being left to chapter 6.

Complete Line, 10 psec/div.

(b) Detail : ,word AMPLIFINR, 2 pseo/Aiv.

Video Waveforii4L.InrIftcture of Circuit, Fig. 5.3. Scan Across thOlOrd AMPLIFIER. . 145.

CHAPTER 6 ADAPTIVE ENCODING

6.1 A SPECIFIC EXAMPLE

The feasibility of several codes based on a digram representation

of the signal was investigated in the last chapter, culminating in the

selection of the code B described in section 5.4. The above code,

though perfectly satisfactory for picture signals, brought to notice

several intriguing aspects which, it will be seen, can be formulated

into a general conclusion with, it is thought useful and novel im-

plications. As indicated in section 5.3, code B produces definite errors at the decoder as a result of the ambiguity of its restricted alpha-

bet; hence, although subjectively acceptable, it is unsuited for

applications to general data transmission. However, these errors

which are a result of the shortcomings of the code have one property of great importance - they are completely deterministic. Thus, given a particular source signa3„ the form of the encoder output and hence

the resultant error are uniquely predictable. This immediately tempts the question of whether it would be possible to devise some preventive action at the encoder in order to nullify the error. This was found to be indeed the case, and such a strategy, specifically applicable to code B, will now be described.

The procedure may be specified by the following set of code rules; 146. the occurrence of a delta symbol (section 5.3.2) is always interpreted at the decoder as one particular transition digram, say the 10 digram. The digrams are made available for readout at the sampling clock rate, every other one being transmitted over the channel.

However, whenever an 01 digram occurs which would normally be transmitted with resultant error, the actual readout is delayed by one sampling interval, and the next digram transmitted instead. The absence of a digram readout at the expected time, (two sampling intervals after the last readout), is interpreted by the decoder as zero level. The procedure should be clear from the typical sequence shown in Fig. 5.1(b). The code is thus adapted to eliminate error at the receiver by exploiting the predictability of the situation. The channel signal, as for code B, is still ternary consisting of positive and negative excursions and zero level. In the adapted code, however, the negative excursion always represents a 10 digram since the 01 digram is forbidden by the adaptive coding process. It is to be noted that the code symbols no longer occur at the uniform digram rate, a complication which will be discussed in a later section. The adapted form of code B was implemented in a rather simple manner. The requirement was to shift the digram readout by one sampling interval whenever an 01 digram occurs. The binary bistable circuit of Fig. 4.5 normally gates the clock pulses so as to provide resampling or readout pulses at the uniform digram rate. The 01 147.

digram pulse was thus used to reset the binary circuit prematurely

between normal clock pulses. As a result the next resampling pulse

occurs one sampling interval after the occurrence of the 01 digram„

as required. It was found that this adaptive process operated with

total success. The output from the decoding unit derived from

the adapted code B was displayed on a monitor and was found after

careful inspection to be identical in every detail to the picture

decoded from a complete set of digrams.

6.2 AN ADAPTIVE ENCODING TECHNIQUE

It is now suggested that the specific example of the last section can be usefully generalised and a proposal for such a generalisation will now be outlined.

In general terms, the situation involves an encoder which is imperfect in some sense as a result of some constraint imposed upon it which, however, produces some otherwise desirable property. A decoder which normally is capable of perfect decoding in the absence of the constraint at the encoder is, in these circumstances, forced to produce erroneous outputs. However, given the above imperfect encoder and normal decoder, the form of the resultant error is completely predictable. This is the crux of the argument. It is therefore suggested that in many cases this fact can be used to modify or adapt the code so as to eliminate or at least minimise 11+8. any overall error.

To clarify the proposal a functional block diagram is shown in

Fig. 6.1(a). It should be emphasised that the practical implementation need not necessarily follow this theoretical scheme, the diagram being intended to serve merely as an illustration of the concepts involved. Referring to the figure, the imperfect encoding-decoding operation is simulated directly at the transmitter by providing a local replica of the receiver decoder. Since the encoder, due to its imperfections, produces a non-uniquely decodable code, the local decoded signal Z will differ from the original input signal X. After allowing for delay in the encoding-decoding process, the two sig.» nals are compared locally in a comparator. The comparator thus provides an indication which is essentially a prediction of error at the receiver decoder. This error prediction Y, is then used to modify the code, by means of the code adaptor, prior to transmission over the channel. The receiver decoder is then able to reconstruct the original signal without error on the basis of the adapted code.

Processing of the code for transmission over the channel is not shown. It is seen that the technique is an adaptive one using predictive feedback at the encoder. It should be emphasised that no claim is made as to the feasibility of this technique in any particular situation; it is simply suggested that it exists and may be applicable depending on the particular circumstances of the situation. Thus 149.

INPUT co DE- REceisiER eNcotIR --HDELAY DER SiGNAL. ADAPTeR De CO X

LOCAL DELAY OEcc•DR-R

coMPARA-PoR

FIG. 6.1.(a). Block Diagram of an Adaptive Encoder.

50%..) RcE. SIGN AL. II III lot of II III 10 00 01 II lo 00 co 00 ol 10 of 10 co I III.III• Rut-n..111-4 G D t G RA WI RE AD ovT 10 01 10 01 10 01 10 01 10 ...... I a ...... a I I TRANSI-rioht DIGRANAS

01 10 10 10 ...... 01 10 . . DIGRAN1 S VE.Lec-r..0 FOR c...c:xle- S-r/A15.o-S

coNTRoL SIGNAL.

ADAPTED coDE. SIGNAL. vvrri-t RAISED Las E. e-NVE.Lo E.

FIG. 6.1.(b). The Adaptive Code of section 6.4. 150 the proposal has the unfortunately vague implications as to actual implementation common to most coding theorems. It should also be pointed out that the technique does not achieve a something -for - nothing gain. It simply provides a controlled means of exchange of a desirable property on the credit side, and a small acceptable incurred penalty on the debit side. It is thought that in many cases where complete elimination of error is not feasible, the adaptive technique may still be applicable in minimising or transforming the errors in some sense, such as to render the decoded message acceptable at the final destination, especially where a human destination is involved. In terms of the adapted version of code

B described in section 6.1, it is seen that, although the implementation does not follow the theoretical block diagram of Fig. 6.1, the implied sequence of functional operations is basically the same; an error is predicted and action is taken on the basis of this prediction, in the form of a code adaptation which consists in this case of shifting the digram readout by one sampling interval. The penalty incurred is a resulting rare partial response after filtering, as will be described in section 6.3. The profit is the complete elimination of code ambiguity and the use of a ternary signal alphabet. 151,

6.3 TRANSMISSION CONSIDERATIONS

The achievement of a suitable code representation of the original source message is one stage in the design of a communication system.

During this stage the assumption is made that connection between the transmitter and the receiver is ideal and thus that the decoder is only required to perform a deterministic reversible operation. It is this stage which has been hitherto considered. The information specified by the code symbols must, however, be represented as a signal for transmission over a channel which imposes the constraints of a noisy environment and certain bandwidth and power limitations.

Thus, a reasonably complete discussion should include the question of sending the code over a practical channel.

In the present context it is necessary to determine the resulting bandwidth compression according to the definition given in section 2.1. This entails the specification of some filter characteristic; the choice can be somewhat arbitrary, with convenience of manipulation as the main object, since the question is purely comparative. On these grounds, it was decided that a filter with a raised cosine impulse response, as shown in Fig. 5.1(c), would be postulated as a model for theoretical discussion. The advantage of such a characteristic is the ease of determining the time response of the filter to various excitations, since the impulse response falls to zero within one sampling interval. The responses of such a filter to a rectangular pulse and step excitation are also shown in 152,

Fig. 5.1(c). It is clear that for code B the limiting width of such

a raised cosine impulse response can be made twice the digram interval, or 44 T, where 4T is a Nyquist sampling interval, without causing intersymbol interference, compared to 2AT for the original.

Hence a bandwidth compression by a factor of two is clearly achieved.

For the case of the adapted version of code B, the matter is somewhat

complicated by the non-uniform generation of code symbols as a result

of the adaptive process, Thus, as is seen from Fig. 5.1(b), identi-

cal symbols spaced by 5ZNT sometimes occur. The response of the filter cannot then reach zero level between symbols, resulting in a

partial response equal to 55% of peak signal amplitude, as indicated in the same figure, This is admittedly an inherent disadvantage of the scheme, but it should be pointed out that picture patterns which cause such responses are very rare.

In order to estimate the effect of practical filtering on the coded and original signals it was decided to perform some pilot experiments. Firstly, it was required to synthesise the ternary signal waveform from the code symbols. Instead of exciting the filter with narrow pulses whose characteristics are difficult to control, it was decided, for the purpose of the experiment to use accurately controlled rectangular pulses of width equal to the digram interval and to compensate for this by adjusting the frequency scale of the filter characteristic used, as will be described later, The main reason for adopting this method of simulation is its experimental 153, simplicity and the large measure of control which it provides over stray parameters. With impulse excitation the filter output is considerably smaller and stray pick-up between input and output becomes very serious. Two bistable circuits identical to the decoding unit of Fig. A.5.3. were used to generate the rectangular ternary signal, the arrangement being shown in Fig. 6.2. One circuit generates a positive signal excursion corresponding to a 11 digram, the other a negative excursion for the 10 digram by means of suitable steering of digrams at the inputs. The two outputs are equalised in amplitude and balanced around zero level by means of two equalising amplifiers and combined by means of an adder onto the input of the filter. As rectangular excitation produces a filter time response 1,5 times wider than the impulse response, (see Fig. 5.1(c) ), allowance was made for this fact by expanding the frequency axis of the filter characteristic by the same factor, thus simulating the effect of impulse excitation with reasonable accuracy (within 10% at any ordinate). In passing, it is thought that the above simulation technique may be found to be of use generally in similar filter experiments. The filter used in the experiment was of the constant K type using a It -section with LC elements as shown in Fig. 6.2. As the nominal bandwidth of the signal is 1.5 Mc, the filter cut-off 154.

+6N/ 1-$4.1.5 . —10 3-N706

DRYING SIGIN1.41- II 5•C 1 .1 12.Y 11-S 1.A.H

+ G V

(Pi 'HO) /3.14706 1. 17

-ta.V nPu#111-isING AMPL.1 'PIERS

Derivation of Ternary Driving Signal for Code B.

0 - 4--LI -

A 10 Frequency Response of Filter. 0 C= 80 pF, R= 1.0 k2. X 20 pull kJ,. 1,,4 J F. --a 3o, ar 05 a 3 1- R*.c4 yam Cy, Mc.

FIG. 6.2. Driving Circuits and Filter for Eye Pattern Experiments. 155,- frequency or the frequency at which attenuation starts was made

2.25 Mc, allowing for the expansion factor. The measured fre... quency characteristic of the filter is shown in Fig. 6.2. The results are in the form of oscilloscope traces of the filter output and are presented in Fig. 6,3. The waveforms were obtained by triggering the oscilloscope externally at the basic clock rate thus producing the characteristic figures. Such waveforms are known as eye-patterns and provide a most useful means of assessing the overall transmission performance of a communication system. Gaussian noise derived from a noise generator was used in the experiment as the input signal to the digram encoder; in this way the code symbols could be arranged to occur with nearly equal probabilities. The eye-pattern displays in one presentation all the possible combinations of signal states and transitions and, in general, the more well-defined the "eye", the more easily can the receiver make decisions about the state of the signal in the presence of noise. Fig. 6.3 shows the unfiltered ternary driving signal (top of page) and the eye patterns produced as a result of applying the following excitations to the filter: the ternary code B signal; a binary signal generated at the digram rate and corresponding to code k of section 5,2.1; the binary un-• coded original signal generated at the Nyquist clock rate; the signal of code B with a peak to peak signal to r.m.s. noise ratio of 14 db; and finally the adapted version of code B. For the case

156.

Ternary Driving Signal for Code B. i

. Code B.

Binary Code A.

Unooded Binary Signal.

Code B, S/N= 1/4. db.

Adaptive Version of Code B.

FIG. 6.3. Eye Patterns for Excitations Specified Above. Filter of Fig. 6.2. 100 nsec/division. 157. of code B the filter used seems quite adequate since the "eyes" are well formed even in the presence of noise. In all the waveforms shown, it should be pointed out, the patterns are not perfectly defined due to the slight asymmetry and ringing present in the rectangular driving signal, as shown.

It is clear that the original binary signal cannot be passed successfully through the filter, a fact evident from the corresponding eye pattern in which intermediate signal states are very apparent.

The eye pattern for the adapted version of code B is rather more complex, due to the non-uniform symbol rate, the interval between successive symbols being either twice or three times the

Nyquist sampling interval, Due to this and the fact that the oscilloscope is synchronised by clock pulses at the Nyquist rate, the "oyes" of the resulting pattern are folded back causing a more complicated pattern. However: careful inspection of the pattern shows the basic ternary structure of the signal and the existence of the partial responses mentioned earlier. It should be stressed that these signal conditions were intentionally accentuated for demonstration by the use of noise as a test signal; with a real picture signal the partial response was difficult to detect unless the tube trace intensity was increased to such an extent as to saturate the rest of the pattern.

It is not proposed to consider the question of deriving the 158. code symbols fromthe filtered signal. This topic is a problem common to all digital communication systems and has received attention elsewhere (Bennett, 5 ). The usual procedure is to resample the filtered wave at the receiver, the correct resampling phase being provided at the receiver either from a separate timing wave, or derived from the signal itself by shock excitation of a high -Q tuned circuit. As a final comment, the existence of the partial response when the adapted code is restricted to a truly half bandwidth filter should be viewed in the perspective of modern data transmission practice. In such applications, a non-minimal shaping characteristic is usually used in the sense that a transmission filter of wider bandwidth than theoretically required is used. This obviously simplifies the problem of reception since the shaping characteristic can then be designed for minimum intersymbol interference and optimum detection criteria (Haber1 18 ; Lerner, 28 ). In this context the adapted code would certainly require only half the bandwidth otherwise needed. It is therefore thought that in this light the objection of the existence of a partial response becomes rather one of academic rigour than of practical importance. 159.

6.4 A PROPOSED ADAPTIVE CODE The adaptive technique described earlier can be applied in the design of a digram based code which offers the advantages of bandwidth compression, power saving and zero mean signal level. A complete method of implementation will be proposed although actual implementation will not be attempted. The coding scheme is based on the restriction of the digram alphabet to the set of symbols 01, 10 and (00 + 11) in the logic sense, as described in experiment C0 section 5.3.3. If such a restriction were acceptable the transition digrams, 01 and 10, could be sent as positive and negative signal excursions, and the

(00 + 11) symbol as zero level. Such a code would have the very desirable properties of a power saving inversely proportional to the probability of occurrence of transition digrams and zero mean signal level the probabilities of occurrence of the 01 and 10 digrams being equal. Primarily, of course,a bandwidth compression by a factor of two would also be achieved. However, the results of experiment C indicate that such a code, using uniform readout of transition digrams and constant (zero order) interpolation between successive digrams at the decoder, produces totally anomalous errors causing quite unacceptable picture reproduction. The solution to the problem of achieving this desirable code whilst eliminating or at least minimising the errors inherent in it, may be obtained by adaptive encoding.

The procedure becomes clear once it is realised that any picture 160. sequence can always be uniquely represented by its transition digrams by suitable selection of either one or the other for readout. For this purpose the digrams are made available for selection at every Nyquist sampling instant. The selection of digrams is illustrated in Fig. 6.1(b), where those digrams required for readout are indicated. By inspection a simple prediction rule emerges: given an uninterrupted chequerboard sequence, if the sequence ends in a

11 digram then the 01 digrams are required for selection; if the sequence ends in an 00 digram then the 10 digrams must be selected.

This rule, which is sufficient for error prevention, indicates that the selection depends on the end digit of an uninterrupted sequence of transition digrams, and thus the decision upon which selection is based must involve storage over the duration of the longest uninterrupted chequerboard sequence. Information determining selection of either a 10 or an 01 digram, in the form of a control signal, can be derived from the 00 and 11 digrams. The required control function is essentially the constant interpolation of the 00 and 11 states inversely in time. This may best be explained by reference to the diagrams of Figs. 6.1(b) and 6.4(a) showing the required control signal. It is seen that this signal is obtained by constant (zero order) interpolation between the 00 and the 11 digram states from right to left in the diagram, or inversely in time. This constitutes the predictive element of the scheme commented on earlier, 161.

"71 IATr- t II II II oo I, .0 oo B B S A B A A A souRce. SIGNAL PrND A DIG ITS

I 0 I I 0 0 I I 0 0 0 1 0 0111— Pioer-rs1R1-4 fte.t• IN i•F-r R

B A A A A s-roRE.D P. f4P-4D S p C2RAisikS

BIB BI A B B A A A A B B A A A A A 14- KecRytize.p Cor-4 -r R 0 L. PAcrrE-V41.4 •

(a)- Derivation of Control Signal.

N ST7NGE. Co T-4-1- Kok.- Re.G1 s-re-R

00 13

50.1.-ec•TED 0% 1.4 STAGE C.oNT Ro RE.C• DIGmArAS -+- I To DELAY N. LIT 0 0 I 1 • CHANNEL

DELAY tv.,6,1- 0 10 %ELECTED to P16RANAS 162.

The derivation of the control signal may be carried out in

a shift register store, as shown in Fig. 6.4(a). The shift

so that the pattern stored in it corresponds conveniently to the

normal time sequence as shown. Each "box" of the register re-

presents one storage stage. If the digrams 00 and 11, denoted by

A and B for convenience, are read into the register, they form a

stored pattern as shown diagrammatically; the control signal is

derived from this patteraby interpolating the states A and B from

right to left within the register. The required code is then

completed by using the control signa► to gate the 01 and 10 digrams

(delayed by a fixed duration), thus effecting the predictive selec-

tion rule defined earlier. There only remains the design of logic required to implement

the control function within the shift register store. This is

now considered. Let the digrams A and B be stored in two separate

control shift register stores. If either input, A or B, is present

the two stores fill up in a complementary fashion as required, and

no further action is necessary. If both inputs are simultaneously

absent or blank, additional inputs have to be read into one register.

The decision as to which register should be supplied has to be delayed

until the arrival of the next active input, A or B. Whilst both

inputs remain blank both registers fill up with "empty" states.

Upon arrival of an active digit, say A, all empty stages must be 163.

switched to state A to achieve the required time-inverse inter-

polation function. The logic elements which fulfil these con-

ditions are shown in Fig. 6.4(b). Corresponding stages of the

two registers are connected with NOT AND gates which provide an

indication if both stages are empty. The arrival of the next

active digit is then gated into the required register by means of

the AND gates shown. Thus the implementation of the control logic involves two shift register stores and three gates per stage The coding scheme is completed by gating the 01 digrams, delayed by the amount of delay in the register, with the output of the 11 control register; similarly, the 10 digrams are gated with the output of the 00 control register. The two selected transition digrams are then combined as positive and negative signal excursions for transmission over the channel.

The size or length of the shift register store depends on the longest uninterrupted chequerboard signal sequence and is a matter which would require further experimental work for an accurate solution. However, in black and white picture material of the in type shown,Jigs. 4.4 and 5.3 it is estimated that such sequences longer than four sampling intervals are very rare. A store length of the same order would it is thought, be adequate.

In conclusion, several important properties of the proposed code will be enumerated. Firstly, the code signal has zero mean level due to the fact that over a long period the probability of 164.

selection of the two transition digrams as code symbols, and hence the probability of positive and negative signal excursions, are equal. This is a highly desirable property in practical transmission networks since it facilitates reactive coupling. Secondly, in the light of remarks made in section 6.3 regarding bandwidth considerations, a bandwidth compression by a factor of two may be claimed. The envelope of a signal sequence passed through a raised cosine impulse response filter is shown in Fig. 6.1(b). Thirdly, both the adapted codes proposed in this chapter are equally applicable

to data as well as picture signals, since inherent code ambiguity has been eliminated by the adaptive process. Lastly, only a fraction of the mean power of the original signal, proportional to the probability of occurrence of a transition digram, is required for the

code proposed in this section. This fraction is estimated con- servatively as about one tenth for a typical black and white picture signal; such a saving may be quite significant in some applications such as communication in spece.. 165.

SUMMARY OF CONCLUSIONS

On the basis of relevant engineering considerations (Chapter

2), it was concluded that the position and compressed video signals of a run-length coded representation of a picture signal may, in preference to other methods, be satisfactorily transmitted over a channel by means of a combined modulation process, in, which the position signal is phase modulated and the video signal amplitude modulated onto a common carrier. A complete analysis, (Chapter 3), of combined amplitude and phase modulation was presented showing that the system is fully capable of operating well within the required limits on error rates. It was also shown that high-level encoding of the position signal by conversion into its digram representation is of no advantage. The bandwidth of the complete modulated process may be satisfactorily contained within twice the baseband width of the compressed video signal, As this reduces the overall bandwidth compression of the run-length coded system by a factor of two, it is thought that the system should have most application for picture material which has inherently high redundancy and is thus amenable to significant data reduction, In this respect it is thought that the future of the system lies squarely in the field of black-and-white facsimile applications and coarsely quantised material in general. The second order statistics of video signals representing 166.

varied picture material were measured experimentally, (Chapter 4). A new technique which is believed to be simpler and more accurate

than previous methods was used, involving conversion of the picture

signal into digital form* The results, expressed as entropy and redundancy measures, for half-tone material are in close agreement with those of earlier workers.. A feature of the technique used

is the availability of results for intermediate levels of quantisa-, tion for each picture; it was thus found that material quantised into eight levels or less shows very significant redundancy... In particular, the large amount of redundancy in black-and- white facsimile material pro jyted the search for an efficient and simple code based on coding theory criteria, (Chapter 5). This led to the implementation and experimental assessment of several codes based on the digram representation of the signal. One particular ternary code was found to produce a nearly perfect subjective rendering of the picture whilst achieving a bandwidth compression by a factor of two.

A coding technique was proposed, (Chapter 6), whereby a non-uniquely decodable code could be adapted at the encoder by means of predictive feedback in such a way as to minimise errors after decoding. This technique was successfully implemented for the code mentioned above. A further realisation 167, of an adaptive code was proposed resulting in both power and bandwidth saving and in a sisnal with zero..mean level. 168. REFERENCES

1. Abramson, N. - "Bandwidth and Spectra of Phase -and -Frequency - Modulated Waves" - iEtE Transacs on Comm. Systems, CS-.11, Dec. 1963, pp.407414. 2. Abramson, N. - "InformatiOn Theory and Coding" - McGraw- Hill, 1963. 3. Arthurs, E. and Dym, H. - "On the Optimum Detection of Digital Signals in the Presence of White Gaussian Noise - A Geometric Interpretation and a Study of Three Basic Data Transmission Systems" - IRE Transacs on Comma Systems, CS 10, Dec. 1962, pp. 336-373. 4. Bell Telephone Laboratories Inc., Pub. - "Transmission Systems for Communications", 1965. 5. Bennett, W.R. - "Statistics of Regenerative Digital Trans- mission" - B.S.T.J., 37, 1958, p.1501. 6. Billings, A.R. - "A Coder for Halving the Bandwidth of Signals" •Proc.I.E.E., 105 B, 1958, p.182. 7. Black, H. - "Modulation Theory" - Van Nostrand, 1953. 8. Blachman, N.M. - "Limiting Frequency - Modulation Spectra" - Information and Control, 1, 1957, p.26. 9. .Brown, D.W. - "The Conversion of Television Signals into Digital Form" - M.Sc. Thesis, University of London, 1962. 10. Cahn, C.R. - "Performance of Digital PM Communication Systems" - IRE Transacs on Comm. Systems, CS,-7, May 1959, pp.3-6. 11. Cherry, E.C. and Gouriet, G.G. - "Some Possibilities for the Compression of Television Signals by Recoding" Proc.I.E.E., 100, III, 1953, p.9. 12. Cherry, E.C. et al - "An Experimental Study of the Possible Bandwidth Compression of Visual Image Signals" - Proc.I.E.E.E., 51, 1963, p.1507. 13. Cuccia, C.L. "Harmonics, Sidebands and Transients in Communication Engineering" - McGraw-Hill, 1952. 169.

14. Dupraz, J. - "Comparaison Theorique de Trois Systemes Digitaux" - L'Onde Electrique, Mar. 1963, p.298. 15. Fano, R.M. - "Transmission of Information" - MIT Press Wiley, 1961. 16. Fink, D., Ed. - "Television Engineering Handbook" - McGraw-Hill, 1957. 17. De France, H. et al - "Characteristics of the SECAM Colour Television System" - Electronic Engineering, 35, 1963, p.578. 18. Haber, F. - "Rapidly Converging Sample Weighting Functions" - I.E.E.E. Transacs on Comm. Systems, CS-12, Mar. 1964, correspondende, p.116. 194 - "Handbook of Statistical Tables" - Addison- Wesley, 1962. 20. Hurley, R.B. - "Transistor Logic Circuits" - Wiley, 1961. 21. Kharkievich, A.A. - "An Outline of the General Theory of Communication" - Moscow, 1955, in Russian. 22. Kretzmer, E.R. - "An Efficient Binary Data Transmission System" I.E.E.E. Transacs on Comm. Systems, CS 12, June 1964, correspondence. 23. Kretzmer, E.R. - "Reduced Alphabet Representation of Tele- vision Signals" - I.R.E. Conv. Rec., IV, 1956, 24. Kretzmer, E.R. - "Statistics of Television Signals" - B.S.T.J., 31, 1952, p.751. 25. Kubba„ M.H. - "Methods of Measuring Picture Detail in Relation to Television Signal Bandwidth Compression" - Ph.D. Thesis, Univ. of London, 1962. 26. Lee, Y.W. - "Statistical Theory of Communication" - McGraw- Hill, 1960. 27. Lender, A. - "The Duobinary Technique for High-Speed Data Transmission" - I.E.E.E. Transacs on Communications and Electronics, May 1963, 82, No.66, 170.

28. Lerner, R.M. - "Representation of Signals" and "Design of Signals", forming chapters 10 and 11 of "Lectures on Communication System Theory" - E. Baghdady, ed. McGraw- Hill, 1961. 29. Middleton, D. - "Introduction to Statistical Communication Theory" - McGraw-Hill, 1960, 30. Pearson, D.E. - "Fidelity Criteria for Visual Image Trans- mission over Noisy Communication Channels" - Ph.D. Thesis, University of London, 1965. 31. Pierce, J.R. and Karlin, J.E. - "Reading Rates and the Information Rate of a Human Channel" - B.S.T.J.., 36, Mar. 1957. 32. Pine, C.S.F. - "Studies in Coding for Bandwidth Reduction of Television Signals" - Ph.D. Thesis, University of London, 1960. 33. Prasada, B. - "Studies of Picture Signal Bandwidth Compression" - Ph.D. Thesis, University of London, 1960. 34. Reza, F.M. - "Introduction to Information Theory" - McGraw - Hill, 1961. 35. Richman, D. - "Colour-Carrier Reference Phase Synchronisation Accuracy in NTSC Color TV" - Proc.I.R.E.„ 42, 1954, p.106. 36. Roberts, L.G. - "Picture Coding Using Pseudo-Random Noise" - I.R.E. Transacs on Information Theory, IT-8, Feb. 1962, p.145. 37. Robinson, A.H. - "Automatic Digital Encoding for Bandwidth Reduction in Visual Image Transmission Systems" - Ph.D. Thesis, University of London, in preparation. 38. Sanders, R.W. - "Communication Efficiency Comparison of Several Communication Systems" - Proc.I.R.E., 48, 1960, p.575. 39. Schreiber, W.F. - "The Measurement of Third Order Probability Distributions of Television Signals" - I.R.E. Transacs on Information Theory, Vol.20 No.3, Sept. 1956. 171.

40. Shannon, C.E. - "Coding Theorems for a Discrete Source with a Fidelity Criterion" - In "Information and Decision Processes" - ed. Robert E. Machol McGraw-Hill, 1960. 41. Shannon, C.E. - "Communication in the Presence of Noise" Proc. I.R.E., 37, 1949, p.10. 42. Shannon, C.E. - "A Mathematical Theory of Communication" University of Illinois Press, Urbana, 1949* 43. Stewart, J.L. - "The Power Spectrum• of a Carrier Frequency- modulated by Gaussian Noise" - Proc. I.R.E., 42, Oct. 1954, p.1539. 44. Sonde, E.D. - "Pulse Transmission by AM, FM and PM in the Presence of Phase Distortion" - B.S.T.J., 40, 1961, P.353. 45, Vieri, B.J. - "Experiments with a Scheme of Data Reduction for Television Signals" - Ph.D. Thesis, University of London, in preparation.

46.Wainstein, and Zubakov„ - "Extraction of Signals from Noise" - Prentice-Hall, 1962. 47. Wheeler, H.A. - "The Solution of Unsymmetrical Sideband Problems" - Proc.I.R.E., 29, 1941, p.446. 48. Wiener, N, - "The Extrapolation, Interpolation, and Smoothing of Stationary Time Series with Engineering Applications" Wiley, 1949. 49. Woodward, P.M. - "The Spectrum of Random Phase Modulation" - T.R.E. Memo, No. 642, Sept. 1952. 50. Wyle, H. et al - "Reduced Time Facsimile Transmission by Digital Coding" - I.R.E. Transacs on Comm. Systems, 9, 1961, p.215. 172.

APPENDIX TO SECTION 2.6

Error Probability of Quantised Signal in Gaussian Noise.

The signal is assumed to have M levels, balanced around zero level, and a limitation on the maximum voltage excursion (or peak power, P ) of A volts. Thus, 1;N = A2 and (M-1)8 = 2A where 8 is the quantal spacing. Hence: A 2 M-.1 8 N N "2- • 3

Assuming random Gaussian noise, the probability of additive noise exceeding III and thus causing an error is, for all levels save the two extreme ones: a 2 2. 1 e .x /2 dx 571 - 18 2a = 20 ( •' ) , say. a5- The probability of error for the two extreme levels 2 1 a -x /2 . dx P2 ( S ) 1/"gt j 8 2d and thus the average probability of error) assuming equiprobable 173. levels is:

p(e) = [(14-2)2 ( g ) 2 M ( (-8;.• )]

2(M-1) M

Equations 1 and 2 thus relate p(e) and P/N. The relation is presented graphically in the text, Fig. 2.4. Computations involved extremal values of the error function, g, which were obtained from Ref. (19)4 174. APPENDIX TO SECTION 3,1

The Decodability of Combined Amplitude and Phase Modulation Let s.(t) be the modulated signal corresponding to the i-th

message from an alphabet of M messages. Then, in most cases of

interest, si(t) can be represented as a summation of D terms, (Lerner, 28; Arthurs and Dym, 3): k=D

silt) = aik 91k(t) ...(1) k=1 where

a ik si(t).0k(t).dt 01 and the set [m] -1 pi(t), 02(00 ok(t), is an orthonomial set, i.e..:

01,(t).01(t).dt = 0 for k 1

' ••03 i = 1 for k = 1

Complete detection may be accomplished at the receiver by providing a set of locally generated signals , multiplying each signal Ok(t) with the received signal si(t) and forming the integral as in equation 2. This provides all the coefficients suchaik asand, hence complete information about the transmitted message.

In combined amplitude and phase modulation, the signal sii(t) is

175,

given by:

(t) = K.A. cos (wt + M sij 1 Mn . j) where the i-th message of one set is represented by the amplitude of the carrier, Ai, and the j-th message of the other set by the

phase 2nj/M. K is a normalising constant for subsequent convenience. As equation 3 can be expanded as a sine and cosine term, it is clear that the desired set of local signals satisfying equations

1 and 2 consists of only two terms:

= cos wit ...(4) = sin wt

Productiategrationofs-1j(t) with cos w t and sin w t yields two outputs al and a2:

Ai al = fsii(t).01(t).dt = cos Mn ...(5)

, a2 s3.3W•952(t)ocit =-A.1 sin Mn 0

Thus, so long as the amplitude of the carrier, A is never reduced to zero, the outputs a1 and a2 provide complete information about both modulating messages. 176.

APPENDIX TO SECTION 3,1

Autocorrelation Function of Combined Amplitude and Phase Modulation The modulated process Y(t) is given by

Y(t) = A(t). cos [wet m(t) 3 ...(1) The autocorrelation function of Y(t) is then [ t (t) + (t+)) ...(2) Ry() =4E [A(t).A(t+T),R.ejcoc

Let A(t) = fl + m.X(t)1 and, for convenience of manipulation, let

X(t + ) = x2

ott 02

Equation 2 may then be rewritten as:

r _i(01_02) -J(01-02) Ry(T) = E e m(xl x2)e

-i(0 -0 ) + m2 1 2 xlx2e j exp jwe'r ...(4) x1,x2, 01 and 02 may be treated as a joint random process whose characteristic function )( is given by:

lx1+ u2x2+ v101+ v202 )((u102;v1,v2) = E [ e gu

If the process is assumed to be Gaussian then

exp 1 4 R (uvu2;v1)'2) Uv uv .u,v] 177.

where Ruv are the correlation coefficients given by:

R = E [x uv u v (7)

Each term of equation 4 may be rewritten in terms of partial derivatives of X and may then be expressed using equation 6

in terms of the coefficients Ruv . Thus, it is found that:

-i(° - ) 1 6 ] exp [R0(%) R0(0) (8)

exp D(t) say.

E[xle-j(01-932) ] = [R (V) R ( ) exp D(1) ...(9)

j Px0(0) R (T) g exp D(t) ...(10) E Lx2e Ox and

"j(931.12) ] = Rx E [X1x 2.e (%).exp D(T)

+ No(%) - 1%co(0) [ Rx0(0) Rox(T)] exp D(V) ...(11)

A detailed derivation of the above four identities is lengthy and laborious; the method is well illustrated in Wainstein and Zubakov (46). Inserting equations 7 to 11 in equation 4, the autocorrelation function of the modulated process is then found to be: 178.

R = 2 exp Dec). cos wc . < 1 + m Rx(1) + F(t)j>

m.exp D('r).sin wet'. G('r) ...(12)

Where, F(1) = px0(1) - Rx (0) 1 [Rx0(0) Roxer)]

G(`t) = R (-t) R ( ) xo x

and D(') = ('t) R(0) ...(15)

It is immediately recognised that if the processes X(t) and M(t) are independent both F(1) and Get) are zero and R ('t) reduces to:

R et) = 3 exp.D(17).cos wc [ 1 + m2.1.x(t)] ...(14) If there is no amplitude modulation of the carrier (i.e. X(t) = 0), then the process is purely phase modulated with autocorrelation function

R (t) = exp D(1). cos w 't

Let W0(f) be the power spectral density of T(t), given by the Fourier Transform of equation 15, Then the power spectral density, (f), of the combined process is given by the Fourier Transform W0,x of equation 14: 2 W0:x (f) = W(f) m .W0(f) x Wx(f) where W x(f) is the power spectral density of X(t). 179.

It is seen, therefore, that the combined spectrum consists

of the sum of the spectrum of pure phase modulation and a

"smearing" component weighted by m2 If m is small then this

component contributes a second order effect to the overall spectrum.

Thus, for example, if both modulating signals have rectangular

spectral densities and m = 0.3, then the fraction of power beyond

the baseband of the modulated signal is only about 9%. It is believed that the above property of the spectrum of combined amplitude and phase modulation, given by equation 16, has not been hitherto noted. 180 APPENDIX TO SECTION 3.4.

Table Hof the Spectral Functions wx, wxaowx, and G(N) of eq. 3.8. The Variance N, given by eq.3.9 is also indloatedQ oc,r4) c—Lua.4-N°"'iz cfw'riiiCINn %

0-.... , ON re\ \r) I-4 0 • \ C-- H C-- H C.- ...0 Alaa.lie 0 0 CO VD r•(\ ON LC\ a... .4- --- v rc\ tf\ CV CV CV H H 0 0 0 0 0 •0 • a • a • • • • • • •

IA ilk) 0 0 O• •D •0 CV r-I CT Lc\ K\ .4 vv -4 M0 C— MC• tr\ C-- re\ H 0 0 to K.\ K\ tc\ CV Cv .--1 H 0 0 0 0 • • • . • • • • • • .

0 OD •D et 0 a-, cr• r-i c‘i c.-- .4- RI 0 co ...z.moo et o• K\ •D CV 0 I --- VI te\ m C\I CV H H 0 0 0 r.5 'Kt- . . . . • • • • • .

te‘ cla...0 c— 0 \c• H H H *4- ON 0 ...... v tc‘ cv c— 0 -.4- c— H re' \ r-I I I 0 K\ .4 .ct Pc\ r\ cv H H 0 0 • • . • At • • • •

C‘I 4,_It CV n:1- •0 CV 0 tr‘ (NJ •ct CV ••••••• U t N 0 N CV CV tc\ C-- H 0 I I c.5 CV It\ it\ •.:4- te% CV H 0 0 0 • • $ • • • • • •

01 EC CO .kt rq 4Z1. 1.4 # '0 e—I c-- 0 re\ *4- Lc\ c0 0 0 0 0 ,_1•4 ck.I is\ *4- -Ki- $t cV H 0 la • 0 • • • • •

k ri ; ‘Z CV 0 VD tc\ V •Co CO 0 H tc\ 0 0 0 0 0 0 . ,:. 4) it\ V\ Nto 01 • • • • •

LC \ 0 11\ 0 is-\ 0 0 0 0 0 0 CV LC\ C— 0 c‘l tc\ 0 tc\ 0 uN • \'%--. • • . • • • • • . 3 .--1 H H cv N r'\ 0\ 18±.

APPENDIX TO SECTION 4,4.3

Circuit Details of Digram Encoder.

A full circuit is shown in Figs. A.4.4.(a) and A.4.4.(b). Semiconductor elements are used throughout. As the clock rate of 6 the system is 6.10 pps: corresponding to a digit interval of about 167 nsec„ most circuits were designed to have rise and fall times

of about 10 nsec. Transistors with cut off frequency, fa, ranging between 60 and 400 Mc were found quite adequate, design being based on the commonly available ASZ 20, ASZ 21, ASY 67,

2N706, 2N708 types. The ASZ 13 (GEX71) diode was used through-

out. All circuits were of the non-saturating type; where feasible voltage drive with current switching was used, this mode providing near optimum transistor performance. Directly coupled logic was used throughout; the logic implementation was somewhat unconventional inasmuch as two separate voltage levels were used, 1.4V and 2.0V. This was required by the unconventional ab and ab gates which had the asset of great simplicity.

1 8 2 .

--12.V

• J3•oV L + 6 V VIDE-0 AMPL-IFIER 62VANITISeR SAMPLER O MoNloS-TA131.:E.. x CLOCK Put-SES vRom

-1Z V

27 6-8 a-7 2:7

2.N 706 2.N1706 AS1" 67 2N7o6 .7_NI 7o 6 47 6.6' 4-7 6.8 ,_,2:0V 82 £3-2- 4-7 2-0V J L 560 8-2

a2. AL LAY 132.

l0 of oo DI GRAM LoG1C. READ ()UT I

'FIG. A.4.4.(a)-I Circuit Diagram of the Digram Encoder of Sec. 1i .4.3.

183.

DIG RANI LOGIC 1p4Pu-r6 FROM LOGIC Re-AV:FT - t2_V

3So Fou R Re..S A t•-tF-1- I 1\1G URCU ti rs 2.1,47o6

22_ -1::)IGRAts./1 ov7PuT" Pvt-Se.5 5V

RE•SA MPI-tt-4 G P1/4)1-SS 2V DIGRAM FoR f}DAFT 1'Ja WORKING

—taV tS I•S 1-5 az.o- 47 z000 70 vlSec ••0 ASz Zoo° AS2 2.0 24,1706 2:2- Z2.o 15 3-1

AND GATE

V -I2..V

22.0 33o 33 33 330 6.8 GEX-71 100 2.7 7-2- 470 T.7071711°. e

10000 82 A5Y 67 6e0 1.0 4-7 t-o 2N-70 T 1'" NIP uT" 4-q 22 O

SHAPER DRIVING CiRC-0

FIG. A.4.4. (b),ICircuit Diagram of the Digram Encoder of Sec. 4.4.3.

—12V +Cy

1.2 ASZ to 1/PI VP to /P2 2.1•4706

BASIC De..c0DING OUTPUT 1.1 So to 1/el ZISTASLE. ecNtAPt-IFIE_R D %odes Asz.

1.2.. Oc 00

50. tot/pz -to t /p2._

DIGRAm STEERING 1~GIc

FIG. A.5.3. Circuit Diagram of Decoding Unit.