<<

Henry Stark and John W. Woods, , , and Random Variables for Engineers, 4th ed., Pearson Education Inc., 2012. ISBN: 978-0-13-231123-6 Chapter 9 Random Processes

Sections 9.1 Basic Definitions 544 9.2 Some Important Random Processes 548 9.3 Continuous-Time Linear Systems with Random Inputs 572 577 9.4 Some Useful Classifications of Random Processes 578 Stationarity 579 9.5 Wide-Sense Stationary Processes and LSI Systems 581 Wide-Sense Stationary Case 582 Power 584 An Interpretation of the Power Spectral Density 586 More on White Noise 590 Stationary Processes and Differential Equations 596 9.6 Periodic and Cyclostationary Processes 600 9.7 Vector Processes and State Equations 606 State Equations 608 Summary 611 Problems 611 References 633

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 1 of 78 ECE 3800 9.1 Basic Concepts

A random process is a collection of time functions and an associated probability description.

When a continuous or discrete or mixed process in time/space can be describe mathematically as a function containing one or more random variables.

 A sinusoidal waveform with a random amplitude.  A sinusoidal waveform with a random phase.  A of digital symbols, each taking on a random value for a defined time period (e.g. amplitude, phase, ).  A (2-D or 3-D movement of a particle)

The entire collection of possible time functions is an ensemble, designated as xt , where one particular member of the ensemble, designated as xt, is a sample function of the ensemble. In general only one sample function of a random process can be observed!

Think of: X t  Asinwt  , 0    2 where A and w are known constants.

Note that once a sample has been observed …

x1 t1   Asinwt1   the function is known for all time, t.

Note that, xt2 is a second time sample of the same random process and does not provide any “new ” about the value of the random .

x1 t2   Asinwt2  

There are many similar ensembles in engineering, where the sample function, once known, provides a continuing solution. In many cases, an entire system design approach is based on either assuming that remains or is removed once actual measurements are taken!

For example, in communications there is a significant difference between coherent (phase and frequency) demodulation and non-coherent (i.e. unknown starting phase) demodulation.

On the other hand, another measurement in a different environment might

x2 t1   A2 sinwt1  2  In this “space” the random variables could take on other values within the defined ranges. Thus an entire “ensemble” of possibilities may exist based on the random variables defined in the random process.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 2 of 78 ECE 3800 For example, assume that there is a known AM transmitted:

st  1 b  Atsinwt

at an undetermined distance the signal is received as

yt  1 b  At sinwt  , 0    2

The received signal is mixed and low pass filtered …

xt  h t  y t  cos wt  ht 1 b  Atsinwt   coswt,0    2

xt  h t  y t  cos wt  ht  1 b  At 0.5sin2  wt   sin  ,0    2

If the filter removes the 2wt term, we have

1 b  At xt  h t  y t  cos w t  sin , 0    2  2

Notice that based on the value of the , the output can change significantly! From producing no output signal, (  0, ), to having the output be positive or negative (  0to or to 2 ). P.S. This is not how you perform non-coherent AM demodulation.

To perform coherent AM demodulation, all I need to do is measured the value of the random

variable and use it to insure that the output is a maximum (i.e. mix with coswt  m , where.

 m   t1

Note: the phase is a function of frequency, time, and distance from the transmitter.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 3 of 78 ECE 3800 From our textbook

Random Stochastic Sequence

Definition 8.1-1. Let ,,P be a . Let  . Let Xn,be a mapping of the  into a space of complex-valued on some index Z. If, for each fixed integer n  Z , Xn, is a random variable, then Xn, is a ransom (stochastic) sequence. The index set Z is all integers,    n  , padded with zeros if necessary,

Definition 9.1-1. Let ,,P be a probability space. then define a mapping of X from the sample space  to a space of continuous time functions. The elements in this space will be called sample functions. This mapping is called a random process if at each fixed time the mapping is a random variable, that is, Xt, for each fixed t on the real line    t  .

Example sets of .

Figure 8.1-1 Illustration of the concept of random sequence X(n,ζ), where the ζ domain (i.e., the sample space Ω) consists of just ten values. (Samples connected only for plot.)

Example sets of random process.

Figure 9.1-1 A random process for a continuous sample space Ω = [0,10].

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 4 of 78 ECE 3800 Example 9.1-2

Separable random process may be constructed by combining a deterministic sequence with one or more random variables.

The classic example already shown is a sinusoid with random amplitude and phase:

X t,  A sin2  f 0 t   

Where the amplitude and phase are R.V. defined based on the probability space selected.

Example 9.1‐3

A random process used to model a continuous sequence of random communication symbols. X t   An pt  Tn n

In a communication class, Dr. Bazuin would typically use the following

X t   An  p t  n T , for pt non zero for 0  t  k T n

Here An is the amplitude and phase of a complex communication symbol and p(t) is the deterministic time function, the simplest of which is a rectangular pulse in time.

This can be used to describe a wide of digital communication systems, including; Phase- Shift Keyed (PSK) or Quadrature Amplitude Modulation (QAM) communication .

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 5 of 78 ECE 3800 The application of the Operator

Moments play an important role and, for Ergodic Processes, they can be estimated from a single process in time of the infinite number that may be possible.

Therefore,

 X t  EX t and the correlation functions (auto- and cross-correlation) * RXX t1,t2   EX t1  X t2  

* RXY t1,t2   EX t1 Yt2   and the functions (auto- and cross-correlation) * K XX t1,t2  EX t1   X t1  X t2   X t2  

* K XY t1,t2  EX t1   X t1  Yt2  Y t2   with * K XX t1,t2  RXX t1,t2   X t1   X t2 

Note that the can be computed from the auto-covariance as * 2 K XX t,t  EX t  X t  X t  X t   X t and the “power” function can be computed from the auto-correlation

* 2 RXX t,t  EX t X t  EX t 

2 2 2 For real X(t) RXX t,t  EX t   X t  X t

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 6 of 78 ECE 3800 Example 9.1‐5 Auto‐correlation of a sinusoid with random phase

Think of: X t  Asinwt  ,      where A and w are known constants. And theta is a uniform pdf covering the unit circle.

The is computed as

 X t  EX t  EAsinwt  

 X t  EX t  A Esinwt    1  t  E X t  A sin wt   d X      2 

A   t  EX t    coswt  X 2  A A  t  EX t    coswt    cos wt     0  0 X 2 2 ( What would happen if 0     instead? )

The auto-correlation is computed as

* * RXX t1,t2  EX t1  X t2   EAsinwt1   Asinwt2    R t ,t  E X t  X t *  A2  E 1  cos w t  t  1  cos w  t  t  2  XX 1 2  1 2  2   1 2  2  1 2  A2 A2 R t ,t   cosw t  t   Ecosw t  t  2 XX 1 2 2 1 2 2 1 2 A2 A2 R t ,t   cosw t  t  0   cosw t  t XX 1 2 2 1 2 2 1 2 ( This works if 0     instead. )

Note that if A was a random variable (independent of phase) we would have … EA2  EA2  R t ,t   cosw t  t  R    cosw XX 1 2 2 1 2 XX 2

and we would still have EA  t  EX t    0  0 X 2

Note: this Random Process is Wide-Sense stationary (mean and variance not a function of time)

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 7 of 78 ECE 3800 Definition 9.1‐3

All correlation and covariance functions are positive semidefinite.

All auto-correlation functions are diagonal dominate.

Using the Cauchy-Schwartz Inequality

RXX t1 ,t2  RXX t1 ,t1  RXX t2 ,t2  which for a WSS random process becomes

RXX    RXX 0

Additional Properties for real, WSS random processes.

2 2 RXX 0   X   X

max RXX    RXX 0

RXX    RXX  

For signals that are the sum of independent random variable, the is the sum of the individual autocorrelation functions.

W t  X t Yt

RWW   RXX   RYY   2  X  Y

If X is ergodic and zero mean and has no periodic component, then

lim RXX    0  

Interpretation of WSS autocorrelation …

The statistical (or probabilistic) similarity of future (or past) samples of a random process to other samples of the process for an ergodic random process.

How similar is a time shifted version of a function to itself?

Nominal definition of … the time base statistics are equivalent to the probabilistic based statistics of a stationary random sequence or process.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 8 of 78 ECE 3800 For the autocorrelation defined as:

  RXX t1,t2  EX1X 2  dx1 dx2 x1x2 f x1, x2  

For WSS processes:

RXX t1,t2  EX tX t    RXX  

If the process is ergodic, the time average is equivalent to the probabilistic expectation, or

T 1  XX   lim xt  x t   dt  x t  x t   T  2T  T and

 XX    RXX  

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 9 of 78 ECE 3800 A strange autocorrelation

A random process has a sample function of the form

A, 0  t  1 X t   0, else where A is a random variable that is uniformly distributed from 0 to 10.

Find the autocorrelation of the process.

1 f a  , for 0  a  10 10

Using

RXX t1 ,t2   EX t1  X t2 

2 RXX t1 ,t2  EA , for 0  t1 ,t2  1

10 1 R t ,t   a 2  da, for 0  t ,t  1 XX 1 2  1 2 0 10

10 a 3 RXX t1 ,t2  , for 0  t1 ,t2  1 30 0

1000 100 R t ,t   , for 0  t ,t  1 XX 1 2 30 3 1 2

RXX t1 ,t2  0, for t1  0, 1  t1 ,or t2  0, 1  t2

Not WSS as it is a function of time!

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 10 of 78 ECE 3800 Example: xt  Asin2  f t for A a uniformly distributed random variable A 2,2

RXX t1 ,t2  EX t1  X t2   EAsin2  f t1  Asin2  f t2 

 2 1  RXX t1 ,t2  EX t1  X t2  E A  cos2  f  t1  t2  cos2  f  t1  t2  2 

1 R t ,t   EA2  cos2  f  t  t  cos2  f  t  t XX 1 2 2 1 2 1 2

for   t2  t1

1 2 R t ,t    cos2  f   cos2  f  t  t XX 1 2 2 12 1 2

16 R t ,t   cos2  f   cos2  f  t  t XX 1 2 24 1 2

A non-! It is still a function of both time variables!

The time based formulation: T 1  XX   lim xt  x t   dt  x t  x t   T  2T  T

1 T  XX   lim Asin2  f t  Asin2  f  t   dt T   2T T

T 2 1 1  XX   A  lim cos2  f    cos2  f  2 t   dt T   2T T 2

A2 A2 1 T  XX    cos2  f     lim cos2  f  2 t   dt T   2 2 2T T

A2 A2     cos2  f     cos2  f  XX 2 2

Acceptable, but the R.V. is still present?! To find a value not dependent upon a R.V  A2  16 E XX   E   cos2  f    cos2  f   2  24

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 11 of 78 ECE 3800 Example: xt  Asin2  f t   for  a uniformly distributed random variable  0,2 

RXX t1 ,t2  EX t1  X t2  EAsin2  f t1   Asin2  f t2  

2 1  RXX t1 ,t2  EX t1  X t2  A  E cos2  f  t1  t2  cos2  f  t1  t2  2  2  A2 A2 R t ,t  EX t  X t   cos2  f  t  t   Ecos2  f  t  t  2  XX 1 2 1 2 2 1 2 2 1 2

Of note is that the phase need only be uniformly distributed over 0 to π in the previous step! A2 R t ,t  EX t  X t   cos2  f  t  t XX 1 2 1 2 2 1 2

for   t2  t1 A2 R    cos2  f   XX 2 but A2 R   R    cos2  f  XX XX 2

Assuming a uniformly distributed random phase “simplifies the problem” !!!

Also of note, if the amplitude is an independent random variable, then EA2  R   cos2  f  XX 2

The time based formulation: T 1  XX   lim xt  x t   dt  x t  x t   T  2T  T 1 T  XX   lim Asin2  f t   Asin2  f  t    dt T   2T T A2 1 T  XX    lim cos2  f   cos2  f  2t   2  dt T   2 2T T A2     cos2  f  XX 2

This appears to be stationary but not technically ergodic … due to the R.V. in the time AC.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 12 of 78 ECE 3800  t  t0  Example: xt  B  rect  for B =+/-A with probability p and (1-p) and t0 a uniformly  T   T T  distributed random variable t0   , . Assume B and t0 are independent.  2 2 

  t1  t0   t1  t0  RXX t1 ,t2  EX t1  X t2  EB  rect   B  rect    T   T 

 2  t1  t0   t1  t0  RXX t1 ,t2  EX t1  X t2  EB  rect   rect    T   T 

As the RV are independent

2   t1  t0   t2  t0  RXX t1 ,t2  EX t1  X t2  EB  Erect   rect    T   T 

2 2   t1  t0   t2  t0  RXX t1 ,t2  A  p   A  1 p  Erect   rect    T   T 

T 2  t  t   t  t  1 R t ,t  A2  rect 1 0  rect 2 0   dt XX 1 2      0 T  T   T  T 2

For t1  0 and t2   T 2   t  1 R 0,  A2  1 rect 0   dt XX     0 T  T  T 2

The can be recognized as being a triangle, extending from –T to T and zero everywhere else.

2    RXX   A tri   T 

0,   T  1 A2   T  , T    0  T RXX    1 A2   T  , 0    T  T  0, T  

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 13 of 78 ECE 3800 The time based formulation: 1 T  XX   lim xt  x t   dt  x t  x t   T   2T T

C 1  t  t0   t   t0   XX   lim B  rect   B  rect   dt C  2C C  T   T 

A change in variable for the integral t  t  t0 . And only integrate over the finite T. T 1 2  t      B 2  1 rect  dt XX     T T  T  2 For 0    T 2 T 1 2 1       B 2  1dt  B 2   T   T  B 2  1 XX   2 2   T  T T  T  2 For  T    0 2 T  1 2 1       B 2  1dt  B 2   T   T  B 2  1 XX   2 2   T T T  T  2 And

2  t    XX   B tri   T 

Not ergodic as taking the expected value of the time autocorrelation … however …

2  t   E XX   EB tri   T 

2 2  t   E XX   A  p   A  1 p tri   T 

2  t   E XX   A tri   T 

This is identical to the probabilistic autocorrelation previously computed!

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 14 of 78 ECE 3800 Some Important Random Processes

Asynchronous Binary Signaling

The pulse values are independent, identically distributed with probability p that amplitude is a and q=1-p that amplitude is –a. The start of the “zeroth” pulse is uniformly distributed from –T/2 to T/2 1 T T pdf D  , for   D  D 2 2

Determine the autocorrelation of the bipolar binary sequence, assuming p=0.5.   t  D  k T  X t   X k  rect  k   T  Note: the rect function is defined as  T T  t  1,   t  rect    2 2 T   0, else  Determine the Autocorrelation RXX t1 ,t2   EX t1  X t2 

    t1  D  n T   t2  D  k T  RXX t1 ,t2  E  X n  rect    X k  rect  n  T  k  T 

    t1  D  n T   t2  D  k T  RXX t1 ,t2  E X n  rect   X k  rect  nk   T   T 

    t1  D  n T   t2  D  k T  RXX t1,t2  E X n  X k  rect   X k  rect  nk   T   T 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 15 of 78 ECE 3800     t1  D  n T   t2  D  k T  RXX t1 ,t2  EX n  X k  Erect   X k  rect  nk    T   T 

For samples more than one period apart, t1  t2  T , we must consider E X  X  p  a  p  a  p  a  1 p   a 1 p  a p  a  1 p  a  1 p   a  k j 

2 2 2 EX k  X j  a  p 2  p  1 p 1 p 

2 2 EX k  X j  a  4 p 4  p 1

2 2 For p=0.5 EX k  X j   a  4 p 4  p 1  0

For samples within one period, t1  t2  T , E X  X  E X 2  p  a 2  1 p   a 2  a 2 k k  k     

2 2 EX k  X k1  a  4p 4 p 1  0

For samples within one period, t1  t2  T , there are two regions to consider, the sample bit overlapping and the area of the next bit.  2   t1  D  k T   t2  D  k T  RXX t1 ,t2  a   Erect   rect  k   T   T 

But the overlapping area … should be triangular. Therefore T  T 1 2 1 2 R    E X  X  dt   E X  X  dt, for T    0 XX   k k1  k k T  T T T 2 2

T  T 1 2 1 2 R    E X  X  dt   E X  X  dt, for 0    T XX   k k  k k1 T  T T T 2 2

or

 T 1 2 R   a 2   1 dt, for T    0 XX   T T 2

T 1 2 R   a 2   1 dt, for 0    T XX   ta  T 2

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 16 of 78 ECE 3800 Therefore  T  a 2  , for  T    0  T RXX     2 T  a  , for 0    T  T or recognizing the structure    R   a 2 1 , for T    T XX     T 

This is simply a triangular function with maximum of a2, extending for a full bit period in both time directions.

For unequal bit probability

  t     a 2  a  4p 2 4 p 1  , for T    T  T T  RXX       2 2 a  4p 4 p 1 , for T  

As there are more of one bit or the other, there is always a positive correlation between bits (the curve is a minimum for p=0.5), that peaks to a2 at  = 0.

Note that if the amplitude is a random variable, the expected value of the bits must be further evaluated. Such as,

2 2 EX k  X k     

2 EX k  X k1   

In general, the autocorrelation of communications signal waveforms is important, particularly when we discuss the power spectral density later in the textbook.

If the signal takes on two levels a and b vs. a and –a, the result would be

EX k  X j  p  a  p  a  p  a  1 p  b 1 p b p  a  1 p b 1 p b

For p = 1/2

2 1 2 1 1 2  a  b  EX k  X j  a  ab  b    4 2 4  2 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 17 of 78 ECE 3800 And 2 2 2 EX k  X k  EX k  p  a  1 pb

For p = 1/2 2 2 2 1 2 2  a  b   a  b  EX k  X k  EX k   a  b       2  2   2 

Therefore,  a  b 2 a  b 2               1 , for T    T  2   2   T  RXX    2  a  b    , for   T  2 

For a = 1, b = 0 and T=1, we have 1 1        1 , for T    T 4 4  T  RXX    1 , for   T 4

Figure 9.2-2 Autocorrelation function of ABS random process for a = 1, b = 0 and T = 1.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 18 of 78 ECE 3800 Examples of discrete waveforms used for communications, , controls, etc.

(a) Unipolar RZ & NRZ, (b) Polar RZ & NRZ , (c) Bipolar NRZ , (d) Split-phase Manchester, (e) Polar quaternary NRZ. From Cahp. 11: A. Bruce Carlson, P.B. Crilly, Communication Systems, 5th ed., McGraw-Hill, 2010. ISBN: 978-0-07-338040-7

In general, a periodic bipolar “pulse” that is shorter in duration than the pulse period will have the autocorrelation function

t    R   A2  w 1 , for  t    t XX    w w t p  tw 

for a tw width pulse existing in a tp time period, assuming that positive and negative levels are equally likely.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 19 of 78 ECE 3800 Digital signal autocorrelation functions give rise to a range of Power Spectral Density results. The following shows some of the expected frequency responses for digital waveforms.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 20 of 78 ECE 3800 Exercise 6‐3.1 – Cooper and McGillem a) An ergodic random process has an autocorrelation function of the form

R XX   9  exp 4   16  cos10  16

Find the mean- value, the mean value, and the variance of the process.

The mean-square (2nd ) is 2 2 2 EX  RXX 0  9 16 16  41     The constant portion of the autocorrelation represents the square of the mean. Therefore 2 2 EX    16 and   4

Finally, the variance can be computed as, 2 2 2 2   EX  EX   RXX 0   4116  25

b) An ergodic random process has an autocorrelation function of the form 4  2  6 RXX   2  1 Find the mean-square value, the mean value, and the variance of the process.

The mean-square (2nd moment) is

2 6 2 2 EX  RXX 0   6     1 The constant portion of the autocorrelation represents the square of the mean. Therefore 4  2  6 4 EX 2   2  lim   4 and   2 t  2 1 1

Finally, the variance can be computed as, 2 2 2 2   EX  EX   RXX 0   6  4  2

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 21 of 78 ECE 3800 Poisson Counting Process

Applications and properties

 arrival times  radioactive decay’  memoryless property  mean arrival rates

Complicated analysis and derivation left for reading in the textbook.

Random Telegraph Signal

The random telegraph signal was originally defined based on a telegraph operator or someone manually sending Morse code. The signal may also represent “zero crossings” in an FM modulated signal.

The signal is a binary signal with random transitions in time.

Figure 9.2-4 Sample function of the random telegraph signal.

Let X(0) = +/-a with equal probability. Use the Poisson arrival process from Chap. 8 as the time of transition to the opposite level. The arrival time is now a R.V.

The probability of signal level correlation at two seperate4 times, assuming different symbols

RXX t1 ,t2  EX t1  X t2  a 2  Pa,a  a   a  P a,a  a   a  P  a,a   a 2  P  a,a 

 a 2  Pa | a  P a  a   a  P a | a  P  a  a   a  P  a | a  P a   a 2  P  a | a  P  a

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 22 of 78 ECE 3800 For P(a)=1/2

1 R t ,t   a 2  Pa | a  P  a | a  P a | a  P  a | a  XX 1 2 2

This becomes the probability of odd or even transitions after the average time interval of the mean of the Poisson arrival time. With this for “the positive time axis” it becomes

   k   k  R   a 2  exp      exp     ,   0 XX         even k0 k! odd k0 k! 

k 2 k    RXX   a  exp      1  ,   0 k0 k!

The summation can be determined as for tau>0

2 RXX   a  exp 2    ,   0

To include both positive and negative time (the property of positive and negative autocorrelation)

2 RXX    a  exp 2    

Figure 9.2-5 The symmetric exponential of an RTS process (a = 2.0, λ = 0.25).

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 23 of 78 ECE 3800 Binary Phase Shift Keying

Figure 9.2-6 System for PSK modulation of Bernoulli random sequence B[n].

The communications symbols represent 180 degree phae shifting of a sinusoidal waveform.

st  cos 2   f c t   a t

where

 a t  k,for k T  t  k 1T

 , if b n  1  2   and n    , if b n  0  2 

Finally

X t  cos 2  f c  t  k T  k k

X t  cos 2  fc  t  k T  cosk sin2  fc  t  k T sin k k

X t   sin2  f c  t  k T sink k

Typically T is selected so that the symbols form “complete” cosine waveforms f c T  integer

  RXX t1,t2  Esin 2  fc  t1  k T sin k  sin 2  f c  t2  n T sin n   k n 

  RXX t1,t2  Esin 2  fc  t1  k T sin k sin 2  f c  t2  n T sin n   n k 

  RXX t1,t2  Esin 2  fc  t1  k T sin 2  f c  t2  n T sin k sin n   n k 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 24 of 78 ECE 3800 Notice that

Esinksinn   k  n

The following becomes similar to the previous “rectangular magnitude becoming a triangular autocorrelation. In , we typically define a filtering function so to remove at twice the frequency of interest.

RXX t1 ,t 2  sin2  f c t1  k T  sin2  f c t 2  k T , 0  t1 ,t 2  T k

This is where the text stops … using some further analysis and assumptions.

1 RXX t1,t2   cos 2  f c  t1  t2  cos 2  fc   t1  t2  2 k T , 0  t1,t2  T 2 k

Setting   t1  t2 vary over two T. In addition, kT is an integer

1 RXX ,t2   cos 2  fc   cos 2  fc    2t2 , T    T and 0  t2  T 2 k

Averaging across all possible t2, (alternately, if the original equations had a random phase component …)

1 RXX ,t2   cos 2   f c   Ecos 2  fc    2t2  2 2 k

The autocorrelation would become

1    RXX  ,t2   cos 2   f c  tri , T    T 2 T 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 25 of 78 ECE 3800 There is an alternate derivation that focuses on “one” symbol cycle for each of the t1 and t2 sequences, particularly if symbols have zero cross-correlation

Esinksinn   k  n

only one symbol and in fact the same symbol time shifted is of interest. If one symbol remains fixed and the other varies in time.

  t   t    RXX t,t   Erect   cos 2  f c t   rect   cos 2  f c  t     T   T  

1   t   t    RXX t,t    Erect   rect  cos 2   fc   cos 2  fc  2t   2  2  T   T  

The expected value of the random phase, component goes to zero and

1   t   t   RXX t,t    Erect   rect   cos 2   fc  2  T   T 

If a time average in t is performed the result becomes.

1    RXX   tri   cos 2   f c  2 T 

If the “symbols” do not have equal probability, there will be a cross-correlation component. Then, the “envelope” of the autocorrelation has a triangular sections (as above) and the rest is a “DC magnitude” that will multiple the cosine “modulated waveform” element. Such as

1    RXX    a tri  cos 2   f c   b  cos 2   f c  2 T 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 26 of 78 ECE 3800 Note: the following notes are from Cooper and McGillem ….

The Autocorrelation Function

The autocorrelation is defined as:   RXX t1,t2  EX1X 2  dx1 dx2 x1x2 f x1, x2   The above function is valid for all processes, stationary and non-stationary. For WSS processes: RXX t1,t2  EX tX t    RXX   If the process is ergodic, the time average is equivalent to the probabilistic expectation, or T 1  XX   lim xt  x t   dt  x t  x t   T  2T  T and  XX    RXX  

Properties of Autocorrelation Functions 2 2 2 1) RXX 0  EX  X or  XX 0  x t

2) RXX   RXX   3) RXX   RXX 0 4) If X has a DC component, then Rxx has a constant factor. X t  X  Nt 2 RXX   X  RNN  5) If X has a periodic component, then Rxx will also have a periodic component of the same period. X t  A coswt  , 0    2 A2 R   EX t X t    cosw  XX 2 6) If X is ergodic and zero mean and has no periodic component, then lim RXX    0   7) Autocorrelation functions can not have an arbitrary shape. One way of specifying shapes permissible is in terms of the of the autocorrelation function.

RXX    0 for all w

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 27 of 78 ECE 3800 The Crosscorrelation Function The crosscorrelation is defined as:   RXY t1,t2  EX1Y2  dx1 dy2 x1y2 f x1, y2     RYX t1,t2  EY1X 2  dy1 dx2 y1x2 f y1, x2   The above function is valid for all processes, jointly stationary and non-stationary. For jointly WSS processes: RXY t1,t2  EX tYt    RXY   RYX t1,t2  EYtX t    RYX   Note: the order of the subscripts is important for cross-correlation! If the processes are jointly ergodic, the time average is equivalent to the probabilistic expectation, or T 1  XY   lim xt  y t   dt  x t  y t   T  2T  T T 1 YX   lim yt  x t   dt  y t  x t  T  2T  T and  XY    RXY   YX    RYX  

Properties of Crosscorrelation Functions 1) The properties of the zoreth lag have no particular significance and do not represent mean-square values. It is true that the “ordered” crosscorrelations must be equal at 0. . RXY 0  RYX 0 or  XY 0  YX 0 2) Crosscorrelation functions are not generally even functions. However, there is an antisymmetry to the ordered crosscorrelations: RXY    RYX   3) The crosscorrelation does not necessarily have its maximum at the zeroth lag.

It can be shown however that RXY    RXX 0 RYY 0 As a note, the crosscorrelation may not achieve this maximum anywhere … 4) If X and Y are statistically independent, then the ordering is not important RXY   EX t Y t    EX t EY t    X Y and

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 28 of 78 ECE 3800 RXY    X Y  RYX   5) If X is a stationary random process and is differentiable with respect to time, the crosscorrelation of the signal and it’s derivative is given by dR   R   XX XX d Similarly, 2 d RXX   RXX    d 2

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 29 of 78 ECE 3800 Measurement of The Autocorrelation Function

We love to use time average for everything. For wide-sense stationary, ergodic random processes, time average are equivalent to statistical or probability based values. T 1  XX   lim xt  x t   dt  x t  x t   T  2T  T

Using this fact, how can we use short-term time averages to generate auto- or cross-correlation functions?

An estimate of the autocorrelation is defined as: T  1 Rˆ   xt  x t   dt XX T   0

Note that the time average is performed across as much of the signal that is available after the time shift by tau.

Digital Time Sample Correlation

In most practical cases, the operation is performed in terms of digital samples taken at specific time intervals, t . For tau based on the available time step, k, with N equating to the available time interval, we have: N k 1 Rˆ kt  xit  x it  kt  t XX N 1 t  kt  i0

N k 1 Rˆ kt  Rˆ k  xi  x i  k  XX XX N 1 k  i0

In computing this autocorrelation, the initial weighting term approaches 1 when k=N. At this point the entire summation consists of one point and is therefore a poor estimate of the autocorrelation. For useful results, k<

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 30 of 78 ECE 3800 It can be shown that the estimated autocorrelation is equivalent to the actual autocorrelation; therefore, this is an unbiased estimate.

 N k  ˆ ˆ 1 ERXX kt  ERXX k  E xi  x i  k   N 1 k    i0 

N k 1 ERˆ kt   Exi  x i  k  XX N 1 k  i0 N k 1   R kt N 1 k  XX i0 1   N  k 1  R kt N 1 k XX

ˆ ERXX kt RXX kt

As noted, the validity of each of the summed autocorrelation lags can and should be brought into question as k approaches N.

Biased Estimate of Autocorrelation

As a result, a biased estimate of the autocorrelation is commonly used. The biased estimate is defined as:

N k ~ 1 R k  xi  x i  k  XX N 1  i0

Here, a constant weight instead of one based on the number of elements summed is used. This estimate has the property that the estimated autocorrelation should decrease as k approaches N.

The expected value of this estimate can be shown to be

~  n  ERXX k  1   RXX kt  N 1

The variance of this estimate can be shown to be (math not done at this level)

M ~ 2 VarR k   R kt 2 XX N  XX k M This equation can be used to estimate the number of time samples needed for a useful estimate. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 31 of 78 ECE 3800 Exercise 6-6.1

Find the cross-correlation of the two functions …

X t  2cos 2  f t   and Yt  10sin2  f t  

Using the time average functions T 1  XY   lim xt  y t   dt  x t  y t   T  2T  T 1 T  XY   lim 2  cos2   f t    10 sin2   f  t     dt T   2T T 1 1 f    20  cos2   f t  sin2   f  t    dt XY 1  f 0 1 f 1    20  f  sin 2   f  2t   2  sin 2   f   dt XY    0 2 1 1 f f    10  f sin 2   f  2t   2  dt 10  f sin 2   f   dt XY       0 0 1 f f 1    10   cos2   f  2t   2 f 10  f sin2   f    dt XY 0  4   f 0 10    2        cos2   f     2   cos 2   f    2 10 sin 2   f  XY         4     f    10    cos4   2   f   2  cos 2   f   2 10 sin2   f  XY 4  10    cos2   f   2  cos 2   f   2 10 sin2   f  XY 4 

 XY    10sin2  f  

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 32 of 78 ECE 3800 Using the probabilistic functions

RXY    Ext yt  

RXY   E2 cos 2  f t   10sin2  f  t   

RXY   20 Ecos2  f t  sin2  f  t   

RXY   10 Esin 2  f  2t  2  f   2  sin2  f  

RXY   10sin 2  f  10 Esin2  f  2t  2  f   2 

From prior understanding of the uniform random phase ….

RXY    10sin2  f  

By the way, it is useful to have basic trig identities handy when dealing with this stuff … 1 1 sina sin b   cosa  b   cosa  b 2 2 1 1 cosa  cos b   cosa  b   cosa  b 2 2 1 1 sina  cos b  sina  b  sina  b 2 2 1 1 cosa sin b   sina  b  sina  b 2 2 and sin2a  2sina cosa cos2a  cosa2  sina2  2cosa2 1 as well as 1 sina 2   1 cos2a 2 1 cosa 2   1 cos2a 2

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 33 of 78 ECE 3800 Functions of a random variable and time

Example: X t  fn a,t where a is a wide-sense stationary ergodic process with a known pdf.

RXX t1 ,t2  RXX 0,   RXX    EX 0 X  

 R t ,t  R t,t   R   X t  X t   pdf a  da XX 1 2 XX  XX   

Exponential

X t  A exp  t u t where A is a uniformly distributed random variable A0, B . B 1 R t ,t  A exp  t  u t  A  exp  t  u t   dA XX 1 2  1 1 2 2 0 B

1 B R t ,t  exp  t  t  u max t ,t   A2 dA XX 1 2 1 2  1 2  B 0

B 2 R t ,t  exp  t  t umax t ,t  XX 1 2 1 2 1 2 3

For t1  0 and t2  

B 2 R 0,   exp , 0    XX 3

For t1  0 and t2   B 2 R 0,   exp ,   0 XX 3

Therefore B 2 R    exp  XX 3

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 34 of 78 ECE 3800 MATLAB Signal Processing Examples

Fig_6_2: Cross Correlation Rxy and Ryx

Run the various y waveforms. Chirp and Sinc are popular and interesting.

Fig_6_3: Auto Correlation random Gaussian noise

Fig_6_4: Auto Correlation smoothers random Gaussian noise

Fig_6_9v2: Sin wave correlation in noise

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 35 of 78 ECE 3800 Section 9.4 Classifications of Random Processes

Definition 9.4‐1.: Let X and Y be random processes.

(a) They are Uncorrelated if * * RXY t1 ,t2  EX t1 Y t2    X t1  Y t2  , for all t1 and t2

(b) They are Orthogonal if * RXY t1 ,t2  EX t1 Y t2   0, for all t1 and t2

(c) They are Independent if for all positive integers n, the nth-order CDF of X and Y factors. That is F x , y , x , y , , x , y ;t ,t , ,t XY 1 1 2 2  n n 1 2  n  FX x1 , x2 ,, xn ;t1 ,t2 ,,tn  FY y1 , y2 ,, yn ;t1 ,t2 ,,tn

Note that if two processes are uncorrelated and one of the is zero, they are orthogonal as well!

Stationarity

A random process is stationary when its statistics do not change with the continuous time . F x , x , , x ;t ,t , ,t X 1 2  n 1 2  n  FX x1 , x2 ,, xn ;t1  T,t2  T,,tn  T

Overall, the CDF and pdf do not change with absolute time. They may have time characteristics, as long as the elements are based on time differences and not absolute time.

FX x1 , x2 ;t1 ,t2  FX x1 , x2 ;t1  t2 ,0

f X x1 , x2 ;t1 ,t 2  f X x1 , x2 ;t1  t 2 ,0

This implies that * RXX t1 ,t2  EX t1  X t2   RXX t1  t2 ,0  RXX  ,0

Definition 9.4‐3.: Wide Sense Stationary

A random process is wide-sense stationary (WSS) when its mean and variance statistics do not change with the continuous time parameter. We also include the autocorrelation being a function of one variable …

* EX t   X t  RXX  , for      ,independednt of t

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 36 of 78 ECE 3800 Power Spectral Density

Definition 9.1‐1: PSD

Let Rxx(t) be an autocorrelation function for a WSS random process. The power spectral density is defined as the Fourier transform of the autocorrealtion function.  S XX w  RXX    RXX   exp  iw  d 

The inverse exists in the form of the inverse transform  1 R t  S w  exp iwt  dw XX 2  XX 

Properties:

1. Sxx(w) is purely real as Rxx(t) is conjugate symmetric

2. If X(t) is a real-valued WSS process, then Sxx(w) is an even function, as Rxx(t) is real and even.

3. Sxx(w)>= 0 for all w.

Wiener–Khinchin Theorem For WSS random processes, the autocorrelation function is time based and has a spectral decomposition given by the power spectral density.

Also see: http://en.wikipedia.org/wiki/Wiener%E2%80%93Khinchin_theorem

Why this is very important … the Fourier Transform of a “single instantiation” of a random process may be meaningless or even impossible to generate. But if the random process can be described in terms of the autocorrelation function (all ergodic, WSS processes), then the power spectral density can be defined.

I can then know what the expected frequency spectrum output looks like and I can design a system to keep the required frequencies and filters out the unneeded frequencies (e.g. noise and interference).

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 37 of 78 ECE 3800 Relation of Spectral Density to the Autocorrelation Function

For “the right” random processes, power spectral density is the Fourier Transform of the autocorrelation:  S XX w  RXX    EX t  X t   exp iw  d  For an ergodic process, we can use time-based processing to arrive at an equivalent result … T 1  XX   lim xt  x t   dt  x t  x t   T  2T  T T 1 EX t  X t    XX   lim xt  x t   dt T  2T  T   T   1   XX   EX t  X t   lim xt  x t   dt  exp  iw  d T  2T    T  T    1    XX   lim xt  x t   exp  iw  d  dt T  2T     T   T    1    XX   lim xt  x t   exp iw t   iwt  d  dt T  2T     T   T    1    XX   lim xt  exp  iwt xt   exp iw t   d  dt T  2T     T   T    1    XX   lim xt  exp  iwt  xt   exp iw t   d  dt T  2T     T  

If there exists X    X w 1 T  XX   lim xt  exp  iwt X w  dt T   2T T 1 T  XX   X w  lim xt  exp i  w t dt T   2T T 2  XX    X w X  w  X w

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 38 of 78 ECE 3800 Properties of the Fourier Transform:  X w  x    x  exp  iw  d 

For x(t) purely real  X w  x    x cos w  i sin w  d 

  X w  x    x  cos w  d  i   x   sin w  d  

  X w  EX w  i OX w   x   cos w  d  i   x  sin w  d  

  E X w   x   cos w  d and OX w   x  sin w  d  

Notice that:   E X w   x   cos w  d   x   cos  w  d  EX  w   

  OX w   x  sin w  d    x  sin  w  d  OX  w   

Therefore, the real part is symmetric and the imaginary part is anti-symmetric!

Note also, for real signals X  w  conjX w  X w*

X(w) is conjugate symmetric about the zero axis.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 39 of 78 ECE 3800 Relating this to a real autocorrelation function where RXX   RXX 

RXX    EX w i OX w

 RXX    RXX  cos w  i sin w  d 

 RXX    RXX t cos  wt  i sin  wt   dt 

  RXX    RXX t  cos wt  dt  i   RXX t sin wt  dt  

RXX    EX w i OX w

Since Rxx is symmetric, we must have that

RXX   RXX  and EX w i OX w  EX w i OX w

For this to be true,  i OX w  i OX w, which can only occur if the odd portion of the Fourier transform is zero! OX w  0 .

This provides information about the power spectral density,

S XX w  RXX    EX w

S XX w  EX w

S XX w  0

The power spectral density necessarily contains no phase information!

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 40 of 78 ECE 3800

Example 9.5‐3

Find the psd of the following autocorrelation function … of the random telegraph.

RXX   exp   , for   0

Find a good Fourier Transform Table … otherwise

 S w  R   exp  j  w  d XX  XX    

 S w  exp     exp  j  w  d XX    

 0 S w  exp    exp  j  w  d  exp    exp  j  w  d XX          0 

 0 S w  exp  j  w     d  exp  j  w    d XX       0 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 41 of 78 ECE 3800  0 exp j  w    exp j  w   S XX w    j  w  0  j  w  

exp  j  w    exp  j  w   0 S XX w       j  w    j  w  

exp j  w   0 expj  w           j  w   j  w  

1 1  j  w    j  w   S w    XX  j  w   j  w  j  w   j  w 

 2 2 S w   XX  w2  2 w2  2

For a=3

Figure 9.5-2 Plot of psd for exponential autocorrelation function.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 42 of 78 ECE 3800 Example 9.5‐4

Find the psd of the triangle autocorrelation function … autocorrelation of rect.

    RXX   tri  or RXX   1 ,   T T  T

T    S w  1   exp  j  w  d XX     T T 

0    T    S w  1  exp  j  w  d  1  exp  j  w  d XX       T T  0  T 

0 0  S w  exp  j  w  d   exp j  w  d XX   T T T T T    exp j  w  d    exp j  w  d 0 0 T

0 T exp j  w  exp j  w  S w   XX   j  w    j  w    T   0 0 1   exp j  w exp j  w       T  j  w  j  w 2   T T 1   exp j  w exp j  w       T  j  w  j  w 2   0

 1 exp j  wT  exp j  wT  1  S XX w          j  w  j  w    j  w  j  w 1  1 T  expj  wT expj  wT     2   2  T w  j  w w  1 T  exp j  wT exp j  wT 1     2  2  T   j  w w w 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 43 of 78 ECE 3800 expj  wT exp j  wT  S w   XX j  w j  w 1 T  expj  wT T exp j  wT       T  j  w j  w  1 2 1 expj  wT exp j  wT      2  2 2  T w T  w w 

2sinwT 2sinwT  2 1 2 coswT  S w       XX w w T w2 T w2

2 1 S w     1 coswT XX T w2

2  wT  2 sin  2 1  wT   2  S XX w    2  2 sin j    T  2 T w  2   wT     2 

Don’t you love the math ?!

Using a table is much faster and easier ….

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 44 of 78 ECE 3800 Deriving the Mean-Square Values from the Power Spectral Density

Using the Fourier transform relation between the Autocorrelation and PSD

 S XX w   RXX   exp  iw  d 

 1 R t  S w  exp iwt  dw XX 2  XX 

The mean squared value of a random process is equal to the 0th lag of the autocorrelation

  2 1 1 EX  RXX 0  S XX w  exp iw  0  dw  S XX w  dw 2  2   

  2 EX  RXX 0   S XX f  exp i2f  0  dw   S XX f  df  

Therefore, to find the second moment, integrate the PSD over all frequencies.

As a note, since the PSD is real and symmetric, the integral can be performed as

 2 1 EX  RXX 0  2  S XX w  dw 2  0

 2 EX  RXX 0  2 S XX f  df 0

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 45 of 78 ECE 3800 Converting between Autocorrelation and Power Spectral Density

Using the properties of the functions we can actually different variations of Transforms!

The power spectral density as a function is always  real,  positive,  and an even function in w/f.

You can convert between the domains using any of the following …

The Fourier Transform in w

 S XX w   RXX   exp  iw  d 

 1 R t  S w  exp iwt  dw XX 2  XX 

The Fourier Transform in f

 S XX f   RXX   exp  i2f  d 

 RXX t   S XX f  exp i2ft  df 

The 2-sided (the jw axis of the s-plane)

 S XX s   RXX   exp  s  d 

j 1 R t  S s  exp st  ds XX j2  XX  j

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 46 of 78 ECE 3800 Notes on using the Laplace Transform

ECE 3100 and ECE 3710 stuff …

(1) When converting from the s-domain to the use:

s  jw or w   js

(2) As an even function, the PSD may be expected to have a polynomial form as: (Hint: no odd powers of w in the numerator or denominator!)

w2n  a w2n2  a w2n4   a w2  a S w  S 2n2 2n4  2 0 XX 0 2m 2m2 2m4 2 w  b2m2w  b2m4w  b2w  b0

This can be factored and expressed as:

cs c s S w   Ts T  s  XX ds  d  s 

To compute the autocorrelation function for 0   use a partial fraction expansion such that

gs g s S w   XX ds d s

and solve for   0 using the LHP poles and zeros as

1 j gs R t   exp st  ds, for t  0 XX    2  j ds

for determining   0 , use the RHP expansion, replace –s with s, perform the Laplace transform and replace t with –t.

Another hint, once you have   0 , make the and skip the math R XX t  RXX  t .

cs  c s Final Note … for S w   T s T  s  XX ds  d  s 

If you “define” T(s) as all the LHP poles and zeros and T(-s) as all the RHP poles and zeros, then T(s) will represent a (1) causal, (2) minimum phase, and (3) stable filter (if there are no poles on the jw axis).

You give me a power spectral density and I can design a filter that passes “signal energy” and filters out as much of the rest as possible!

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 47 of 78 ECE 3800 Example: Inverse Laplace Transform. 2 A2   2 A2   S w  X  XX 2 2 2     w 1 2    w   X 

Substitute s for w 2  A2   2  A2   S s   XX 2 2   s    s   s Partial fraction expansion 2 k0 k1 k0    s k1    s 2  A   S XX s       s   s   s    s   s    s

 k0  k1  s  0  k0  k1 k  k    2  A2    2k  2  A2 0 1 0

A2 A2 S s   XX   s   s

Taking the LHP Laplace Transform  A2  L   A exp t for t  0   s   Taking the RHP with –s and then –t.  A2  L   A2  exp t  A2  exp   t  A2  exp t for t  0    s   Combining we have

R   A2  exp    t XX  

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 48 of 78 ECE 3800 7-6.3 A stationary random process has a spectral density of. 5, 10  w  20 S XX w   0, else

(a) Find the mean-square value of the process. 1  2  R 0   S w  dw   S w  dw XX   XX   XX  2   2  0

1 20 1 10 1 20 R 0   5 dw   5 dw  2   5 dw XX     2  10 2  20 2  10

10 20 10 50 R 0   w   20 10  XX 2  10 2  

(b) Find the auto-correlation function the process. 1  R t   S w  exp j  w t  dw XX   XX    2  

5  20 10  R t   exp j  w t  dw  exp j  w t  dw XX      2   10 20 

20 10 5  expj  wt expj  wt  R t     XX 2   j t j t   10 20  5  expj  20  t exp j 10 t exp j  10 t expj  20 t  RXX t       2   j  t j t j t j t  5 expj  20 t exp j  20 t exp j 10 t exp j 10 t  R t       XX      2   j t j t   j t j t 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 49 of 78 ECE 3800 5  2  j  sin20 t 2  j sin10 t 5 RXX t       sin20  t  sin 10 t 2   j t j t   t

5   20 10   20 10  10 RXX t  2 sin t   cos  t   sin5 t  cos 15 t   t   2   2   t 50 sin5t 50  5 t  RXX t    cos15 t  sinc   cos15 t  5t    

(c) Find the value of the auto-correlation function at t=0.. 50 sin5  0 50  5  0  RXX 0    cos15  0  sinc   cos15  0  5 0     50 50 R 0   1  1   1  1 XX   50 R 0  XX 

It must produce the same result!

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 50 of 78 ECE 3800 White Noise

Noise is inherently defined as a random process. You may be familiar with “thermal” noise, based on the energy of an atom and the mean-free path that it can travel.  As a random process, whenever “white noise” is measured, the values are uncorrelated with each other, not matter how close together the samples are taken in time.  Further, we envision “white noise” as containing all spectral content, with no explicit peaks or valleys in the power spectral density.

As a result, we define “White Noise” as RXX    S0  t N S w  S  0 XX 0 2

This is an approximation or simplification because the area of the power spectral density is infinite!

Nominally, noise is defined within a bandwidth to describe the power. For example,

Thermal noise at the input of a receiver is defined in terms of kT, Boltzmann’s constant times absolute temperature, in terms of Watts/Hz. Thus there is kT Watts of noise power in every Hz of bandwidth. For communications, this is equivalent to –174 dBm/Hz or –204 dBW/Hz.

For typical applications, we are interested in Band-Limited White Noise where

 N 0 S0  f  W S XX w   2  0 W  f

The equivalent noise power is then:

W 2 N0 EX  RXX 0  S0  dw  2 W  S0  2 W   N0 W  2 W

For communications, we use kTB where W=B and N0=kT.

How much noise power, in dBm, would I say that there is in a 1 MHz bandwidth?

dBkTB  dBkT  dBB  174  60  114 dBm

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 51 of 78 ECE 3800 Receiver Sensitivity

What does it mean when you buy a radio receiver?

For a great receiver (spectrum analyzer grade), assume a 200 kHz FM radio bandwidth.

Noise Power kT -174. dBm/Hz

Equivalent Noise Bandwidth B 53. dB Hz

Receiver Noise Figure NF 10. dB

Signal Detection Threshold D 8. dB

Minimum Detectable Signal MDS -103. dBm

FM radio stations can transmit up to 1 Megawatt  +90 dBm

Why doesn’t your receiver get blasted? Path loss, distance, higher noise figure, receiving antenna inefficiency, etc. https://en.wikipedia.org/wiki/Path_loss

https://en.wikipedia.org/wiki/Friis_transmission_equation

But notice that your commercial; receiver is in microvolts, where 2.0 uV is very good. Power into 50 ohms is V^2/R or 8e-14 W = -130.97 dBW  -101 dBm.

-103 dBm  1.6 uV

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 52 of 78 ECE 3800 FM Radio Design Diagram – Something you may encounter in the future

 Assume input to be digitized by a 12-bit ADC with 60 dB SNR

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 53 of 78 ECE 3800 Band Limited White Noise N S  0 f  W  0 2 S XX w   0 W  f

The equivalent noise power is then: W 2 EX  RXX 0   S0  dw  2 W  S0 W But what about the autocorrelation? W RXX t   S0  exp i2ft  df W

W expi2ft  expi2Wt exp i2Wt  RXX t  S0     S0      i2t W  i2t i2t 

2 i sini2Wt R t  S  XX 0 i2t

xt For sincxt  xt

RXX t  2 W  S0 sinc2Wt

Using the concept of correlation, for what values will the autocorrelation be zero? (At these delays in time, sampled would be uncorrelated with previous samples!)

2Wt  k k t  for k  1, 2, 2W 

Sampling at 1/2W seems to be a good idea, but isn’t that the Nyquist rate!!

Also note, noise passed through a filter becomes band-limited, and the narrower the filter the smaller the noise power … but the wider is the sinc autocorrelation function.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 54 of 78 ECE 3800 Noise and Filtered Noise Matlab Simulation

Based on Cooper and McGillem HW Problem 6-4.6.

x=randn(N,1); % zero mean, unit power random signal [b,a] = butter(4,20/500); y=filter(b,a,x); % applying a digital filter y=y/std(y); % normalizing the output power

Rxx=xcorr(x)/(N+1); Ryy=xcorr(y)/(N+1);

DFTx = fftshift(fft(x))/N; DFTy = fftshift(fft(y))/N;

DFTRxx = fftshift(fft(Rxx,2*N))/N; DFTRyy = fftshift(fft(Ryy,2*N))/N;

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 55 of 78 ECE 3800

Pink_Noise … If we call constant at all frequencies white noise, then noise in a limited low frequency band is sometimes called pink noise.

OK. I’m getting ahead …. we just did this.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 56 of 78 ECE 3800 The Cross-Spectral Density

Why not form the power spectral response of the cross-correlation function?

The Fourier Transform in w

  S XY w   RXY   exp  iw  d and SYX w   RYX   exp  iw  d  

  1 1 R t  S w  exp iwt  dw and R t  S w  exp iwt  dw XY 2  XY YX 2  YX  

Properties of the functions

S XY w  conjSYX w

Since the cross-correlation is real,  the real portion of the spectrum is even  the imaginary portion of the spectrum is odd

There are no other important (assumed) properties to describe

Note: the trick using the Laplace transform to form the positive and negative portions of the “time-based” cross-correlation is required to determine the correct “inverse transform” of the “Cross” Power Spectral Density.

OK … Time to talk about linear transfer functions … filters!.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 57 of 78 ECE 3800 Section 9.3 Continuous-Time Linear Systems with Random Inputs

Linear system requirements:

Definition 9.3-1 Let x1(t) and x2(t) be two deterministic time functions and let a1 and a2 be two scalar constants. Let the linear system be described by the operator equation yt  Lxt then the system is linear if “linear super-position holds”

La1  x1 t  a2  x2 t  a1  Lx1 t a2  Lx2 t for all admissible functions x1 and x2 and all scalars a1 and a2.

For x(t), a random process, y(t) will also be a random process.

Linear transformation of signals: in the yt  ht xt

xt ht yt 

Linear transformation of signals: multiplication in the Laplace domain

Ys  Hs X s

X s H s Ys

The convolution (applying a causal filter)  yt   x t    h   d 0 or t yt   h t    x   d  Where for physical realize-ability, causality, and constraints we require  ht  0 for t  0 and  ht  dt   

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 58 of 78 ECE 3800 Example: Applying a linear filter to a random process ht  5 exp 3t for t  0

X t  M  4  cos2t   where M and  are independent random variables,  uniformly distributed [0,2].

We can perform the filter function since an explicit formula for the random process is known. t yt   h t    x   d  t yt  5 exp 3 t    M  4  cos 2   d  t t yt  5 M   exp 3 t    d  20   exp 3 t    cos 2   d   exp 3t   t yt  5  M  3  t 10   exp 3t    exp i2  i  exp  i2  i  d  t 5 M  exp 3t    exp i2  i exp 3t    exp  i2  i  yt  10    3  3  i2 3  i2   5 M  expi2t  i  exp i2t  i  yt  10    3  3  i2 3  i2  5  M  3  i2 expi2t  i  3  i2 exp i2t  i  yt  10   3  9  4  5  M 20 yt    3 cos2t    2 sin 2t   3 13

Linear filtering will change the magnitude and phase of sinusoidal signals (DC too!).

X t  M  4  cos2t  

5 5 yt   M   4  cos2t    ,   33.69 3 13

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 59 of 78 ECE 3800 Expected value operator with linear systems

For a causal linear system we would have  yt   x t    h   d 0 and taking the expected value   Eyt  E xt    h   d  0   Eyt   Ext    h  d 0  Eyt   t    h   d 0 For x(t) WSS   Eyt     h  d     h   d 0 0

Notice the condition for a physically realizable system!

The coherent gain of a filter is defined as:

 h  h t  dt  H 0 gain     0

Therefore, EY t  EX  hgain  EX  H 0

 Note that: H f   h t  exp  i  2  f t  dt 

 For a causal filter H f   h t  exp  i  2  f  t  dt 0

 At f=0 H 0   h t  dt 0

And Eyt    H 0

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 60 of 78 ECE 3800 What about a cross‐correlation? (Converting an auto‐correlation to cross‐correlation)

For a linear system we would have  yt   x t    h   d 

And performing a cross-correlation (assuming real R.V. and processing)    E x t  y t  E x t  x t    h   d 1 2  1  2          E x t  y t  E x t  x t    h   d 1 2   1 2       E x t  y t  E x t  x t    h   d 1 2  1 2     E x t  y t  R t ,t    h   d 1 2  XX 1 2 

For x(t) WSS  E x t  y t   R   R     h   d    XY  XX    

Ext  yt    RXY    RXX   h 

What about the other way … YX instead of XY

And performing a cross-correlation (assuming real R.V. and processing)    E y t  x t  E x t    h   d  x t 1 2   1 2       E y t  x t  E x t    x t  h   d 1 2   1 2     E y t  x t  E x t    x t  h   d 1 2  1 2    E y t  x t  R t  ,t  h   d 1 2  XX 1 2 

For x(t) WSS … see the next page

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 61 of 78 ECE 3800 For x(t) WSS  E y t  x t   R   R t   t    h   d    YX  XX      E y t  x t   R   R     h   d    YX  XX     Perform a change of variable for lamba to “-kappa” (assuming h(t) is real, see text for complex0  E y t  x t   R   R     h    d    YX  XX      Therefore  E y t  x t   R   R     h    d    YX  XX     

Eyt  xt    RYX    RXX   h 

What about the auto‐correlation of y(t)?

And performing an auto-correlation (assuming real R.V. and processing)     E y t  y t  R t ,t  E x t    h   d  x t    h   d 1 2 YY 1 2   1 1 1 1  2 2 2 2         E y t  y t  R t ,t  E  x t    x t    h   d  h   d 1 2 YY 1 2   1 1 2 2 2 2 1 1       E y t  y t  R t ,t  E x t    x t    h   d  h   d 1 2 YY 1 2   1 1 2 2 2 2 1 1    E y t  y t  R t ,t  R t   ,t    h   d  h   d 1 2 YY 1 2  XX 1 1 2 2 2 2 1 1 

For x(t) WSS   E y t  y t   R   R       h   h   d  d    YY  XX  2 1  1 2 2 1      E y t  y t   R   R       h   d  h   d    YY  XX  1  2 2 2  1 1  

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 62 of 78 ECE 3800 The output autocorrelation can also be defined in terms of the cross-correlation as  E y t  y t   R   R     h   d    YY  XY 1  1  1 

Eyt  yt    RYY    RXY   h 

The cross-correlation can be used to determine the output auto-correlation!

Continue in this concept, the cross correlation is also a convolution. Therefore,

Eyt  y t   RYY    RXX   h  h 

If h(t) is complex, the term in h(-t) must be a conjugate.

The Value at a System Output

Based on the output autocorrelation formula   E y t 2  R 0  R     h   h   d  d  YY  XX 2 1  1  2  2 1      E y t 2  R 0  h   R     h   d  d  YY  2  XX 2 1  1  1  2     E y t 2  R 0  h   R   h   d  YY  XX   

Based on the input to output cross-correlation formula  E y t  y t   R   R   h   d    YY  XY   

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 63 of 78 ECE 3800 Example: White Noise Inputs to a causal filter

N Let R t  0  t XX 2

  E Y t 2  R 0  h   R     h   d  d  YY  1  XX 1 2  2  2  1 0  0 

 N  E Y t 2  R 0  h   0      h   d  d  YY  1  1 2 2 2  1 0  0 2 

 2 N 0 EY t  RYY 0   h1 h 1  d1 2  0

 2 N 0 2 EY t  RYY 0   h1  d1 2  0 For a white noise process, the mean squared (or 2nd moment) is proportional to the filter power.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 64 of 78 ECE 3800 Example: RC filter

The RC low-pass filter 1 1 1 H s  s C   R C 1 1 s  R C s  1 R  R C s C Inverse Laplace Transform 1   t  ht   exp  ut R  C  R C  Coherent Gain of the RC Filter  h  h t  dt  H 0 gain     0

 1   t  h   exp  dt gain    0 R  C  R  C 

   t    t  exp  exp  1  R C  1 R C h   0     gain R C 1 R C 1 R C R C       0  hgain  1 exp   exp   1   R C   R C 

If driven by a white noise process, what is the output power? N  EY t 2  0  h 2  d 2  0  2 2 N  1     EY t  0     exp   d 2 R C  R C  0

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 65 of 78 ECE 3800 2  2 N  1    2    EY t  0    exp   d 2  R C    R C  0    2    2 exp  2 N  1   R C  EY t  0    0 2  R C   2 R C

2 N 0 1 1 EY t    1  N 0  2  2  R  C 4  R  C

Comparing Noise Power in the filter bandwidth

N 1 W N B Power in band-limited noise E N 2  0  1 dw  0  1 df  W   1  1 2 2 W 2 B

2 N 0 2W W ENW    N 0   N 0  B where W is in rad/sec and B in Hz 2 2 2

2 1 The noise power in an RC EYRC  N0  4RC

For an equivalent band-limited noise process to have the same power (assume a brick wall filter)

2 N0W 1 2 ENW   N0   EYRC  2 4RC W 1 N   N  0 2 0 4RC  Therefore W  2RC W 1 or  B  where B is in Hz 2 4RC

Note that the nominal -3dB band (½ power) of an RC network is

1 1 W  or B  3dB RC 3dB 2   RC

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 66 of 78 ECE 3800 Comparing these two, the equivalent noise bandwidth is greater than the –3dB bandwidth by

 2 W  W or B   B 2 3dB 4 3dB

Note: B in Hz and W in rad/sec.

1

0.8

0.6

0.4

0.2

0

-0.2 0 0.5 1 1.5 2 2.5 3 3.5 4

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 67 of 78 ECE 3800 The power spectral density output of linear systems

The first cross-spectral density

RXY    RXX   h 

 S XY w   RXY   exp  iw  d 

 S w  R   h   exp  iw  d XY  XX    

Using convolution identities of the Fourier Transform (if you want the proof it isn’t bad, just tedious)

S XY w  S XX w Hw

The second cross-spectral density

RYX    RXX   h 

 SYX w   RYX   exp  iw  d 

 S w  R   h  *  exp  iw  d YX  XX      

Using convolution identities of the Fourier Transform (if you want the proof it isn’t bad, just tedious)

* SYX w  S XX w H w

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 68 of 78 ECE 3800 The output power spectral density becomes

RYY    RXX   h  h 

 S w  R   exp  iw  d YY  YY    

 S w  R   h   h   exp  iw  d YY  XX      

Using convolution identities of the Fourier Transform

* SYY w  S XX w H w H w

2 SYY w  S XX w H w

This is a very significant result that provides a similar advantage for the power spectral density computation as the Fourier transform does for the convolution.

This leads to the following table.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 69 of 78 ECE 3800 Additional Topics

System analysis with a noise input …

r t  x t ht  yt  n t  Where the signal of interest is x(t), n(t) is a noise or interfering process. The signal plus noise is r(t) and the received system output is y(t) which has been filtered.

We have rt  xt nt

Assuming WSS with x and n independent and n zero mean

RRR   Er t  r t   Ext nt xt   nt  

RRR   Ex t  x t    xt nt    nt xt    nt n t   

RRR   RXX   Ext nt   Ent xt   RNN 

RRR   RXX   2  X   N  RNN  

RRR    RXX   RNN  

And then

* RYY   R XX   RNN   h  h 

RYY   RXX   h  h  RNN   h  h 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 70 of 78 ECE 3800 Signal‐to‐Noise‐Ratio SNR (always done for powers)

The signal-to-noise ratio is the power ratio of the signal power to the noise power.

P 2 Signal EX t  RXX 0 The input SNR is defined as  2  PNoise ENt  RNN 0

P 2 Signal EX t ht  RXX   h  h  0 The output SNR is defined as  2  PNoise ENt  h t RNN   h   h   0

For a white noise process and assuming the “filter” does not change the input signal (unity gain), but strictly reduces the noise power by the equivalent noise bandwidth of the filter.

  1 2 N We have R 0  S w  H w  dw  0  h  2  d YY   XX     2  2 

With appropriate filtering with unity gain where the signal exists and bandwidth reduction for the noise 1  R 0  S w  dw  N  B YY   XX  0 EQ 2 

or RYY 0  RXX 0 N 0  BEQ

PSignal R 0 The output SNR is defined as  XX PNoise N 0  BEQ

The narrower the filter applied prior to signal processing, the greater the SNR of the signal! Therefore, always apply an analog filter prior to processing the signal of interest!

From the definition of band-limited noise power, the equation for the equivalent noise bandwidth (performed as a previous example)

 N  E N t  h t 2  R 0  h  2  d  0  h  2  d   NN  1  1  1 1  2 

  1 1 2 B   h t 2  dt   H f  df EQ     2  2 

 Under the unity gain condition 1   ht  dt 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 71 of 78 ECE 3800 Otherwise, the equivalent noise bandwidth can be defined as

 1 2 2  B   H f  df EQ 2  maxH f 

For a real, low pass filter this simplifies to

 1 2 2  B   H f  df EQ 2  H 0 

Using Parseval’s Theorem

   2 1 2 2  ht  dt   H w  dw   H f  df  2  

   ht 2  dt  ht 2  dt   2  BEQ  2  2 H 0      ht  dt  

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 72 of 78 ECE 3800 Examples of Linear System Frequency‐Domain Analysis

Noise in a linear feedback system loop. Ns

A X s s s 1 Ys

1

Linear superposition of X to Y and N to Y.

A Y s  X s  Y s  Ns s  s 1

 A  A Y s  1    X s  N s  s  s 1  s  s 1

s 2  s  A A Y s      X s  N s  s  s 1  s  s 1

A s 2  s Y s   X s   Ns s 2  s  A s 2  s  A

There are effectively two filters, one applied to X and a second apply to N.

A s 2  s H s  and H s  X s 2  s  A N s 2  s  A

Y s  H X s X s H N s Ns

Generic definition of output Power Spectral Density:

2 2 SYY w  H X w  S XX w H N w  S NN w

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 73 of 78 ECE 3800 Change in the input to output signal to noise ratio.

 S w  dw  XX   SNRIn   S w  dw  NN  

  H w 2  S w  dw H 0 2  S w  X  XX  X   XX    SNROut     2 N 2 H w  S w  dw 0 H w  dw  N  NN   N   2 

  S w S w  XX   XX    SNROut    2 N H w N 0  BEQ 0 N  dw 2  2  H X 0

Where

 H w 2 B  N  dw EQ  2  H 0 X If the noise is added at the x signal input, the expected definition of noise equivalent bandwidth results.

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 74 of 78 ECE 3800 Systems that Maximize Signal-to-Noise Ratio

2 PSignal E st SNR is defined as    PNoise N0  BEQ

Define for an input signal st nt

Define for a filtered output signal so t no t

For a linear system, we have:  so t  no t   h  s t    n t    d 0

The input SNR can be describe as

2 PSignal E st SNR     in 2 PNoise Ent 

The output SNR can be described as

2 2 PSignal E s t E s t SNR    o    o  out P 2 N  B Noise Eno t o EQ

2        E h  s t    d       0   SNR    out  1 N   ht 2  dt o 2  0

Using Schwartz’s Inequality the numerator becomes

2             2   2  E  h  s t    d   E  h   d   E  s t    d   0   0   0 

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 75 of 78 ECE 3800 Applying this inequality, an SNR inequality can be defined as

     2   2   h  d   E s t    d        SNR   0   0  out  1 N   ht 2  dt o 2  0

Canceling the filter terms, we have   2  2  SNRout   E st    d  No     0 

To achieve the maximum SNR, the equality condition of Schwartz’s Inequality must hold, or

2            2   2   h  s t    d    h   d   s t    d            0   0   0 

This condition can be met for h  K  st  u where K is an arbitrary gain constant.

The desired impulse response is simply the time inverse of the signal waveform at time t, a fixed moment chosen for optimality. If this is done, the maximum filter power (with K=1) can be computed as   t  h 2  d   s t   2  d   s  2  d   t 0 0 

2 maxSNRout  t No

This filter concept is called a .

If you wanted to detect a burst waveform that has been transmitted, to maximize the received SNR, the receiving filter should be the time inverse of the signal transmitted!

Note and caution: when using such a filter, the received signal maximum SNR will occur when the signal and convolved filter perfectly overlap. This moment in time occurs when the “complete” burst has been received by the system. If measuring the time-of-flight of the burst, the moment is exactly the filter length longer than the time-of-flight. (Think about where the leading edge of the signal-of-interest is when transmitted, when first received, and when fully present in the filter).

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 76 of 78 ECE 3800 The Matched Filter

Wikipedia: https://en.wikipedia.org/wiki/Matched_filter

“In signal processing, a matched filter is obtained by correlating a known signal, or template, with an unknown signal to detect the presence of the template in the unknown signal.[1][2] This is equivalent to convolving the unknown signal with a conjugated time-reversed version of the template. The matched filter is the optimal linear filter for maximizing the signal to noise ratio (SNR) in the presence of additive stochastic noise. Matched filters are commonly used in radar, in which a known signal is sent out, and the reflected signal is examined for common elements of the out-going signal. Pulse compression is an example of matched filtering. It is so called because impulse response is matched to input pulse signals. Two-dimensional matched filters are commonly used in image processing, e.g., to improve SNR for X-ray. Matched filtering is a demodulation technique with LTI (linear time ) filters to maximize SNR.[3].”

Applications:

 Radar  Sonar  Pulse Compression  Digital Communications (Correlation detectors)  GPS pseudo-random sequence correlation

If you are looking for a signal, maximize the output of the filter when the signal is input! You will have a matched filter!

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 77 of 78 ECE 3800 and Windowing Data

We can window the data prior to autocorrelation.  this will cause a change in the energy that must be compensated 1 T AvgEnergy   xt  w t 2  dt T 0 1 T 1 T EAvgEnergy    Ext 2  wt 2  dt  Ext 2    wt 2  dt T 0 T 0

For a rectangular window, this becomes EAvgEnergy  Ext2 

If an alternate window is used, a power scaling will be necessary to maintain the same estimated energy.

For a given window, function we can apply: 1 T scale    wt 2  dt T 0

So that T wt 1 2 w t   w t  w t  dt scale     scale T 0

This also applies to the sampled data computations where N 1 wn 1 2 wscale n   wn   wm scale N m0

Now, multiple smaller results may be processed to generate an average.

The spectral resolution (number of frequency bins) necessarily decreases, but now multiple spectra and can be averaged together based on “unique” time sample sets of the data.

It is useful for observing time varying phenomena and capturing the values when the desired signal is present.

See WaterFallDemo.m for a spectral ….

Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 78 of 78 ECE 3800