Henry Stark and John W. Woods, Probability, Statistics, and Random Variables for Engineers, 4th ed., Pearson Education Inc., 2012. ISBN: 978-0-13-231123-6 Chapter 9 Random Processes
Sections 9.1 Basic Definitions 544 9.2 Some Important Random Processes 548 9.3 Continuous-Time Linear Systems with Random Inputs 572 White Noise 577 9.4 Some Useful Classifications of Random Processes 578 Stationarity 579 9.5 Wide-Sense Stationary Processes and LSI Systems 581 Wide-Sense Stationary Case 582 Power Spectral Density 584 An Interpretation of the Power Spectral Density 586 More on White Noise 590 Stationary Processes and Differential Equations 596 9.6 Periodic and Cyclostationary Processes 600 9.7 Vector Processes and State Equations 606 State Equations 608 Summary 611 Problems 611 References 633
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 1 of 78 ECE 3800 9.1 Basic Concepts
A random process is a collection of time functions and an associated probability description.
When a continuous or discrete or mixed process in time/space can be describe mathematically as a function containing one or more random variables.
A sinusoidal waveform with a random amplitude. A sinusoidal waveform with a random phase. A sequence of digital symbols, each taking on a random value for a defined time period (e.g. amplitude, phase, frequency). A random walk (2-D or 3-D movement of a particle)
The entire collection of possible time functions is an ensemble, designated as xt , where one particular member of the ensemble, designated as xt, is a sample function of the ensemble. In general only one sample function of a random process can be observed!
Think of: X t Asinwt , 0 2 where A and w are known constants.
Note that once a sample has been observed …
x1 t1 Asinwt1 the function is known for all time, t.
Note that, xt2 is a second time sample of the same random process and does not provide any “new information” about the value of the random variable.
x1 t2 Asinwt2
There are many similar ensembles in engineering, where the sample function, once known, provides a continuing solution. In many cases, an entire system design approach is based on either assuming that randomness remains or is removed once actual measurements are taken!
For example, in communications there is a significant difference between coherent (phase and frequency) demodulation and non-coherent (i.e. unknown starting phase) demodulation.
On the other hand, another measurement in a different environment might measure
x2 t1 A2 sinwt1 2 In this “space” the random variables could take on other values within the defined ranges. Thus an entire “ensemble” of possibilities may exist based on the random variables defined in the random process.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 2 of 78 ECE 3800 For example, assume that there is a known AM signal transmitted:
st 1 b Atsinwt
at an undetermined distance the signal is received as
yt 1 b At sinwt , 0 2
The received signal is mixed and low pass filtered …
xt h t y t cos wt ht 1 b Atsinwt coswt,0 2
xt h t y t cos wt ht 1 b At 0.5sin2 wt sin ,0 2
If the filter removes the 2wt term, we have
1 b At xt h t y t cos w t sin , 0 2 2
Notice that based on the value of the random variable, the output can change significantly! From producing no output signal, ( 0, ), to having the output be positive or negative ( 0to or to 2 ). P.S. This is not how you perform non-coherent AM demodulation.
To perform coherent AM demodulation, all I need to do is measured the value of the random
variable and use it to insure that the output is a maximum (i.e. mix with coswt m , where.
m t1
Note: the phase is a function of frequency, time, and distance from the transmitter.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 3 of 78 ECE 3800 From our textbook
Random Stochastic Sequence
Definition 8.1-1. Let ,,P be a probability space. Let . Let Xn,be a mapping of the sample space into a space of complex-valued sequences on some index set Z. If, for each fixed integer n Z , Xn, is a random variable, then Xn, is a ransom (stochastic) sequence. The index set Z is all integers, n , padded with zeros if necessary,
Definition 9.1-1. Let ,,P be a probability space. then define a mapping of X from the sample space to a space of continuous time functions. The elements in this space will be called sample functions. This mapping is called a random process if at each fixed time the mapping is a random variable, that is, Xt, for each fixed t on the real line t .
Example sets of random sequence.
Figure 8.1-1 Illustration of the concept of random sequence X(n,ζ), where the ζ domain (i.e., the sample space Ω) consists of just ten values. (Samples connected only for plot.)
Example sets of random process.
Figure 9.1-1 A random process for a continuous sample space Ω = [0,10].
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 4 of 78 ECE 3800 Example 9.1-2
Separable random process may be constructed by combining a deterministic sequence with one or more random variables.
The classic example already shown is a sinusoid with random amplitude and phase:
X t, A sin2 f 0 t
Where the amplitude and phase are R.V. defined based on the probability space selected.
Example 9.1‐3
A random process used to model a continuous sequence of random communication symbols. X t An pt Tn n
In a communication class, Dr. Bazuin would typically use the following
X t An p t n T , for pt non zero for 0 t k T n
Here An is the amplitude and phase of a complex communication symbol and p(t) is the deterministic time function, the simplest of which is a rectangular pulse in time.
This can be used to describe a wide range of digital communication systems, including; Phase- Shift Keyed (PSK) or Quadrature Amplitude Modulation (QAM) communication signals.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 5 of 78 ECE 3800 The application of the Expected Value Operator
Moments play an important role and, for Ergodic Processes, they can be estimated from a single process in time of the infinite number that may be possible.
Therefore,
X t EX t and the correlation functions (auto- and cross-correlation) * RXX t1,t2 EX t1 X t2
* RXY t1,t2 EX t1 Yt2 and the covariance functions (auto- and cross-correlation) * K XX t1,t2 EX t1 X t1 X t2 X t2
* K XY t1,t2 EX t1 X t1 Yt2 Y t2 with * K XX t1,t2 RXX t1,t2 X t1 X t2
Note that the variance can be computed from the auto-covariance as * 2 K XX t,t EX t X t X t X t X t and the “power” function can be computed from the auto-correlation
* 2 RXX t,t EX t X t EX t
2 2 2 For real X(t) RXX t,t EX t X t X t
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 6 of 78 ECE 3800 Example 9.1‐5 Auto‐correlation of a sinusoid with random phase
Think of: X t Asinwt , where A and w are known constants. And theta is a uniform pdf covering the unit circle.
The mean is computed as
X t EX t EAsinwt
X t EX t A Esinwt 1 t E X t A sin wt d X 2
A t EX t coswt X 2 A A t EX t coswt cos wt 0 0 X 2 2 ( What would happen if 0 instead? )
The auto-correlation is computed as
* * RXX t1,t2 EX t1 X t2 EAsinwt1 Asinwt2 R t ,t E X t X t * A2 E 1 cos w t t 1 cos w t t 2 XX 1 2 1 2 2 1 2 2 1 2 A2 A2 R t ,t cosw t t Ecosw t t 2 XX 1 2 2 1 2 2 1 2 A2 A2 R t ,t cosw t t 0 cosw t t XX 1 2 2 1 2 2 1 2 ( This works if 0 instead. )
Note that if A was a random variable (independent of phase) we would have … EA2 EA2 R t ,t cosw t t R cosw XX 1 2 2 1 2 XX 2
and we would still have EA t EX t 0 0 X 2
Note: this Random Process is Wide-Sense stationary (mean and variance not a function of time)
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 7 of 78 ECE 3800 Definition 9.1‐3
All correlation and covariance functions are positive semidefinite.
All auto-correlation functions are diagonal dominate.
Using the Cauchy-Schwartz Inequality
RXX t1 ,t2 RXX t1 ,t1 RXX t2 ,t2 which for a WSS random process becomes
RXX RXX 0
Additional Properties for real, WSS random processes.
2 2 RXX 0 X X
max RXX RXX 0
RXX RXX
For signals that are the sum of independent random variable, the autocorrelation is the sum of the individual autocorrelation functions.
W t X t Yt
RWW RXX RYY 2 X Y
If X is ergodic and zero mean and has no periodic component, then
lim RXX 0
Interpretation of WSS autocorrelation …
The statistical (or probabilistic) similarity of future (or past) samples of a random process to other samples of the process for an ergodic random process.
How similar is a time shifted version of a function to itself?
Nominal definition of ergodicity … the time base statistics are equivalent to the probabilistic based statistics of a stationary random sequence or process.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 8 of 78 ECE 3800 For the autocorrelation defined as:
RXX t1,t2 EX1X 2 dx1 dx2 x1x2 f x1, x2
For WSS processes:
RXX t1,t2 EX tX t RXX
If the process is ergodic, the time average is equivalent to the probabilistic expectation, or
T 1 XX lim xt x t dt x t x t T 2T T and
XX RXX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 9 of 78 ECE 3800 A strange autocorrelation
A random process has a sample function of the form
A, 0 t 1 X t 0, else where A is a random variable that is uniformly distributed from 0 to 10.
Find the autocorrelation of the process.
1 f a , for 0 a 10 10
Using
RXX t1 ,t2 EX t1 X t2
2 RXX t1 ,t2 EA , for 0 t1 ,t2 1
10 1 R t ,t a 2 da, for 0 t ,t 1 XX 1 2 1 2 0 10
10 a 3 RXX t1 ,t2 , for 0 t1 ,t2 1 30 0
1000 100 R t ,t , for 0 t ,t 1 XX 1 2 30 3 1 2
RXX t1 ,t2 0, for t1 0, 1 t1 ,or t2 0, 1 t2
Not WSS as it is a function of time!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 10 of 78 ECE 3800 Example: xt Asin2 f t for A a uniformly distributed random variable A 2,2
RXX t1 ,t2 EX t1 X t2 EAsin2 f t1 Asin2 f t2
2 1 RXX t1 ,t2 EX t1 X t2 E A cos2 f t1 t2 cos2 f t1 t2 2
1 R t ,t EA2 cos2 f t t cos2 f t t XX 1 2 2 1 2 1 2
for t2 t1
1 2 R t ,t cos2 f cos2 f t t XX 1 2 2 12 1 2
16 R t ,t cos2 f cos2 f t t XX 1 2 24 1 2
A non-stationary process! It is still a function of both time variables!
The time based formulation: T 1 XX lim xt x t dt x t x t T 2T T
1 T XX lim Asin2 f t Asin2 f t dt T 2T T
T 2 1 1 XX A lim cos2 f cos2 f 2 t dt T 2T T 2
A2 A2 1 T XX cos2 f lim cos2 f 2 t dt T 2 2 2T T
A2 A2 cos2 f cos2 f XX 2 2
Acceptable, but the R.V. is still present?! To find a value not dependent upon a R.V A2 16 E XX E cos2 f cos2 f 2 24
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 11 of 78 ECE 3800 Example: xt Asin2 f t for a uniformly distributed random variable 0,2
RXX t1 ,t2 EX t1 X t2 EAsin2 f t1 Asin2 f t2
2 1 RXX t1 ,t2 EX t1 X t2 A E cos2 f t1 t2 cos2 f t1 t2 2 2 A2 A2 R t ,t EX t X t cos2 f t t Ecos2 f t t 2 XX 1 2 1 2 2 1 2 2 1 2
Of note is that the phase need only be uniformly distributed over 0 to π in the previous step! A2 R t ,t EX t X t cos2 f t t XX 1 2 1 2 2 1 2
for t2 t1 A2 R cos2 f XX 2 but A2 R R cos2 f XX XX 2
Assuming a uniformly distributed random phase “simplifies the problem” !!!
Also of note, if the amplitude is an independent random variable, then EA2 R cos2 f XX 2
The time based formulation: T 1 XX lim xt x t dt x t x t T 2T T 1 T XX lim Asin2 f t Asin2 f t dt T 2T T A2 1 T XX lim cos2 f cos2 f 2t 2 dt T 2 2T T A2 cos2 f XX 2
This appears to be stationary but not technically ergodic … due to the R.V. in the time AC.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 12 of 78 ECE 3800 t t0 Example: xt B rect for B =+/-A with probability p and (1-p) and t0 a uniformly T T T distributed random variable t0 , . Assume B and t0 are independent. 2 2
t1 t0 t1 t0 RXX t1 ,t2 EX t1 X t2 EB rect B rect T T
2 t1 t0 t1 t0 RXX t1 ,t2 EX t1 X t2 EB rect rect T T
As the RV are independent
2 t1 t0 t2 t0 RXX t1 ,t2 EX t1 X t2 EB Erect rect T T
2 2 t1 t0 t2 t0 RXX t1 ,t2 A p A 1 p Erect rect T T
T 2 t t t t 1 R t ,t A2 rect 1 0 rect 2 0 dt XX 1 2 0 T T T T 2
For t1 0 and t2 T 2 t 1 R 0, A2 1 rect 0 dt XX 0 T T T 2
The integral can be recognized as being a triangle, extending from –T to T and zero everywhere else.
2 RXX A tri T
0, T 1 A2 T , T 0 T RXX 1 A2 T , 0 T T 0, T
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 13 of 78 ECE 3800 The time based formulation: 1 T XX lim xt x t dt x t x t T 2T T
C 1 t t0 t t0 XX lim B rect B rect dt C 2C C T T
A change in variable for the integral t t t0 . And only integrate over the finite interval T. T 1 2 t B 2 1 rect dt XX T T T 2 For 0 T 2 T 1 2 1 B 2 1dt B 2 T T B 2 1 XX 2 2 T T T T 2 For T 0 2 T 1 2 1 B 2 1dt B 2 T T B 2 1 XX 2 2 T T T T 2 And
2 t XX B tri T
Not ergodic as taking the expected value of the time autocorrelation … however …
2 t E XX EB tri T
2 2 t E XX A p A 1 p tri T
2 t E XX A tri T
This is identical to the probabilistic autocorrelation previously computed!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 14 of 78 ECE 3800 Some Important Random Processes
Asynchronous Binary Signaling
The pulse values are independent, identically distributed with probability p that amplitude is a and q=1-p that amplitude is –a. The start of the “zeroth” pulse is uniformly distributed from –T/2 to T/2 1 T T pdf D , for D D 2 2
Determine the autocorrelation of the bipolar binary sequence, assuming p=0.5. t D k T X t X k rect k T Note: the rect function is defined as T T t 1, t rect 2 2 T 0, else Determine the Autocorrelation RXX t1 ,t2 EX t1 X t2
t1 D n T t2 D k T RXX t1 ,t2 E X n rect X k rect n T k T
t1 D n T t2 D k T RXX t1 ,t2 E X n rect X k rect nk T T
t1 D n T t2 D k T RXX t1,t2 E X n X k rect X k rect nk T T
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 15 of 78 ECE 3800 t1 D n T t2 D k T RXX t1 ,t2 EX n X k Erect X k rect nk T T
For samples more than one period apart, t1 t2 T , we must consider E X X p a p a p a 1 p a 1 p a p a 1 p a 1 p a k j
2 2 2 EX k X j a p 2 p 1 p 1 p
2 2 EX k X j a 4 p 4 p 1
2 2 For p=0.5 EX k X j a 4 p 4 p 1 0
For samples within one period, t1 t2 T , E X X E X 2 p a 2 1 p a 2 a 2 k k k
2 2 EX k X k1 a 4p 4 p 1 0
For samples within one period, t1 t2 T , there are two regions to consider, the sample bit overlapping and the area of the next bit. 2 t1 D k T t2 D k T RXX t1 ,t2 a Erect rect k T T
But the overlapping area … should be triangular. Therefore T T 1 2 1 2 R E X X dt E X X dt, for T 0 XX k k1 k k T T T T 2 2
T T 1 2 1 2 R E X X dt E X X dt, for 0 T XX k k k k1 T T T T 2 2
or
T 1 2 R a 2 1 dt, for T 0 XX T T 2
T 1 2 R a 2 1 dt, for 0 T XX ta T 2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 16 of 78 ECE 3800 Therefore T a 2 , for T 0 T RXX 2 T a , for 0 T T or recognizing the structure R a 2 1 , for T T XX T
This is simply a triangular function with maximum of a2, extending for a full bit period in both time directions.
For unequal bit probability
t a 2 a 4p 2 4 p 1 , for T T T T RXX 2 2 a 4p 4 p 1 , for T
As there are more of one bit or the other, there is always a positive correlation between bits (the curve is a minimum for p=0.5), that peaks to a2 at = 0.
Note that if the amplitude is a random variable, the expected value of the bits must be further evaluated. Such as,
2 2 EX k X k
2 EX k X k1
In general, the autocorrelation of communications signal waveforms is important, particularly when we discuss the power spectral density later in the textbook.
If the signal takes on two levels a and b vs. a and –a, the result would be
EX k X j p a p a p a 1 p b 1 p b p a 1 p b 1 p b
For p = 1/2
2 1 2 1 1 2 a b EX k X j a ab b 4 2 4 2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 17 of 78 ECE 3800 And 2 2 2 EX k X k EX k p a 1 pb
For p = 1/2 2 2 2 1 2 2 a b a b EX k X k EX k a b 2 2 2
Therefore, a b 2 a b 2 1 , for T T 2 2 T RXX 2 a b , for T 2
For a = 1, b = 0 and T=1, we have 1 1 1 , for T T 4 4 T RXX 1 , for T 4
Figure 9.2-2 Autocorrelation function of ABS random process for a = 1, b = 0 and T = 1.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 18 of 78 ECE 3800 Examples of discrete waveforms used for communications, signal processing, controls, etc.
(a) Unipolar RZ & NRZ, (b) Polar RZ & NRZ , (c) Bipolar NRZ , (d) Split-phase Manchester, (e) Polar quaternary NRZ. From Cahp. 11: A. Bruce Carlson, P.B. Crilly, Communication Systems, 5th ed., McGraw-Hill, 2010. ISBN: 978-0-07-338040-7
In general, a periodic bipolar “pulse” that is shorter in duration than the pulse period will have the autocorrelation function
t R A2 w 1 , for t t XX w w t p tw
for a tw width pulse existing in a tp time period, assuming that positive and negative levels are equally likely.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 19 of 78 ECE 3800 Digital signal autocorrelation functions give rise to a range of Power Spectral Density results. The following shows some of the expected frequency responses for digital waveforms.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 20 of 78 ECE 3800 Exercise 6‐3.1 – Cooper and McGillem a) An ergodic random process has an autocorrelation function of the form
R XX 9 exp 4 16 cos10 16
Find the mean-square value, the mean value, and the variance of the process.
The mean-square (2nd moment) is 2 2 2 EX RXX 0 9 16 16 41 The constant portion of the autocorrelation represents the square of the mean. Therefore 2 2 EX 16 and 4
Finally, the variance can be computed as, 2 2 2 2 EX EX RXX 0 4116 25
b) An ergodic random process has an autocorrelation function of the form 4 2 6 RXX 2 1 Find the mean-square value, the mean value, and the variance of the process.
The mean-square (2nd moment) is
2 6 2 2 EX RXX 0 6 1 The constant portion of the autocorrelation represents the square of the mean. Therefore 4 2 6 4 EX 2 2 lim 4 and 2 t 2 1 1
Finally, the variance can be computed as, 2 2 2 2 EX EX RXX 0 6 4 2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 21 of 78 ECE 3800 Poisson Counting Process
Applications and properties
arrival times radioactive decay’ memoryless property mean arrival rates
Complicated analysis and derivation left for reading in the textbook.
Random Telegraph Signal
The random telegraph signal was originally defined based on a telegraph operator or someone manually sending Morse code. The signal may also represent “zero crossings” in an FM modulated signal.
The signal is a binary signal with random transitions in time.
Figure 9.2-4 Sample function of the random telegraph signal.
Let X(0) = +/-a with equal probability. Use the Poisson arrival process from Chap. 8 as the time of transition to the opposite level. The arrival time is now a R.V.
The probability of signal level correlation at two seperate4 times, assuming different symbols
RXX t1 ,t2 EX t1 X t2 a 2 Pa,a a a P a,a a a P a,a a 2 P a,a
a 2 Pa | a P a a a P a | a P a a a P a | a P a a 2 P a | a P a
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 22 of 78 ECE 3800 For P(a)=1/2
1 R t ,t a 2 Pa | a P a | a P a | a P a | a XX 1 2 2
This becomes the probability of odd or even transitions after the average time interval of the mean of the Poisson arrival time. With this for “the positive time axis” it becomes
k k R a 2 exp exp , 0 XX even k0 k! odd k0 k!
k 2 k RXX a exp 1 , 0 k0 k!
The summation can be determined as for tau>0
2 RXX a exp 2 , 0
To include both positive and negative time (the property of positive and negative autocorrelation)
2 RXX a exp 2
Figure 9.2-5 The symmetric exponential correlation function of an RTS process (a = 2.0, λ = 0.25).
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 23 of 78 ECE 3800 Binary Phase Shift Keying
Figure 9.2-6 System for PSK modulation of Bernoulli random sequence B[n].
The communications symbols represent 180 degree phae shifting of a sinusoidal waveform.
st cos 2 f c t a t
where
a t k,for k T t k 1T
, if b n 1 2 and n , if b n 0 2
Finally
X t cos 2 f c t k T k k
X t cos 2 fc t k T cosk sin2 fc t k T sin k k
X t sin2 f c t k T sink k
Typically T is selected so that the symbols form “complete” cosine waveforms f c T integer
RXX t1,t2 Esin 2 fc t1 k T sin k sin 2 f c t2 n T sin n k n
RXX t1,t2 Esin 2 fc t1 k T sin k sin 2 f c t2 n T sin n n k
RXX t1,t2 Esin 2 fc t1 k T sin 2 f c t2 n T sin k sin n n k
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 24 of 78 ECE 3800 Notice that
Esinksinn k n
The following becomes similar to the previous “rectangular magnitude becoming a triangular autocorrelation. In addition, we typically define a filtering function so to remove frequencies at twice the frequency of interest.
RXX t1 ,t 2 sin2 f c t1 k T sin2 f c t 2 k T , 0 t1 ,t 2 T k
This is where the text stops … using some further analysis and assumptions.
1 RXX t1,t2 cos 2 f c t1 t2 cos 2 fc t1 t2 2 k T , 0 t1,t2 T 2 k
Setting t1 t2 vary over two T. In addition, kT is an integer
1 RXX ,t2 cos 2 fc cos 2 fc 2t2 , T T and 0 t2 T 2 k
Averaging across all possible t2, (alternately, if the original equations had a random phase component …)
1 RXX ,t2 cos 2 f c Ecos 2 fc 2t2 2 2 k
The autocorrelation would become
1 RXX ,t2 cos 2 f c tri , T T 2 T
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 25 of 78 ECE 3800 There is an alternate derivation that focuses on “one” symbol cycle for each of the t1 and t2 sequences, particularly if symbols have zero cross-correlation
Esinksinn k n
only one symbol and in fact the same symbol time shifted is of interest. If one symbol remains fixed and the other varies in time.
t t RXX t,t Erect cos 2 f c t rect cos 2 f c t T T
1 t t RXX t,t Erect rect cos 2 fc cos 2 fc 2t 2 2 T T
The expected value of the random phase, component goes to zero and
1 t t RXX t,t Erect rect cos 2 fc 2 T T
If a time average in t is performed the result becomes.
1 RXX tri cos 2 f c 2 T
If the “symbols” do not have equal probability, there will be a cross-correlation component. Then, the “envelope” of the autocorrelation has a triangular sections (as above) and the rest is a “DC magnitude” that will multiple the cosine “modulated waveform” element. Such as
1 RXX a tri cos 2 f c b cos 2 f c 2 T
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 26 of 78 ECE 3800 Note: the following notes are from Cooper and McGillem ….
The Autocorrelation Function
The autocorrelation is defined as: RXX t1,t2 EX1X 2 dx1 dx2 x1x2 f x1, x2 The above function is valid for all processes, stationary and non-stationary. For WSS processes: RXX t1,t2 EX tX t RXX If the process is ergodic, the time average is equivalent to the probabilistic expectation, or T 1 XX lim xt x t dt x t x t T 2T T and XX RXX
Properties of Autocorrelation Functions 2 2 2 1) RXX 0 EX X or XX 0 x t
2) RXX RXX 3) RXX RXX 0 4) If X has a DC component, then Rxx has a constant factor. X t X Nt 2 RXX X RNN 5) If X has a periodic component, then Rxx will also have a periodic component of the same period. X t A coswt , 0 2 A2 R EX t X t cosw XX 2 6) If X is ergodic and zero mean and has no periodic component, then lim RXX 0 7) Autocorrelation functions can not have an arbitrary shape. One way of specifying shapes permissible is in terms of the Fourier transform of the autocorrelation function.
RXX 0 for all w
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 27 of 78 ECE 3800 The Crosscorrelation Function The crosscorrelation is defined as: RXY t1,t2 EX1Y2 dx1 dy2 x1y2 f x1, y2 RYX t1,t2 EY1X 2 dy1 dx2 y1x2 f y1, x2 The above function is valid for all processes, jointly stationary and non-stationary. For jointly WSS processes: RXY t1,t2 EX tYt RXY RYX t1,t2 EYtX t RYX Note: the order of the subscripts is important for cross-correlation! If the processes are jointly ergodic, the time average is equivalent to the probabilistic expectation, or T 1 XY lim xt y t dt x t y t T 2T T T 1 YX lim yt x t dt y t x t T 2T T and XY RXY YX RYX
Properties of Crosscorrelation Functions 1) The properties of the zoreth lag have no particular significance and do not represent mean-square values. It is true that the “ordered” crosscorrelations must be equal at 0. . RXY 0 RYX 0 or XY 0 YX 0 2) Crosscorrelation functions are not generally even functions. However, there is an antisymmetry to the ordered crosscorrelations: RXY RYX 3) The crosscorrelation does not necessarily have its maximum at the zeroth lag.
It can be shown however that RXY RXX 0 RYY 0 As a note, the crosscorrelation may not achieve this maximum anywhere … 4) If X and Y are statistically independent, then the ordering is not important RXY EX t Y t EX t EY t X Y and
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 28 of 78 ECE 3800 RXY X Y RYX 5) If X is a stationary random process and is differentiable with respect to time, the crosscorrelation of the signal and it’s derivative is given by dR R XX XX d Similarly, 2 d RXX RXX d 2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 29 of 78 ECE 3800 Measurement of The Autocorrelation Function
We love to use time average for everything. For wide-sense stationary, ergodic random processes, time average are equivalent to statistical or probability based values. T 1 XX lim xt x t dt x t x t T 2T T
Using this fact, how can we use short-term time averages to generate auto- or cross-correlation functions?
An estimate of the autocorrelation is defined as: T 1 Rˆ xt x t dt XX T 0
Note that the time average is performed across as much of the signal that is available after the time shift by tau.
Digital Time Sample Correlation
In most practical cases, the operation is performed in terms of digital samples taken at specific time intervals, t . For tau based on the available time step, k, with N equating to the available time interval, we have: N k 1 Rˆ kt xit x it kt t XX N 1 t kt i0
N k 1 Rˆ kt Rˆ k xi x i k XX XX N 1 k i0
In computing this autocorrelation, the initial weighting term approaches 1 when k=N. At this point the entire summation consists of one point and is therefore a poor estimate of the autocorrelation. For useful results, k< Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 30 of 78 ECE 3800 It can be shown that the estimated autocorrelation is equivalent to the actual autocorrelation; therefore, this is an unbiased estimate. N k ˆ ˆ 1 ERXX kt ERXX k E xi x i k N 1 k i0 N k 1 ERˆ kt Exi x i k XX N 1 k i0 N k 1 R kt N 1 k XX i0 1 N k 1 R kt N 1 k XX ˆ ERXX kt RXX kt As noted, the validity of each of the summed autocorrelation lags can and should be brought into question as k approaches N. Biased Estimate of Autocorrelation As a result, a biased estimate of the autocorrelation is commonly used. The biased estimate is defined as: N k ~ 1 R k xi x i k XX N 1 i0 Here, a constant weight instead of one based on the number of elements summed is used. This estimate has the property that the estimated autocorrelation should decrease as k approaches N. The expected value of this estimate can be shown to be ~ n ERXX k 1 RXX kt N 1 The variance of this estimate can be shown to be (math not done at this level) M ~ 2 VarR k R kt 2 XX N XX k M This equation can be used to estimate the number of time samples needed for a useful estimate. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 31 of 78 ECE 3800 Exercise 6-6.1 Find the cross-correlation of the two functions … X t 2cos 2 f t and Yt 10sin2 f t Using the time average functions T 1 XY lim xt y t dt x t y t T 2T T 1 T XY lim 2 cos2 f t 10 sin2 f t dt T 2T T 1 1 f 20 cos2 f t sin2 f t dt XY 1 f 0 1 f 1 20 f sin 2 f 2t 2 sin 2 f dt XY 0 2 1 1 f f 10 f sin 2 f 2t 2 dt 10 f sin 2 f dt XY 0 0 1 f f 1 10 cos2 f 2t 2 f 10 f sin2 f dt XY 0 4 f 0 10 2 cos2 f 2 cos 2 f 2 10 sin 2 f XY 4 f 10 cos4 2 f 2 cos 2 f 2 10 sin2 f XY 4 10 cos2 f 2 cos 2 f 2 10 sin2 f XY 4 XY 10sin2 f Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 32 of 78 ECE 3800 Using the probabilistic functions RXY Ext yt RXY E2 cos 2 f t 10sin2 f t RXY 20 Ecos2 f t sin2 f t RXY 10 Esin 2 f 2t 2 f 2 sin2 f RXY 10sin 2 f 10 Esin2 f 2t 2 f 2 From prior understanding of the uniform random phase …. RXY 10sin2 f By the way, it is useful to have basic trig identities handy when dealing with this stuff … 1 1 sina sin b cosa b cosa b 2 2 1 1 cosa cos b cosa b cosa b 2 2 1 1 sina cos b sina b sina b 2 2 1 1 cosa sin b sina b sina b 2 2 and sin2a 2sina cosa cos2a cosa2 sina2 2cosa2 1 as well as 1 sina 2 1 cos2a 2 1 cosa 2 1 cos2a 2 Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 33 of 78 ECE 3800 Functions of a random variable and time Example: X t fn a,t where a is a wide-sense stationary ergodic process with a known pdf. RXX t1 ,t2 RXX 0, RXX EX 0 X R t ,t R t,t R X t X t pdf a da XX 1 2 XX XX Exponential X t A exp t u t where A is a uniformly distributed random variable A0, B . B 1 R t ,t A exp t u t A exp t u t dA XX 1 2 1 1 2 2 0 B 1 B R t ,t exp t t u max t ,t A2 dA XX 1 2 1 2 1 2 B 0 B 2 R t ,t exp t t umax t ,t XX 1 2 1 2 1 2 3 For t1 0 and t2 B 2 R 0, exp , 0 XX 3 For t1 0 and t2 B 2 R 0, exp , 0 XX 3 Therefore B 2 R exp XX 3 Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 34 of 78 ECE 3800 MATLAB Signal Processing Examples Fig_6_2: Cross Correlation Rxy and Ryx Run the various y waveforms. Chirp and Sinc are popular and interesting. Fig_6_3: Auto Correlation random Gaussian noise Fig_6_4: Auto Correlation smoothers random Gaussian noise Fig_6_9v2: Sin wave correlation in noise Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 35 of 78 ECE 3800 Section 9.4 Classifications of Random Processes Definition 9.4‐1.: Let X and Y be random processes. (a) They are Uncorrelated if * * RXY t1 ,t2 EX t1 Y t2 X t1 Y t2 , for all t1 and t2 (b) They are Orthogonal if * RXY t1 ,t2 EX t1 Y t2 0, for all t1 and t2 (c) They are Independent if for all positive integers n, the nth-order CDF of X and Y factors. That is F x , y , x , y , , x , y ;t ,t , ,t XY 1 1 2 2 n n 1 2 n FX x1 , x2 ,, xn ;t1 ,t2 ,,tn FY y1 , y2 ,, yn ;t1 ,t2 ,,tn Note that if two processes are uncorrelated and one of the means is zero, they are orthogonal as well! Stationarity A random process is stationary when its statistics do not change with the continuous time parameter. F x , x , , x ;t ,t , ,t X 1 2 n 1 2 n FX x1 , x2 ,, xn ;t1 T,t2 T,,tn T Overall, the CDF and pdf do not change with absolute time. They may have time characteristics, as long as the elements are based on time differences and not absolute time. FX x1 , x2 ;t1 ,t2 FX x1 , x2 ;t1 t2 ,0 f X x1 , x2 ;t1 ,t 2 f X x1 , x2 ;t1 t 2 ,0 This implies that * RXX t1 ,t2 EX t1 X t2 RXX t1 t2 ,0 RXX ,0 Definition 9.4‐3.: Wide Sense Stationary A random process is wide-sense stationary (WSS) when its mean and variance statistics do not change with the continuous time parameter. We also include the autocorrelation being a function of one variable … * EX t X t RXX , for ,independednt of t Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 36 of 78 ECE 3800 Power Spectral Density Definition 9.1‐1: PSD Let Rxx(t) be an autocorrelation function for a WSS random process. The power spectral density is defined as the Fourier transform of the autocorrealtion function. S XX w RXX RXX exp iw d The inverse exists in the form of the inverse transform 1 R t S w exp iwt dw XX 2 XX Properties: 1. Sxx(w) is purely real as Rxx(t) is conjugate symmetric 2. If X(t) is a real-valued WSS process, then Sxx(w) is an even function, as Rxx(t) is real and even. 3. Sxx(w)>= 0 for all w. Wiener–Khinchin Theorem For WSS random processes, the autocorrelation function is time based and has a spectral decomposition given by the power spectral density. Also see: http://en.wikipedia.org/wiki/Wiener%E2%80%93Khinchin_theorem Why this is very important … the Fourier Transform of a “single instantiation” of a random process may be meaningless or even impossible to generate. But if the random process can be described in terms of the autocorrelation function (all ergodic, WSS processes), then the power spectral density can be defined. I can then know what the expected frequency spectrum output looks like and I can design a system to keep the required frequencies and filters out the unneeded frequencies (e.g. noise and interference). Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 37 of 78 ECE 3800 Relation of Spectral Density to the Autocorrelation Function For “the right” random processes, power spectral density is the Fourier Transform of the autocorrelation: S XX w RXX EX t X t exp iw d For an ergodic process, we can use time-based processing to arrive at an equivalent result … T 1 XX lim xt x t dt x t x t T 2T T T 1 EX t X t XX lim xt x t dt T 2T T T 1 XX EX t X t lim xt x t dt exp iw d T 2T T T 1 XX lim xt x t exp iw d dt T 2T T T 1 XX lim xt x t exp iw t iwt d dt T 2T T T 1 XX lim xt exp iwt xt exp iw t d dt T 2T T T 1 XX lim xt exp iwt xt exp iw t d dt T 2T T If there exists X X w 1 T XX lim xt exp iwt X w dt T 2T T 1 T XX X w lim xt exp i w t dt T 2T T 2 XX X w X w X w Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 38 of 78 ECE 3800 Properties of the Fourier Transform: X w x x exp iw d For x(t) purely real X w x x cos w i sin w d X w x x cos w d i x sin w d X w EX w i OX w x cos w d i x sin w d E X w x cos w d and OX w x sin w d Notice that: E X w x cos w d x cos w d EX w OX w x sin w d x sin w d OX w Therefore, the real part is symmetric and the imaginary part is anti-symmetric! Note also, for real signals X w conjX w X w* X(w) is conjugate symmetric about the zero axis. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 39 of 78 ECE 3800 Relating this to a real autocorrelation function where RXX RXX RXX EX w i OX w RXX RXX cos w i sin w d RXX RXX t cos wt i sin wt dt RXX RXX t cos wt dt i RXX t sin wt dt RXX EX w i OX w Since Rxx is symmetric, we must have that RXX RXX and EX w i OX w EX w i OX w For this to be true, i OX w i OX w, which can only occur if the odd portion of the Fourier transform is zero! OX w 0 . This provides information about the power spectral density, S XX w RXX EX w S XX w EX w S XX w 0 The power spectral density necessarily contains no phase information! Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 40 of 78 ECE 3800 Example 9.5‐3 Find the psd of the following autocorrelation function … of the random telegraph. RXX exp , for 0 Find a good Fourier Transform Table … otherwise S w R exp j w d XX XX S w exp exp j w d XX 0 S w exp exp j w d exp exp j w d XX 0 0 S w exp j w d exp j w d XX 0 Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 41 of 78 ECE 3800 0 exp j w exp j w S XX w j w 0 j w exp j w exp j w 0 S XX w j w j w exp j w 0 expj w j w j w 1 1 j w j w S w XX j w j w j w j w 2 2 S w XX w2 2 w2 2 For a=3 Figure 9.5-2 Plot of psd for exponential autocorrelation function. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 42 of 78 ECE 3800 Example 9.5‐4 Find the psd of the triangle autocorrelation function … autocorrelation of rect. RXX tri or RXX 1 , T T T T S w 1 exp j w d XX T T 0 T S w 1 exp j w d 1 exp j w d XX T T 0 T 0 0 S w exp j w d exp j w d XX T T T T T exp j w d exp j w d 0 0 T 0 T exp j w exp j w S w XX j w j w T 0 0 1 exp j w exp j w T j w j w 2 T T 1 exp j w exp j w T j w j w 2 0 1 exp j wT exp j wT 1 S XX w j w j w j w j w 1 1 T expj wT expj wT 2 2 T w j w w 1 T exp j wT exp j wT 1 2 2 T j w w w Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 43 of 78 ECE 3800 expj wT exp j wT S w XX j w j w 1 T expj wT T exp j wT T j w j w 1 2 1 expj wT exp j wT 2 2 2 T w T w w 2sinwT 2sinwT 2 1 2 coswT S w XX w w T w2 T w2 2 1 S w 1 coswT XX T w2 2 wT 2 sin 2 1 wT 2 S XX w 2 2 sin j T 2 T w 2 wT 2 Don’t you love the math ?! Using a table is much faster and easier …. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 44 of 78 ECE 3800 Deriving the Mean-Square Values from the Power Spectral Density Using the Fourier transform relation between the Autocorrelation and PSD S XX w RXX exp iw d 1 R t S w exp iwt dw XX 2 XX The mean squared value of a random process is equal to the 0th lag of the autocorrelation 2 1 1 EX RXX 0 S XX w exp iw 0 dw S XX w dw 2 2 2 EX RXX 0 S XX f exp i2f 0 dw S XX f df Therefore, to find the second moment, integrate the PSD over all frequencies. As a note, since the PSD is real and symmetric, the integral can be performed as 2 1 EX RXX 0 2 S XX w dw 2 0 2 EX RXX 0 2 S XX f df 0 Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 45 of 78 ECE 3800 Converting between Autocorrelation and Power Spectral Density Using the properties of the functions we can actually different variations of Transforms! The power spectral density as a function is always real, positive, and an even function in w/f. You can convert between the domains using any of the following … The Fourier Transform in w S XX w RXX exp iw d 1 R t S w exp iwt dw XX 2 XX The Fourier Transform in f S XX f RXX exp i2f d RXX t S XX f exp i2ft df The 2-sided Laplace Transform (the jw axis of the s-plane) S XX s RXX exp s d j 1 R t S s exp st ds XX j2 XX j Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 46 of 78 ECE 3800 Notes on using the Laplace Transform ECE 3100 and ECE 3710 stuff … (1) When converting from the s-domain to the frequency domain use: s jw or w js (2) As an even function, the PSD may be expected to have a polynomial form as: (Hint: no odd powers of w in the numerator or denominator!) w2n a w2n2 a w2n4 a w2 a S w S 2n2 2n4 2 0 XX 0 2m 2m2 2m4 2 w b2m2w b2m4w b2w b0 This can be factored and expressed as: cs c s S w Ts T s XX ds d s To compute the autocorrelation function for 0 use a partial fraction expansion such that gs g s S w XX ds d s and solve for 0 using the LHP poles and zeros as 1 j gs R t exp st ds, for t 0 XX 2 j ds for determining 0 , use the RHP expansion, replace –s with s, perform the Laplace transform and replace t with –t. Another hint, once you have 0 , make the image and skip the math R XX t RXX t . cs c s Final Note … for S w T s T s XX ds d s If you “define” T(s) as all the LHP poles and zeros and T(-s) as all the RHP poles and zeros, then T(s) will represent a (1) causal, (2) minimum phase, and (3) stable filter (if there are no poles on the jw axis). You give me a power spectral density and I can design a filter that passes “signal energy” and filters out as much of the rest as possible! Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 47 of 78 ECE 3800 Example: Inverse Laplace Transform. 2 A2 2 A2 S w X XX 2 2 2 w 1 2 w X Substitute s for w 2 A2 2 A2 S s XX 2 2 s s s Partial fraction expansion 2 k0 k1 k0 s k1 s 2 A S XX s s s s s s s k0 k1 s 0 k0 k1 k k 2 A2 2k 2 A2 0 1 0 A2 A2 S s XX s s Taking the LHP Laplace Transform A2 L A exp t for t 0 s Taking the RHP with –s and then –t. A2 L A2 exp t A2 exp t A2 exp t for t 0 s Combining we have R A2 exp t XX Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 48 of 78 ECE 3800 7-6.3 A stationary random process has a spectral density of. 5, 10 w 20 S XX w 0, else (a) Find the mean-square value of the process. 1 2 R 0 S w dw S w dw XX XX XX 2 2 0 1 20 1 10 1 20 R 0 5 dw 5 dw 2 5 dw XX 2 10 2 20 2 10 10 20 10 50 R 0 w 20 10 XX 2 10 2 (b) Find the auto-correlation function the process. 1 R t S w exp j w t dw XX XX 2 5 20 10 R t exp j w t dw exp j w t dw XX 2 10 20 20 10 5 expj wt expj wt R t XX 2 j t j t 10 20 5 expj 20 t exp j 10 t exp j 10 t expj 20 t RXX t 2 j t j t j t j t 5 expj 20 t exp j 20 t exp j 10 t exp j 10 t R t XX 2 j t j t j t j t Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 49 of 78 ECE 3800 5 2 j sin20 t 2 j sin10 t 5 RXX t sin20 t sin 10 t 2 j t j t t 5 20 10 20 10 10 RXX t 2 sin t cos t sin5 t cos 15 t t 2 2 t 50 sin5t 50 5 t RXX t cos15 t sinc cos15 t 5t (c) Find the value of the auto-correlation function at t=0.. 50 sin5 0 50 5 0 RXX 0 cos15 0 sinc cos15 0 5 0 50 50 R 0 1 1 1 1 XX 50 R 0 XX It must produce the same result! Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 50 of 78 ECE 3800 White Noise Noise is inherently defined as a random process. You may be familiar with “thermal” noise, based on the energy of an atom and the mean-free path that it can travel. As a random process, whenever “white noise” is measured, the values are uncorrelated with each other, not matter how close together the samples are taken in time. Further, we envision “white noise” as containing all spectral content, with no explicit peaks or valleys in the power spectral density. As a result, we define “White Noise” as RXX S0 t N S w S 0 XX 0 2 This is an approximation or simplification because the area of the power spectral density is infinite! Nominally, noise is defined within a bandwidth to describe the power. For example, Thermal noise at the input of a receiver is defined in terms of kT, Boltzmann’s constant times absolute temperature, in terms of Watts/Hz. Thus there is kT Watts of noise power in every Hz of bandwidth. For communications, this is equivalent to –174 dBm/Hz or –204 dBW/Hz. For typical applications, we are interested in Band-Limited White Noise where N 0 S0 f W S XX w 2 0 W f The equivalent noise power is then: W 2 N0 EX RXX 0 S0 dw 2 W S0 2 W N0 W 2 W For communications, we use kTB where W=B and N0=kT. How much noise power, in dBm, would I say that there is in a 1 MHz bandwidth? dBkTB dBkT dBB 174 60 114 dBm Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 51 of 78 ECE 3800 Receiver Sensitivity What does it mean when you buy a radio receiver? For a great receiver (spectrum analyzer grade), assume a 200 kHz FM radio bandwidth. Noise Power kT -174. dBm/Hz Equivalent Noise Bandwidth B 53. dB Hz Receiver Noise Figure NF 10. dB Signal Detection Threshold D 8. dB Minimum Detectable Signal MDS -103. dBm FM radio stations can transmit up to 1 Megawatt +90 dBm Why doesn’t your receiver get blasted? Path loss, distance, higher noise figure, receiving antenna inefficiency, etc. https://en.wikipedia.org/wiki/Path_loss https://en.wikipedia.org/wiki/Friis_transmission_equation But notice that your commercial; receiver is in microvolts, where 2.0 uV is very good. Power into 50 ohms is V^2/R or 8e-14 W = -130.97 dBW -101 dBm. -103 dBm 1.6 uV Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 52 of 78 ECE 3800 FM Radio Design Diagram – Something you may encounter in the future Assume input to be digitized by a 12-bit ADC with 60 dB SNR Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 53 of 78 ECE 3800 Band Limited White Noise N S 0 f W 0 2 S XX w 0 W f The equivalent noise power is then: W 2 EX RXX 0 S0 dw 2 W S0 W But what about the autocorrelation? W RXX t S0 exp i2ft df W W expi2ft expi2Wt exp i2Wt RXX t S0 S0 i2t W i2t i2t 2 i sini2Wt R t S XX 0 i2t xt For sincxt xt RXX t 2 W S0 sinc2Wt Using the concept of correlation, for what values will the autocorrelation be zero? (At these delays in time, sampled data would be uncorrelated with previous samples!) 2Wt k k t for k 1, 2, 2W Sampling at 1/2W seems to be a good idea, but isn’t that the Nyquist rate!! Also note, noise passed through a filter becomes band-limited, and the narrower the filter the smaller the noise power … but the wider is the sinc autocorrelation function. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 54 of 78 ECE 3800 Noise and Filtered Noise Matlab Simulation Based on Cooper and McGillem HW Problem 6-4.6. x=randn(N,1); % zero mean, unit power random signal [b,a] = butter(4,20/500); y=filter(b,a,x); % applying a digital filter y=y/std(y); % normalizing the output power Rxx=xcorr(x)/(N+1); Ryy=xcorr(y)/(N+1); DFTx = fftshift(fft(x))/N; DFTy = fftshift(fft(y))/N; DFTRxx = fftshift(fft(Rxx,2*N))/N; DFTRyy = fftshift(fft(Ryy,2*N))/N; Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 55 of 78 ECE 3800 Pink_Noise … If we call constant at all frequencies white noise, then noise in a limited low frequency band is sometimes called pink noise. OK. I’m getting ahead …. we just did this. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 56 of 78 ECE 3800 The Cross-Spectral Density Why not form the power spectral response of the cross-correlation function? The Fourier Transform in w S XY w RXY exp iw d and SYX w RYX exp iw d 1 1 R t S w exp iwt dw and R t S w exp iwt dw XY 2 XY YX 2 YX Properties of the functions S XY w conjSYX w Since the cross-correlation is real, the real portion of the spectrum is even the imaginary portion of the spectrum is odd There are no other important (assumed) properties to describe Note: the trick using the Laplace transform to form the positive and negative portions of the “time-based” cross-correlation is required to determine the correct “inverse transform” of the “Cross” Power Spectral Density. OK … Time to talk about linear transfer functions … filters!. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 57 of 78 ECE 3800 Section 9.3 Continuous-Time Linear Systems with Random Inputs Linear system requirements: Definition 9.3-1 Let x1(t) and x2(t) be two deterministic time functions and let a1 and a2 be two scalar constants. Let the linear system be described by the operator equation yt Lxt then the system is linear if “linear super-position holds” La1 x1 t a2 x2 t a1 Lx1 t a2 Lx2 t for all admissible functions x1 and x2 and all scalars a1 and a2. For x(t), a random process, y(t) will also be a random process. Linear transformation of signals: convolution in the time domain yt ht xt xt ht yt Linear transformation of signals: multiplication in the Laplace domain Ys Hs X s X s H s Ys The convolution Integrals (applying a causal filter) yt x t h d 0 or t yt h t x d Where for physical realize-ability, causality, and stability constraints we require ht 0 for t 0 and ht dt Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 58 of 78 ECE 3800 Example: Applying a linear filter to a random process ht 5 exp 3t for t 0 X t M 4 cos2t where M and are independent random variables, uniformly distributed [0,2]. We can perform the filter function since an explicit formula for the random process is known. t yt h t x d t yt 5 exp 3 t M 4 cos 2 d t t yt 5 M exp 3 t d 20 exp 3 t cos 2 d exp 3t t yt 5 M 3 t 10 exp 3t exp i2 i exp i2 i d t 5 M exp 3t exp i2 i exp 3t exp i2 i yt 10 3 3 i2 3 i2 5 M expi2t i exp i2t i yt 10 3 3 i2 3 i2 5 M 3 i2 expi2t i 3 i2 exp i2t i yt 10 3 9 4 5 M 20 yt 3 cos2t 2 sin 2t 3 13 Linear filtering will change the magnitude and phase of sinusoidal signals (DC too!). X t M 4 cos2t 5 5 yt M 4 cos2t , 33.69 3 13 Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 59 of 78 ECE 3800 Expected value operator with linear systems For a causal linear system we would have yt x t h d 0 and taking the expected value Eyt E xt h d 0 Eyt Ext h d 0 Eyt t h d 0 For x(t) WSS Eyt h d h d 0 0 Notice the condition for a physically realizable system! The coherent gain of a filter is defined as: h h t dt H 0 gain 0 Therefore, EY t EX hgain EX H 0 Note that: H f h t exp i 2 f t dt For a causal filter H f h t exp i 2 f t dt 0 At f=0 H 0 h t dt 0 And Eyt H 0 Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 60 of 78 ECE 3800 What about a cross‐correlation? (Converting an auto‐correlation to cross‐correlation) For a linear system we would have yt x t h d And performing a cross-correlation (assuming real R.V. and processing) E x t y t E x t x t h d 1 2 1 2 E x t y t E x t x t h d 1 2 1 2 E x t y t E x t x t h d 1 2 1 2 E x t y t R t ,t h d 1 2 XX 1 2 For x(t) WSS E x t y t R R h d XY XX Ext yt RXY RXX h What about the other way … YX instead of XY And performing a cross-correlation (assuming real R.V. and processing) E y t x t E x t h d x t 1 2 1 2 E y t x t E x t x t h d 1 2 1 2 E y t x t E x t x t h d 1 2 1 2 E y t x t R t ,t h d 1 2 XX 1 2 For x(t) WSS … see the next page Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 61 of 78 ECE 3800 For x(t) WSS E y t x t R R t t h d YX XX E y t x t R R h d YX XX Perform a change of variable for lamba to “-kappa” (assuming h(t) is real, see text for complex0 E y t x t R R h d YX XX Therefore E y t x t R R h d YX XX Eyt xt RYX RXX h What about the auto‐correlation of y(t)? And performing an auto-correlation (assuming real R.V. and processing) E y t y t R t ,t E x t h d x t h d 1 2 YY 1 2 1 1 1 1 2 2 2 2 E y t y t R t ,t E x t x t h d h d 1 2 YY 1 2 1 1 2 2 2 2 1 1 E y t y t R t ,t E x t x t h d h d 1 2 YY 1 2 1 1 2 2 2 2 1 1 E y t y t R t ,t R t ,t h d h d 1 2 YY 1 2 XX 1 1 2 2 2 2 1 1 For x(t) WSS E y t y t R R h h d d YY XX 2 1 1 2 2 1 E y t y t R R h d h d YY XX 1 2 2 2 1 1 Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 62 of 78 ECE 3800 The output autocorrelation can also be defined in terms of the cross-correlation as E y t y t R R h d YY XY 1 1 1 Eyt yt RYY RXY h The cross-correlation can be used to determine the output auto-correlation! Continue in this concept, the cross correlation is also a convolution. Therefore, Eyt y t RYY RXX h h If h(t) is complex, the term in h(-t) must be a conjugate. The Mean Square Value at a System Output Based on the output autocorrelation formula E y t 2 R 0 R h h d d YY XX 2 1 1 2 2 1 E y t 2 R 0 h R h d d YY 2 XX 2 1 1 1 2 E y t 2 R 0 h R h d YY XX Based on the input to output cross-correlation formula E y t y t R R h d YY XY Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 63 of 78 ECE 3800 Example: White Noise Inputs to a causal filter N Let R t 0 t XX 2 E Y t 2 R 0 h R h d d YY 1 XX 1 2 2 2 1 0 0 N E Y t 2 R 0 h 0 h d d YY 1 1 2 2 2 1 0 0 2 2 N 0 EY t RYY 0 h1 h 1 d1 2 0 2 N 0 2 EY t RYY 0 h1 d1 2 0 For a white noise process, the mean squared (or 2nd moment) is proportional to the filter power. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 64 of 78 ECE 3800 Example: RC filter The RC low-pass filter 1 1 1 H s s C R C 1 1 s R C s 1 R R C s C Inverse Laplace Transform 1 t ht exp ut R C R C Coherent Gain of the RC Filter h h t dt H 0 gain 0 1 t h exp dt gain 0 R C R C t t exp exp 1 R C 1 R C h 0 gain R C 1 R C 1 R C R C 0 hgain 1 exp exp 1 R C R C If driven by a white noise process, what is the output power? N EY t 2 0 h 2 d 2 0 2 2 N 1 EY t 0 exp d 2 R C R C 0 Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 65 of 78 ECE 3800 2 2 N 1 2 EY t 0 exp d 2 R C R C 0 2 2 exp 2 N 1 R C EY t 0 0 2 R C 2 R C 2 N 0 1 1 EY t 1 N 0 2 2 R C 4 R C Comparing Noise Power in the filter bandwidth N 1 W N B Power in band-limited noise E N 2 0 1 dw 0 1 df W 1 1 2 2 W 2 B 2 N 0 2W W ENW N 0 N 0 B where W is in rad/sec and B in Hz 2 2 2 2 1 The noise power in an RC EYRC N0 4RC For an equivalent band-limited noise process to have the same power (assume a brick wall filter) 2 N0W 1 2 ENW N0 EYRC 2 4RC W 1 N N 0 2 0 4RC Therefore W 2RC W 1 or B where B is in Hz 2 4RC Note that the nominal -3dB band (½ power) of an RC network is 1 1 W or B 3dB RC 3dB 2 RC Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 66 of 78 ECE 3800 Comparing these two, the equivalent noise bandwidth is greater than the –3dB bandwidth by 2 W W or B B 2 3dB 4 3dB Note: B in Hz and W in rad/sec. 1 0.8 0.6 0.4 0.2 0 -0.2 0 0.5 1 1.5 2 2.5 3 3.5 4 Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 67 of 78 ECE 3800 The power spectral density output of linear systems The first cross-spectral density RXY RXX h S XY w RXY exp iw d S w R h exp iw d XY XX Using convolution identities of the Fourier Transform (if you want the proof it isn’t bad, just tedious) S XY w S XX w Hw The second cross-spectral density RYX RXX h SYX w RYX exp iw d S w R h * exp iw d YX XX Using convolution identities of the Fourier Transform (if you want the proof it isn’t bad, just tedious) * SYX w S XX w H w Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 68 of 78 ECE 3800 The output power spectral density becomes RYY RXX h h S w R exp iw d YY YY S w R h h exp iw d YY XX Using convolution identities of the Fourier Transform * SYY w S XX w H w H w 2 SYY w S XX w H w This is a very significant result that provides a similar advantage for the power spectral density computation as the Fourier transform does for the convolution. This leads to the following table. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 69 of 78 ECE 3800 Additional Topics System analysis with a noise input … r t x t ht yt n t Where the signal of interest is x(t), n(t) is a noise or interfering process. The signal plus noise is r(t) and the received system output is y(t) which has been filtered. We have rt xt nt Assuming WSS with x and n independent and n zero mean RRR Er t r t Ext nt xt nt RRR Ex t x t xt nt nt xt nt n t RRR RXX Ext nt Ent xt RNN RRR RXX 2 X N RNN RRR RXX RNN And then * RYY R XX RNN h h RYY RXX h h RNN h h Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 70 of 78 ECE 3800 Signal‐to‐Noise‐Ratio SNR (always done for powers) The signal-to-noise ratio is the power ratio of the signal power to the noise power. P 2 Signal EX t RXX 0 The input SNR is defined as 2 PNoise ENt RNN 0 P 2 Signal EX t ht RXX h h 0 The output SNR is defined as 2 PNoise ENt h t RNN h h 0 For a white noise process and assuming the “filter” does not change the input signal (unity gain), but strictly reduces the noise power by the equivalent noise bandwidth of the filter. 1 2 N We have R 0 S w H w dw 0 h 2 d YY XX 2 2 With appropriate filtering with unity gain where the signal exists and bandwidth reduction for the noise 1 R 0 S w dw N B YY XX 0 EQ 2 or RYY 0 RXX 0 N 0 BEQ PSignal R 0 The output SNR is defined as XX PNoise N 0 BEQ The narrower the filter applied prior to signal processing, the greater the SNR of the signal! Therefore, always apply an analog filter prior to processing the signal of interest! From the definition of band-limited noise power, the equation for the equivalent noise bandwidth (performed as a previous example) N E N t h t 2 R 0 h 2 d 0 h 2 d NN 1 1 1 1 2 1 1 2 B h t 2 dt H f df EQ 2 2 Under the unity gain condition 1 ht dt Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 71 of 78 ECE 3800 Otherwise, the equivalent noise bandwidth can be defined as 1 2 2 B H f df EQ 2 maxH f For a real, low pass filter this simplifies to 1 2 2 B H f df EQ 2 H 0 Using Parseval’s Theorem 2 1 2 2 ht dt H w dw H f df 2 ht 2 dt ht 2 dt 2 BEQ 2 2 H 0 ht dt Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 72 of 78 ECE 3800 Examples of Linear System Frequency‐Domain Analysis Noise in a linear feedback system loop. Ns A X s s s 1 Ys 1 Linear superposition of X to Y and N to Y. A Y s X s Y s Ns s s 1 A A Y s 1 X s N s s s 1 s s 1 s 2 s A A Y s X s N s s s 1 s s 1 A s 2 s Y s X s Ns s 2 s A s 2 s A There are effectively two filters, one applied to X and a second apply to N. A s 2 s H s and H s X s 2 s A N s 2 s A Y s H X s X s H N s Ns Generic definition of output Power Spectral Density: 2 2 SYY w H X w S XX w H N w S NN w Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 73 of 78 ECE 3800 Change in the input to output signal to noise ratio. S w dw XX SNRIn S w dw NN H w 2 S w dw H 0 2 S w X XX X XX SNROut 2 N 2 H w S w dw 0 H w dw N NN N 2 S w S w XX XX SNROut 2 N H w N 0 BEQ 0 N dw 2 2 H X 0 Where H w 2 B N dw EQ 2 H 0 X If the noise is added at the x signal input, the expected definition of noise equivalent bandwidth results. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 74 of 78 ECE 3800 Systems that Maximize Signal-to-Noise Ratio 2 PSignal E st SNR is defined as PNoise N0 BEQ Define for an input signal st nt Define for a filtered output signal so t no t For a linear system, we have: so t no t h s t n t d 0 The input SNR can be describe as 2 PSignal E st SNR in 2 PNoise Ent The output SNR can be described as 2 2 PSignal E s t E s t SNR o o out P 2 N B Noise Eno t o EQ 2 E h s t d 0 SNR out 1 N ht 2 dt o 2 0 Using Schwartz’s Inequality the numerator becomes 2 2 2 E h s t d E h d E s t d 0 0 0 Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 75 of 78 ECE 3800 Applying this inequality, an SNR inequality can be defined as 2 2 h d E s t d SNR 0 0 out 1 N ht 2 dt o 2 0 Canceling the filter terms, we have 2 2 SNRout E st d No 0 To achieve the maximum SNR, the equality condition of Schwartz’s Inequality must hold, or 2 2 2 h s t d h d s t d 0 0 0 This condition can be met for h K st u where K is an arbitrary gain constant. The desired impulse response is simply the time inverse of the signal waveform at time t, a fixed moment chosen for optimality. If this is done, the maximum filter power (with K=1) can be computed as t h 2 d s t 2 d s 2 d t 0 0 2 maxSNRout t No This filter concept is called a matched filter. If you wanted to detect a burst waveform that has been transmitted, to maximize the received SNR, the receiving filter should be the time inverse of the signal transmitted! Note and caution: when using such a filter, the received signal maximum SNR will occur when the signal and convolved filter perfectly overlap. This moment in time occurs when the “complete” burst has been received by the system. If measuring the time-of-flight of the burst, the moment is exactly the filter length longer than the time-of-flight. (Think about where the leading edge of the signal-of-interest is when transmitted, when first received, and when fully present in the filter). Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 76 of 78 ECE 3800 The Matched Filter Wikipedia: https://en.wikipedia.org/wiki/Matched_filter “In signal processing, a matched filter is obtained by correlating a known signal, or template, with an unknown signal to detect the presence of the template in the unknown signal.[1][2] This is equivalent to convolving the unknown signal with a conjugated time-reversed version of the template. The matched filter is the optimal linear filter for maximizing the signal to noise ratio (SNR) in the presence of additive stochastic noise. Matched filters are commonly used in radar, in which a known signal is sent out, and the reflected signal is examined for common elements of the out-going signal. Pulse compression is an example of matched filtering. It is so called because impulse response is matched to input pulse signals. Two-dimensional matched filters are commonly used in image processing, e.g., to improve SNR for X-ray. Matched filtering is a demodulation technique with LTI (linear time invariant) filters to maximize SNR.[3].” Applications: Radar Sonar Pulse Compression Digital Communications (Correlation detectors) GPS pseudo-random sequence correlation If you are looking for a signal, maximize the output of the filter when the signal is input! You will have a matched filter! Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 77 of 78 ECE 3800 Periodograms and Windowing Data We can window the data prior to autocorrelation. this will cause a change in the energy that must be compensated 1 T AvgEnergy xt w t 2 dt T 0 1 T 1 T EAvgEnergy Ext 2 wt 2 dt Ext 2 wt 2 dt T 0 T 0 For a rectangular window, this becomes EAvgEnergy Ext2 If an alternate window is used, a power scaling will be necessary to maintain the same estimated energy. For a given window, function we can apply: 1 T scale wt 2 dt T 0 So that T wt 1 2 w t w t w t dt scale scale T 0 This also applies to the sampled data computations where N 1 wn 1 2 wscale n wn wm scale N m0 Now, multiple smaller results may be processed to generate an average. The spectral resolution (number of frequency bins) necessarily decreases, but now multiple spectra and can be averaged together based on “unique” time sample sets of the data. It is useful for observing time varying phenomena and capturing the values when the desired signal is present. See WaterFallDemo.m for a spectral periodogram …. Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012. B.J. Bazuin, Fall 2016 78 of 78 ECE 3800