Computational Neuroscience - Lecture 3

Kugiumtzis Dimitris

Department of Electrical and Computer Engineering, Faculty of Engineering, Aristotle University of Thessaloniki, Greece e-mail: [email protected] http:\\users.auth.gr\dkugiu

22 May 2018

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 1 Single-channel EEG analysis - Introduction

2 Features of signal morphology

3 Features from linear analysis

4 Features from nonlinear analysis

Outline

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 2 Features of signal morphology

3 Features from linear analysis

4 Features from nonlinear analysis

Outline

1 Single-channel EEG analysis - Introduction

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 3 Features from linear analysis

4 Features from nonlinear analysis

Outline

1 Single-channel EEG analysis - Introduction

2 Features of signal morphology

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 4 Features from nonlinear analysis

Outline

1 Single-channel EEG analysis - Introduction

2 Features of signal morphology

3 Features from linear analysis

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Outline

1 Single-channel EEG analysis - Introduction

2 Features of signal morphology

3 Features from linear analysis

4 Features from nonlinear analysis

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Eyeball judgement: Doctor or EEGer may diagnose abnormalities by visual inspection of EEG signals (in ROI). Alternatively: Quantify automatically information from EEG signal (in ROI) ⇒ univariate analysis: study the static and dynamic properties of the time series (signal), model the time series.

Three main perspectives: 1 Characteristics of signal morphology, e.g. identify bursts.

2 stochastic process (linear analysis), e.g. Xt = φ0 + φ1Xt−1 + ... + φpXt−p + t

3 nonlinear dynamics and chaos (nonlinear analysis), e.g. d St = f (St−1), St ∈ IR , Xt = h(St ) + t

Single-channel EEG analysis - Introduction Region of Interest (ROI) of the brain

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Alternatively: Quantify automatically information from EEG signal (in ROI) ⇒ univariate time series analysis: study the static and dynamic properties of the time series (signal), model the time series.

Three main perspectives: 1 Characteristics of signal morphology, e.g. identify bursts.

2 stochastic process (linear analysis), e.g. Xt = φ0 + φ1Xt−1 + ... + φpXt−p + t

3 nonlinear dynamics and chaos (nonlinear analysis), e.g. d St = f (St−1), St ∈ IR , Xt = h(St ) + t

Single-channel EEG analysis - Introduction Region of Interest (ROI) of the brain Eyeball judgement: Doctor or EEGer may diagnose abnormalities by visual inspection of EEG signals (in ROI).

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 univariate time series analysis: study the static and dynamic properties of the time series (signal), model the time series.

Three main perspectives: 1 Characteristics of signal morphology, e.g. identify bursts.

2 stochastic process (linear analysis), e.g. Xt = φ0 + φ1Xt−1 + ... + φpXt−p + t

3 nonlinear dynamics and chaos (nonlinear analysis), e.g. d St = f (St−1), St ∈ IR , Xt = h(St ) + t

Single-channel EEG analysis - Introduction Region of Interest (ROI) of the brain Eyeball judgement: Doctor or EEGer may diagnose abnormalities by visual inspection of EEG signals (in ROI). Alternatively: Quantify automatically information from EEG signal (in ROI) ⇒

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Three main perspectives: 1 Characteristics of signal morphology, e.g. identify bursts.

2 stochastic process (linear analysis), e.g. Xt = φ0 + φ1Xt−1 + ... + φpXt−p + t

3 nonlinear dynamics and chaos (nonlinear analysis), e.g. d St = f (St−1), St ∈ IR , Xt = h(St ) + t

Single-channel EEG analysis - Introduction Region of Interest (ROI) of the brain Eyeball judgement: Doctor or EEGer may diagnose abnormalities by visual inspection of EEG signals (in ROI). Alternatively: Quantify automatically information from EEG signal (in ROI) ⇒ univariate time series analysis: study the static and dynamic properties of the time series (signal), model the time series.

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 2 stochastic process (linear analysis), e.g. Xt = φ0 + φ1Xt−1 + ... + φpXt−p + t

3 nonlinear dynamics and chaos (nonlinear analysis), e.g. d St = f (St−1), St ∈ IR , Xt = h(St ) + t

Single-channel EEG analysis - Introduction Region of Interest (ROI) of the brain Eyeball judgement: Doctor or EEGer may diagnose abnormalities by visual inspection of EEG signals (in ROI). Alternatively: Quantify automatically information from EEG signal (in ROI) ⇒ univariate time series analysis: study the static and dynamic properties of the time series (signal), model the time series.

Three main perspectives: 1 Characteristics of signal morphology, e.g. identify bursts.

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 3 nonlinear dynamics and chaos (nonlinear analysis), e.g. d St = f (St−1), St ∈ IR , Xt = h(St ) + t

Single-channel EEG analysis - Introduction Region of Interest (ROI) of the brain Eyeball judgement: Doctor or EEGer may diagnose abnormalities by visual inspection of EEG signals (in ROI). Alternatively: Quantify automatically information from EEG signal (in ROI) ⇒ univariate time series analysis: study the static and dynamic properties of the time series (signal), model the time series.

Three main perspectives: 1 Characteristics of signal morphology, e.g. identify bursts.

2 stochastic process (linear analysis), e.g. Xt = φ0 + φ1Xt−1 + ... + φpXt−p + t

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Single-channel EEG analysis - Introduction Region of Interest (ROI) of the brain Eyeball judgement: Doctor or EEGer may diagnose abnormalities by visual inspection of EEG signals (in ROI). Alternatively: Quantify automatically information from EEG signal (in ROI) ⇒ univariate time series analysis: study the static and dynamic properties of the time series (signal), model the time series.

Three main perspectives: 1 Characteristics of signal morphology, e.g. identify bursts.

2 stochastic process (linear analysis), e.g. Xt = φ0 + φ1Xt−1 + ... + φpXt−p + t

3 nonlinear dynamics and chaos (nonlinear analysis), e.g. d St = f (St−1), St ∈ IR , Xt = h(St ) + t

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Example: α-wave (8-13 Hz) is reduced in children and in the elderly, and in patients with dementia, schizophrenia, stroke, and epilepsy

Clinical applications: predicting epileptic seizures classifying sleep stages measuring depth of anesthesia detection and monitoring of brain injury detecting abnormal brain states others ...

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Clinical applications: predicting epileptic seizures classifying sleep stages measuring depth of anesthesia detection and monitoring of brain injury detecting abnormal brain states others ...

Example: α-wave (8-13 Hz) is reduced in children and in the elderly, and in patients with dementia, schizophrenia, stroke, and epilepsy

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Pn A feature can be normalized, fi := fi / j=1 fj , where n number of windows. If fi is measured in several (many) channels, fi,j , for channels j = 1,..., K: take average at each time window, e.g. 1 PK fi := K j=1 fi,j .

Features of signal morphology: Indices () that capture some characteristic of the shape of the signal. : center (, ) dispersion (, SD, interquartile ) higher moments (, )

Features of signal morphology

Any feature fi is computed on sliding windows i (overlapped or non-overlapped) across the EEG signal (of length nw ).

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 If fi is measured in several (many) channels, fi,j , for channels j = 1,..., K: take average at each time window, e.g. 1 PK fi := K j=1 fi,j .

Features of signal morphology: Indices (statistics) that capture some characteristic of the shape of the signal. Descriptive statistics: center (mean, median) dispersion (variance, SD, ) higher moments (skewness, kurtosis)

Features of signal morphology

Any feature fi is computed on sliding windows i (overlapped or non-overlapped) across the EEG signal (of length nw ). Pn A feature can be normalized, fi := fi / j=1 fj , where n number of windows.

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Features of signal morphology: Indices (statistics) that capture some characteristic of the shape of the signal. Descriptive statistics: center (mean, median) dispersion (variance, SD, interquartile range) higher moments (skewness, kurtosis)

Features of signal morphology

Any feature fi is computed on sliding windows i (overlapped or non-overlapped) across the EEG signal (of length nw ). Pn A feature can be normalized, fi := fi / j=1 fj , where n number of windows. If fi is measured in several (many) channels, fi,j , for channels j = 1,..., K: take average at each time window, e.g. 1 PK fi := K j=1 fi,j .

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Descriptive statistics: center (mean, median) dispersion (variance, SD, interquartile range) higher moments (skewness, kurtosis)

Features of signal morphology

Any feature fi is computed on sliding windows i (overlapped or non-overlapped) across the EEG signal (of length nw ). Pn A feature can be normalized, fi := fi / j=1 fj , where n number of windows. If fi is measured in several (many) channels, fi,j , for channels j = 1,..., K: take average at each time window, e.g. 1 PK fi := K j=1 fi,j .

Features of signal morphology: Indices (statistics) that capture some characteristic of the shape of the signal.

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Features of signal morphology

Any feature fi is computed on sliding windows i (overlapped or non-overlapped) across the EEG signal (of length nw ). Pn A feature can be normalized, fi := fi / j=1 fj , where n number of windows. If fi is measured in several (many) channels, fi,j , for channels j = 1,..., K: take average at each time window, e.g. 1 PK fi := K j=1 fi,j .

Features of signal morphology: Indices (statistics) that capture some characteristic of the shape of the signal. Descriptive statistics: center (mean, median) dispersion (variance, SD, interquartile range) higher moments (skewness, kurtosis)

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 r   mobility = Var dx(t) /Var(x(t)), dt estimates the SD of PX X (f ) (along the frequency axis)   complexity = mobility dx(t) /mobility(x(t)), dt estimates similarity of the signal to a pure sine wave (deviation of complexity from one).

[Hjorth, Electroenceph. Clin. Neuroph., 1970]

Hjorth parameters: Pnf activity = variance = total power, Var(x(t)) = i=0 PXX (fi ).

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3   complexity = mobility dx(t) /mobility(x(t)), dt estimates similarity of the signal to a pure sine wave (deviation of complexity from one).

[Hjorth, Electroenceph. Clin. Neuroph., 1970]

Hjorth parameters: Pnf activity = variance = total power, Var(x(t)) = i=0 PXX (fi ). r   mobility = Var dx(t) /Var(x(t)), dt estimates the SD of PX X (f ) (along the frequency axis)

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 [Hjorth, Electroenceph. Clin. Neuroph., 1970]

Hjorth parameters: Pnf activity = variance = total power, Var(x(t)) = i=0 PXX (fi ). r   mobility = Var dx(t) /Var(x(t)), dt estimates the SD of PX X (f ) (along the frequency axis)   complexity = mobility dx(t) /mobility(x(t)), dt estimates similarity of the signal to a pure sine wave (deviation of complexity from one).

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Hjorth parameters: Pnf activity = variance = total power, Var(x(t)) = i=0 PXX (fi ). r   mobility = Var dx(t) /Var(x(t)), dt estimates the SD of PX X (f ) (along the frequency axis)   complexity = mobility dx(t) /mobility(x(t)), dt estimates similarity of the signal to a pure sine wave (deviation of complexity from one).

[Hjorth, Electroenceph. Clin. Neuroph., 1970]

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Pnw −1 2 Nonlinear energy: NLE(i) = t=2 |x(t) − x(t − 1)x(t + 1)|.

Pnw −1 Line Length: L(i) = t=1 |x(t + 1) − x(t)|. Pn Normalized line length: L(i) := L(i)/ j=1 L(j). Median over all channels: L(i) := medianL(i, j), j denotes channel.

[Koolen et al, Clin. Neuroph., 2014]

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Pnw −1 Line Length: L(i) = t=1 |x(t + 1) − x(t)|. Pn Normalized line length: L(i) := L(i)/ j=1 L(j). Median over all channels: L(i) := medianL(i, j), j denotes channel.

[Koolen et al, Clin. Neuroph., 2014]

Pnw −1 2 Nonlinear energy: NLE(i) = t=2 |x(t) − x(t − 1)x(t + 1)|. Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 ... and many other features of signal morphology: Fisher information (from normalized spectrum of singular values) Katz’s fractal dimension (from line length and largest distance from start) Petrosian Fractal Dimension (PFD) (from number of sign changes in the signal derivative) Hurst exponent (the Rescaled Range statistics (R/S)) Detrended Fluctuation Analysis (DFA) (long range correlation measure)

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Power in bands: δ[0, 4]Hz, θ[4, 8]Hz, α[8, 13]Hz, β[13, 30]Hz, γ > 30Hz, e.g. for α-band power: P (α)= Pfk =13 P (f ) XX fk =8 XX k Pnf or relative α-band power: RPXX (α)= PXX (α)/ i=0 PXX (fi )

In the same way the bands are defined on .

Spectral edge frequency (SEF), e.g. SEF(90) is the frequency at 90 90 Pf Pfs /2 90% of total power, f : f =0 PXX (f )/ f =0 PXX (f ) = 0.90 median frequency: SEF(50)

Features of linear analysis -

Features from power spectrum PXX (fk ), fk = 0,..., fs /2,

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 e.g. for α-band power: P (α)= Pfk =13 P (f ) XX fk =8 XX k Pnf or relative α-band power: RPXX (α)= PXX (α)/ i=0 PXX (fi )

In the same way the bands are defined on wavelets.

Spectral edge frequency (SEF), e.g. SEF(90) is the frequency at 90 90 Pf Pfs /2 90% of total power, f : f =0 PXX (f )/ f =0 PXX (f ) = 0.90 median frequency: SEF(50)

Features of linear analysis - frequency domain

Features from power spectrum PXX (fk ), fk = 0,..., fs /2, Power in bands: δ[0, 4]Hz, θ[4, 8]Hz, α[8, 13]Hz, β[13, 30]Hz, γ > 30Hz,

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 In the same way the bands are defined on wavelets.

Spectral edge frequency (SEF), e.g. SEF(90) is the frequency at 90 90 Pf Pfs /2 90% of total power, f : f =0 PXX (f )/ f =0 PXX (f ) = 0.90 median frequency: SEF(50)

Features of linear analysis - frequency domain

Features from power spectrum PXX (fk ), fk = 0,..., fs /2, Power in bands: δ[0, 4]Hz, θ[4, 8]Hz, α[8, 13]Hz, β[13, 30]Hz, γ > 30Hz, e.g. for α-band power: P (α)= Pfk =13 P (f ) XX fk =8 XX k Pnf or relative α-band power: RPXX (α)= PXX (α)/ i=0 PXX (fi )

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Spectral edge frequency (SEF), e.g. SEF(90) is the frequency at 90 90 Pf Pfs /2 90% of total power, f : f =0 PXX (f )/ f =0 PXX (f ) = 0.90 median frequency: SEF(50)

Features of linear analysis - frequency domain

Features from power spectrum PXX (fk ), fk = 0,..., fs /2, Power in bands: δ[0, 4]Hz, θ[4, 8]Hz, α[8, 13]Hz, β[13, 30]Hz, γ > 30Hz, e.g. for α-band power: P (α)= Pfk =13 P (f ) XX fk =8 XX k Pnf or relative α-band power: RPXX (α)= PXX (α)/ i=0 PXX (fi )

In the same way the bands are defined on wavelets.

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Features of linear analysis - frequency domain

Features from power spectrum PXX (fk ), fk = 0,..., fs /2, Power in bands: δ[0, 4]Hz, θ[4, 8]Hz, α[8, 13]Hz, β[13, 30]Hz, γ > 30Hz, e.g. for α-band power: P (α)= Pfk =13 P (f ) XX fk =8 XX k Pnf or relative α-band power: RPXX (α)= PXX (α)/ i=0 PXX (fi )

In the same way the bands are defined on wavelets.

Spectral edge frequency (SEF), e.g. SEF(90) is the frequency at 90 90 Pf Pfs /2 90% of total power, f : f =0 PXX (f )/ f =0 PXX (f ) = 0.90 median frequency: SEF(50)

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Example:

5 datasets of 100 single-channel EEG of N = 4096 (fs = 173.61 Hz) A: scalp EEG, healthy eyes open B: scalp EEG, healthy eyes closed C, D: intracranial EEG, interictal period E: intracranial EEG, ictal period http://epileptologie-bonn.de/cms/front_content.php?idcat=193&lang=3

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Example:

5 datasets of 100 single-channel EEG of N = 4096 (fs = 173.61 Hz) A: scalp EEG, healthy eyes open B: scalp EEG, healthy eyes closed C, D: intracranial EEG, interictal period E: intracranial EEG, ictal period http://epileptologie-bonn.de/cms/front_content.php?idcat=193&lang=3

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 [Bao et al, Comput Intell Neurosci, 2011]

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Assuming the linear stochastic process Xt = φ0 + φ1Xt−1 + ... + φpXt−p + t Autoregressive model: xˆt = φˆ0 + φˆ1xt−1 + ... + φˆpxt−p 1 Pn 2 Mean square error of fit: MSE= n−p t=p+1(xt − xˆt )

Features of linear analysis -

Pn−τ cX (τ) t=1 (xt −x¯)(xt+τ −x¯) : rX (τ)= 2 = Pn−τ 2 sX t=1 (xt −x¯)

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 1 Pn 2 Mean square error of fit: MSE= n−p t=p+1(xt − xˆt )

Features of linear analysis - time domain

Pn−τ cX (τ) t=1 (xt −x¯)(xt+τ −x¯) Autocorrelation: rX (τ)= 2 = Pn−τ 2 sX t=1 (xt −x¯) Assuming the linear stochastic process Xt = φ0 + φ1Xt−1 + ... + φpXt−p + t Autoregressive model: xˆt = φˆ0 + φˆ1xt−1 + ... + φˆpxt−p

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Features of linear analysis - time domain

Pn−τ cX (τ) t=1 (xt −x¯)(xt+τ −x¯) Autocorrelation: rX (τ)= 2 = Pn−τ 2 sX t=1 (xt −x¯) Assuming the linear stochastic process Xt = φ0 + φ1Xt−1 + ... + φpXt−p + t Autoregressive model: xˆt = φˆ0 + φˆ1xt−1 + ... + φˆpxt−p 1 Pn 2 Mean square error of fit: MSE= n−p t=p+1(xt − xˆt )

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Entropy: information from each sample of X (assume proper discretization of X ) X H(X )= pX (x) log pX (x) x Mutual information: information for Y knowing X and vice versa X pXY (x, y) I (X , Y )= H(X )+H(Y )−H(X , Y ) = p (x, y) log XY p (x)p (y) x,y X Y

For X → Xt and Y → Xt+τ , Delayed mutual information:

X pXt Xt+τ (xt , xt+τ ) IX (τ)= I (Xt , Xt+τ ) = pXt Xt+τ (xt , xt+τ ) log pXt (xt )pXt+τ (xt+τ ) xt ,xt+τ n To compute IX (τ) make a partition of {xt }t=1, a partition of n {yt }t=1 and compute probabilities for each cell from the relative frequency.

Features of nonlinear analysis - 1 Extension of autocorrelation to linear and nonlinear correlation

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Mutual information: information for Y knowing X and vice versa X pXY (x, y) I (X , Y )= H(X )+H(Y )−H(X , Y ) = p (x, y) log XY p (x)p (y) x,y X Y

For X → Xt and Y → Xt+τ , Delayed mutual information:

X pXt Xt+τ (xt , xt+τ ) IX (τ)= I (Xt , Xt+τ ) = pXt Xt+τ (xt , xt+τ ) log pXt (xt )pXt+τ (xt+τ ) xt ,xt+τ n To compute IX (τ) make a partition of {xt }t=1, a partition of n {yt }t=1 and compute probabilities for each cell from the relative frequency.

Features of nonlinear analysis - 1 Extension of autocorrelation to linear and nonlinear correlation Entropy: information from each sample of X (assume proper discretization of X ) X H(X )= pX (x) log pX (x) x

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 For X → Xt and Y → Xt+τ , Delayed mutual information:

X pXt Xt+τ (xt , xt+τ ) IX (τ)= I (Xt , Xt+τ ) = pXt Xt+τ (xt , xt+τ ) log pXt (xt )pXt+τ (xt+τ ) xt ,xt+τ n To compute IX (τ) make a partition of {xt }t=1, a partition of n {yt }t=1 and compute probabilities for each cell from the relative frequency.

Features of nonlinear analysis - 1 Extension of autocorrelation to linear and nonlinear correlation Entropy: information from each sample of X (assume proper discretization of X ) X H(X )= pX (x) log pX (x) x Mutual information: information for Y knowing X and vice versa X pXY (x, y) I (X , Y )= H(X )+H(Y )−H(X , Y ) = p (x, y) log XY p (x)p (y) x,y X Y

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 n To compute IX (τ) make a partition of {xt }t=1, a partition of n {yt }t=1 and compute probabilities for each cell from the relative frequency.

Features of nonlinear analysis - 1 Extension of autocorrelation to linear and nonlinear correlation Entropy: information from each sample of X (assume proper discretization of X ) X H(X )= pX (x) log pX (x) x Mutual information: information for Y knowing X and vice versa X pXY (x, y) I (X , Y )= H(X )+H(Y )−H(X , Y ) = p (x, y) log XY p (x)p (y) x,y X Y

For X → Xt and Y → Xt+τ , Delayed mutual information:

X pXt Xt+τ (xt , xt+τ ) IX (τ)= I (Xt , Xt+τ ) = pXt Xt+τ (xt , xt+τ ) log pXt (xt )pXt+τ (xt+τ ) xt ,xt+τ

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Features of nonlinear analysis - 1 Extension of autocorrelation to linear and nonlinear correlation Entropy: information from each sample of X (assume proper discretization of X ) X H(X )= pX (x) log pX (x) x Mutual information: information for Y knowing X and vice versa X pXY (x, y) I (X , Y )= H(X )+H(Y )−H(X , Y ) = p (x, y) log XY p (x)p (y) x,y X Y

For X → Xt and Y → Xt+τ , Delayed mutual information:

X pXt Xt+τ (xt , xt+τ ) IX (τ)= I (Xt , Xt+τ ) = pXt Xt+τ (xt , xt+τ ) log pXt (xt )pXt+τ (xt+τ ) xt ,xt+τ n To compute IX (τ) make a partition of {xt }t=1, a partition of n {yt }t=1 and compute probabilities for each cell from the relative frequency.

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 m All models require the state space reconstruction, points xt ∈ IR from scalar xt . Delay embedding: xt = [xt , xt−τ ,..., xt−(m−1)τ ]. Nonlinear model: xt+1 = f (xt )

1 parametric models, e.g. polynomial autoregressive models

2 semilocal (black-box) models, e.g. neural networks

3 local models, e.g. nearest neighbor models.

For the prediction of xt+1 having x1, x2,..., xt .

Find the k nearest neighbors of xt , {xt(1),..., xt(k)} Fit a linear autoregressive model on the k neighbors. xt(i)+1 = φ0 + φ1xt(i) + ··· + φmxt(i)−(m−1)τ + t+1 ˆ ˆ ˆ Predictx ˆt+1 = φ0 + φ1xt + ··· + φmxt−(m−1)τ

Features of nonlinear analysis - 2 Extension of linear autoregressive models to nonlinear models

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Nonlinear model: xt+1 = f (xt )

1 parametric models, e.g. polynomial autoregressive models

2 semilocal (black-box) models, e.g. neural networks

3 local models, e.g. nearest neighbor models.

For the prediction of xt+1 having x1, x2,..., xt .

Find the k nearest neighbors of xt , {xt(1),..., xt(k)} Fit a linear autoregressive model on the k neighbors. xt(i)+1 = φ0 + φ1xt(i) + ··· + φmxt(i)−(m−1)τ + t+1 ˆ ˆ ˆ Predictx ˆt+1 = φ0 + φ1xt + ··· + φmxt−(m−1)τ

Features of nonlinear analysis - 2 Extension of linear autoregressive models to nonlinear models m All models require the state space reconstruction, points xt ∈ IR from scalar xt . Delay embedding: xt = [xt , xt−τ ,..., xt−(m−1)τ ].

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 For the prediction of xt+1 having x1, x2,..., xt .

Find the k nearest neighbors of xt , {xt(1),..., xt(k)} Fit a linear autoregressive model on the k neighbors. xt(i)+1 = φ0 + φ1xt(i) + ··· + φmxt(i)−(m−1)τ + t+1 ˆ ˆ ˆ Predictx ˆt+1 = φ0 + φ1xt + ··· + φmxt−(m−1)τ

Features of nonlinear analysis - 2 Extension of linear autoregressive models to nonlinear models m All models require the state space reconstruction, points xt ∈ IR from scalar xt . Delay embedding: xt = [xt , xt−τ ,..., xt−(m−1)τ ]. Nonlinear model: xt+1 = f (xt )

1 parametric models, e.g. polynomial autoregressive models

2 semilocal (black-box) models, e.g. neural networks

3 local models, e.g. nearest neighbor models.

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Features of nonlinear analysis - 2 Extension of linear autoregressive models to nonlinear models m All models require the state space reconstruction, points xt ∈ IR from scalar xt . Delay embedding: xt = [xt , xt−τ ,..., xt−(m−1)τ ]. Nonlinear model: xt+1 = f (xt )

1 parametric models, e.g. polynomial autoregressive models

2 semilocal (black-box) models, e.g. neural networks

3 local models, e.g. nearest neighbor models.

For the prediction of xt+1 having x1, x2,..., xt .

Find the k nearest neighbors of xt , {xt(1),..., xt(k)} Fit a linear autoregressive model on the k neighbors. xt(i)+1 = φ0 + φ1xt(i) + ··· + φmxt(i)−(m−1)τ + t+1 ˆ ˆ ˆ Predictx ˆt+1 = φ0 + φ1xt + ··· + φmxt−(m−1)τ

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Approximate entropy, ApEn (uses a bandwidth in the estimation of pX(x)) Sample entropy (similar to ApEn) Permutation entropy (entropy on ranks of the components in xt )

Spectral entropy (entropy on PXX (f )) Others, e.g. fuzzy entropy, multiscale entropy.

Features of nonlinear analysis - 3

Entropy: Estimates of the entropy of xt = [xt , xt−τ ,..., xt−(m−1)τ ]: P H(X)= x pX(x) log pX(x)

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Sample entropy (similar to ApEn) Permutation entropy (entropy on ranks of the components in xt )

Spectral entropy (entropy on PXX (f )) Others, e.g. fuzzy entropy, multiscale entropy.

Features of nonlinear analysis - 3

Entropy: Estimates of the entropy of xt = [xt , xt−τ ,..., xt−(m−1)τ ]: P H(X)= x pX(x) log pX(x) Approximate entropy, ApEn (uses a bandwidth in the estimation of pX(x))

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Permutation entropy (entropy on ranks of the components in xt )

Spectral entropy (entropy on PXX (f )) Others, e.g. fuzzy entropy, multiscale entropy.

Features of nonlinear analysis - 3

Entropy: Estimates of the entropy of xt = [xt , xt−τ ,..., xt−(m−1)τ ]: P H(X)= x pX(x) log pX(x) Approximate entropy, ApEn (uses a bandwidth in the estimation of pX(x)) Sample entropy (similar to ApEn)

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Spectral entropy (entropy on PXX (f )) Others, e.g. fuzzy entropy, multiscale entropy.

Features of nonlinear analysis - 3

Entropy: Estimates of the entropy of xt = [xt , xt−τ ,..., xt−(m−1)τ ]: P H(X)= x pX(x) log pX(x) Approximate entropy, ApEn (uses a bandwidth in the estimation of pX(x)) Sample entropy (similar to ApEn) Permutation entropy (entropy on ranks of the components in xt )

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Others, e.g. fuzzy entropy, multiscale entropy.

Features of nonlinear analysis - 3

Entropy: Estimates of the entropy of xt = [xt , xt−τ ,..., xt−(m−1)τ ]: P H(X)= x pX(x) log pX(x) Approximate entropy, ApEn (uses a bandwidth in the estimation of pX(x)) Sample entropy (similar to ApEn) Permutation entropy (entropy on ranks of the components in xt )

Spectral entropy (entropy on PXX (f ))

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Features of nonlinear analysis - 3

Entropy: Estimates of the entropy of xt = [xt , xt−τ ,..., xt−(m−1)τ ]: P H(X)= x pX(x) log pX(x) Approximate entropy, ApEn (uses a bandwidth in the estimation of pX(x)) Sample entropy (similar to ApEn) Permutation entropy (entropy on ranks of the components in xt )

Spectral entropy (entropy on PXX (f )) Others, e.g. fuzzy entropy, multiscale entropy.

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 correlation dimension, estimates the fractal dimension of xt = [xt , xt−τ ,..., xt−(m−1)τ ] (assuming chaos): m d(xi , xj ): distance of two points in IR ν scaling of probability of distance with r, p(d(xi , xj ) < r) ∝ r , e.g. ν: the dimension, estimated by the slope of log p(r) vs log r. Higuchi dimension (distance is defined in a different way). Lempel-Ziv algorithmic complexity (turning the signal to a series of symbols by discretization and searching for new symbol patterns in the series).

Source: [Kugiumtzis et al, Int. J of Bioelectromagnetism, 2007]

[Kugiumtzis and Tsimpiris, J. Stat. Softw., 2010], MATS module in Matlab

Features of nonlinear analysis - 4

Dimension and Complexity:

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Higuchi dimension (distance is defined in a different way). Lempel-Ziv algorithmic complexity (turning the signal to a series of symbols by discretization and searching for new symbol patterns in the series).

Source: [Kugiumtzis et al, Int. J of Bioelectromagnetism, 2007]

[Kugiumtzis and Tsimpiris, J. Stat. Softw., 2010], MATS module in Matlab

Features of nonlinear analysis - 4

Dimension and Complexity:

correlation dimension, estimates the fractal dimension of xt = [xt , xt−τ ,..., xt−(m−1)τ ] (assuming chaos): m d(xi , xj ): distance of two points in IR ν scaling of probability of distance with r, p(d(xi , xj ) < r) ∝ r , e.g. ν: the dimension, estimated by the slope of log p(r) vs log r.

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Lempel-Ziv algorithmic complexity (turning the signal to a series of symbols by discretization and searching for new symbol patterns in the series).

Source: [Kugiumtzis et al, Int. J of Bioelectromagnetism, 2007]

[Kugiumtzis and Tsimpiris, J. Stat. Softw., 2010], MATS module in Matlab

Features of nonlinear analysis - 4

Dimension and Complexity:

correlation dimension, estimates the fractal dimension of xt = [xt , xt−τ ,..., xt−(m−1)τ ] (assuming chaos): m d(xi , xj ): distance of two points in IR ν scaling of probability of distance with r, p(d(xi , xj ) < r) ∝ r , e.g. ν: the dimension, estimated by the slope of log p(r) vs log r. Higuchi dimension (distance is defined in a different way).

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Source: [Kugiumtzis et al, Int. J of Bioelectromagnetism, 2007]

[Kugiumtzis and Tsimpiris, J. Stat. Softw., 2010], MATS module in Matlab

Features of nonlinear analysis - 4

Dimension and Complexity:

correlation dimension, estimates the fractal dimension of xt = [xt , xt−τ ,..., xt−(m−1)τ ] (assuming chaos): m d(xi , xj ): distance of two points in IR ν scaling of probability of distance with r, p(d(xi , xj ) < r) ∝ r , e.g. ν: the dimension, estimated by the slope of log p(r) vs log r. Higuchi dimension (distance is defined in a different way). Lempel-Ziv algorithmic complexity (turning the signal to a series of symbols by discretization and searching for new symbol patterns in the series).

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Features of nonlinear analysis - 4

Dimension and Complexity:

correlation dimension, estimates the fractal dimension of xt = [xt , xt−τ ,..., xt−(m−1)τ ] (assuming chaos): m d(xi , xj ): distance of two points in IR ν scaling of probability of distance with r, p(d(xi , xj ) < r) ∝ r , e.g. ν: the dimension, estimated by the slope of log p(r) vs log r. Higuchi dimension (distance is defined in a different way). Lempel-Ziv algorithmic complexity (turning the signal to a series of symbols by discretization and searching for new symbol patterns in the series).

Source: [Kugiumtzis et al, Int. J of Bioelectromagnetism, 2007]

[Kugiumtzis and Tsimpiris, J. Stat. Softw., 2010], MATS module in Matlab

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3 Example: 5 datasets of healthy extracranial EEG (A,B) and intracranial interictal (C,D) and ictal (E) EEG

[Bao et al, Comput Intell Neurosci, 2011]

Kugiumtzis Dimitris Computational Neuroscience - Lecture 3