Pymrt Documentation Release 1.0.0

pymrt Documentation Release 1.0.0

Tinghui Wang

May 14, 2018

Contents:

1 Python Environment Setup 1 1.1 Python 3.6 Environment...... 1 1.2 Setup Tensorﬂow...... 2 1.3 Setup Mayavi...... 2 1.4 Known Issues...... 3

2 Multi-Target Tracking 5 2.1 Dynamic Models for Multi-target Tracking...... 5 2.2 Gaussian Component...... 10 2.3 Gaussian-Mixture Probability Hypothesis Density Filter...... 12 2.4 API...... 16

3 Reference 21 3.1 Multi-Target Tracking...... 21

4 Indices and tables 23

Bibliography 25

i ii CHAPTER 1

Python Environment Setup

As the packages has a few package dependencies and get them set up properly for your python environment may be tricky sometimes. The following provides a general suggestion on how to set up each packages for better performance on various operating systems.

1.1 Python 3.6 Environment

On MacOS and Windows 10, I personally recommend MiniConda package installer for python environment. Mini- Conda provides most numerical calculation packages such as numpy and scipy, pre-compiled for all operating systems with Intel Math kernel Library - probably the best performance you can get as pre-compiled binaries. It also includes virtual environment management so that you can have multiple Python environments co-existing on your machine. MiniConda is installed with minimal packages and you can add additional packages that you need incrementally to keep a minimal footprint on your hard drive.

1.1.1 Windows Setup

Setup MiniConda on windows, simply follow the link and download the installer for you Windows OS. Run the installer to install MiniConda to your machine.

1.1.2 Ubuntu 18.04 Setup

Setup MiniConda on Linux, simple follow the link and download the bash installer for Linux operationg system. In terminal, make the downloaded bash script executable, and run it with sudo command as follows.

$ sudo ./Miniconda3-latest-Linux-x86_64.sh

In my case, I installed it under /opt/miniconda3 so that it is accessible for all users. Add the path of Miniconda to ~/.bashrc:

1 pymrt Documentation, Release 1.0.0

export PATH="/opt/miniconda3/bin:$PATH"

1.1.3 MacOS Setup

In MacOS, I would recommend using homebrew as package manager to install MiniConda. Install homebrew in terminal:

$ /usr/bin/ruby -e" $(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/

˓→master/install)"

The command above fetches homebrew from GitHub and installs Xcode command line tools as well. Add homebrew to path

$ echo export PATH='/usr/local/bin:$PATH' >> ~/.profile

Install basic MiniConda environment using homebrew

$ brew install wget $ wget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh $ bash Miniconda3-latest-MacOSX-x86_64.sh $ rm ~/Miniconda3-latest-MacOSX-x86_64.sh $ echo export PATH='~/miniconda3/bin:$PATH' >> ~/.profile $ source ~/.profile $ conda install anaconda $ conda update --all

1.2 Setup Tensorﬂow

For install tensorflow for your operating system, you can find instruction on tensorflow document page here. Note that on Windows with Conda environment, there is a chance that an out-dated html5lib package dependency may break the Conda setup. As a walk around, you can run $pip install html5lib==1.0b10 to correct it. The fix has been merged into tensorflow source tree, but has not released yet.

1.3 Setup Mayavi

Mayavi library is used for 3D plot and visualization. However, set it up properly takes quite some work.

1.3.1 Windows Setup

First, make sure that you have Visual Studio installed. In my case, I use VS2017 Community for compilation. More- over, in VS2017, you need to enable *Python Support* and have *Python Native Development Tools* installed. Start *x64 Native Tools Command Prompt for VS2017* in start menu and import Conda Python environment scripts (usually named as activate.bat). The default one for base environment is at Scripts\activate.bat under conda installation directory. (Replace C:\Anaconda3 in the following command with your installation path of conda).

2 Chapter 1. Python Environment Setup pymrt Documentation, Release 1.0.0

> C:\Anaconda3\Scripts\activate.bat C:\Anaconda3

Install pyside 1.2.4

> conda install -c conda-forge pyside=1.2.4

However, if you have pyqt package installed on your system, you may see it fails with error complaining about version conﬂicts. Remove pyqt ﬁrst.

> conda uninstall pyqt

Install VTK from clinicalgraphics

> conda install -c clinicalgraphics vtk

Due to various bugs and compatibility issue, install mayavi, traits and pyface from source (Github).

> pip install git+https://github.com/enthought/envisage.git > pip install git+https://github.com/enthought/traitsui.git > pip install git+https://github.com/enthought/pyface.git > pip install git+https://github.com/enthought/mayavi.git

1.3.2 Ubuntu 18.04 Setup

You can install the mayavi in the same way as in Windows. In addition to the previous steps, you als need to install libcanberra-gtk-module and libcanberra-gtk3-module using system package manager.

$ sudo apt install libcanberra-gtk-module libcanberra-gtk3-module

1.3.3 MacOS Setup

First, install VTK using homebrew.

$ brew install vtk --with-python3 --without-python --with-qt

Install pyside 1.2.4 conda install pyside

Due to various bugs and compatibility issue, install mayavi, traits and pyface from source (Github). It takes a while to install and compile all of them from the source.

$ pip install git+https://github.com/enthought/envisage.git $ pip install git+https://github.com/enthought/traitsui.git $ pip install git+https://github.com/enthought/pyface.git $ pip install git+https://github.com/enthought/mayavi.git

1.4 Known Issues

1.4.1 mlab.axes() causes exception in Mayavi 4.5.0

When use mlab.axes, the following exception message is observed in terminal:

1.4. Known Issues 3 pymrt Documentation, Release 1.0.0

TypeError: SetInputData argument1:method requires a vtkDataSet, a vtkPolyDataNormals was provided.( in _wrap_call)

AttributeError:'PolyDataNormals' object has no attribute'bounds'

You can ﬁnd ﬁx on mayavi github page at #474.

1.4.2 UnicodeDecodeError while trying to close mayavi

When you close mayavi window, you may saw the following error and the window is not closed unless you kills it using processor manager. The message may read:

UnicodeDecodeError:'utf-8' codec can't decode byte 0xff in position 3: invalid start

˓→byte

The issued is tracked on mayavi Github page at #576. The ﬁx for the issue is merged to master branch on Feb 14, 2018.

4 Chapter 1. Python Environment Setup CHAPTER 2

Multi-Target Tracking

2.1 Dynamic Models for Multi-target Tracking

Namespace pymrt.tracking.models includes multiple classes for commonly used dynamic models. They serve as the basis of state transition modelling for multi-target tracking algorithm.

2.1.1 GM-based Constant Velocity Model pymrt.tracking.models.CVModel provides a base class for constant velocity maneuvering object in n- dimensional space. The model is composed of the following parts of information:

State Space

The state of an object maneuvering in a n-dimensional space with constant velocity can be expressed as an array composed of its location in the space and velocity at a given time.

푇 푥 = [푠푑1 , 푠푑2 , ..., 푠푑푛 , 푣푑1 , 푣푑2 , ..., 푣푑푛 ]

The state space where 푥 is drawn from is usually R2푛.

State Update

If the object is moving according to a constant velocity in the n-dimensional space, then the new state (after time 푡) can be expressed as

푇 푥푡+1 = [푠푑1 + 푣푑1 푡, 푠푑2 + 푣푑2 푡, ..., 푠푑푛 + 푣푑푛 푡, 푣푑1 , 푣푑2 , ..., 푣푑푛 ] In matrix format, we can rewrite it as

푥푡+1 = 퐹 · 푥푡

5 pymrt Documentation, Release 1.0.0

where [︂퐼 퐼 푡]︂ 퐹 = 푛 푛 0 퐼푛

퐼푛 here is identity matrix of n-dimension.

State Error Estimation

Well, there is always error associated with a model, and to simulate the error, the following error term (only ﬁrst order is considered) is added. 푇 Given disturbance 푤 = [푤푑0 , 푤푑1 , ..., 푤푑푛 ] , the state updated with error estimation can be written in matrix form as follows:

푥푡+1 = 퐹 · 푥푡 + 퐺 · 푤

where [︂퐼 퐼 푡]︂ 퐹 = 푛 푛 0 퐼푛

and [︂ 1 푡2퐼 ]︂ 퐺 = 2 푛 푡퐼푛

Measurement

Measurement, sometimes also referred to as observation, of a target at state 푥 can be derived with the following equation

푧푡 = 퐻 · 푥푡

퐻 is called measurement multiplier, and 푧푡 is the observation of the target at time 푡. Usually, only the space location of target can be measured, so the measurement multipler 퐻 is

[︂퐼 ]︂ 퐻 = 푛 0푛

Measurement Error Estimation

Measurement will have error. The error is modeled by a uncertainty measure around the exact measurement. In Gaussian Mixture model, it is represented by a multi-variant Gaussian covariance matrix 퐷. If each dimension are independent of each other,

2 퐷 = 휎 퐼푛

2.1.2 GM-based Constant Velocity Model with Track ID

pymrt.tracking.models.CVIDModel provides a base class for constant velocity maneuvering object in n- dimensional space with track ID embedded in state vector.

6 Chapter 2. Multi-Target Tracking pymrt Documentation, Release 1.0.0

State Space

In order to accomplish data-track association during target tracking, the track ID needs to be append to state vector. At a given time, the state vector is described as follows

푇 푥 = [푠푑0 , 푠푑1 , ..., 푠푑푛 , 푣푑0 , 푣푑1 , ..., 푣푑푛 , 푡푖푑]

The state space is usually R2푛 × N.

State Update

The state update, without considering spawning (one object separated into two), can be expressed as follows:

푥푡+1 = 퐹 · 푥푡

where ⎡ ⎤ 퐼푛 퐼푛푡 0 퐹 = ⎣ 0 퐼푛 0⎦ 0 0 1

State Error Estimation

푇 Given disturbance 푤 = [푤푑0 , 푤푑1 , ..., 푤푑푛 ] , the state updated with error estimation can be written in matrix form as follows:

푥푡+1 = 퐹 · 푥푡 + 퐺 · 푤

where ⎡ ⎤ 퐼푛 퐼푛푡 0 퐹 = ⎣ 0 퐼푛 0⎦ 0 0 1 and ⎡ 1 2 ⎤ 2 푡 퐼푛 퐺 = ⎣ 푡퐼푛 ⎦ 0

Measurement

Measurement equation of the model can be expressed as:

푧푡 = 퐻 · 푥푡 where ⎡ ⎤ 퐼푛 퐻 = ⎣0푛⎦ 0

2.1.3 Other Model Parameters for MTT

In additional to the model parameters mentioned above, there are additional parameters deﬁned for the tracking algorithm to use.

2.1. Dynamic Models for Multi-target Tracking 7 pymrt Documentation, Release 1.0.0

Target Birth

In Gaussian-Mixture PHD algorithm, target birth is represented by a series of Gaussian Mixtures, where the sum of all GMs represents the possibility of target birth in the state space.

Target Persistence

Target persistence is modeled by parameter 푝푠. 푝푠 is the probability of the target being persistent into the next time step. 1 − 푝푠 gives the probability of the target death at the current time step.

Target Detection

In some cases, targets are not picked up by the observers. Parameter 푝푑 gives the probability of the detection of a target in at current time step.

Clutter Process

Observations are also noisy. All the false-alarms in the observation are modeled by the clutter process. It is usually safe to assume that the clutter process is a Poison Point Process with parameter 휆푐, i.e. the number of false-alarms follows Poisson distribution of 휆푐, where the exact observed position of those false-alarms follows a spacial distribution 푐(푧). 푐(푧) can be represented as a series of Gaussian Mixtures or a uniform distribution across space (constant).

Gaussian Mixture Approximation

While propagating Gaussian Mixtures over time causes an exponential growth of the number of GMs, the following parameters are set for the trade-off between computation and accuracy.

푔푚푇 is the truncating threshold. If the weight of a Gaussian Mixture is below the threshold, it is considered insigniﬁ- cant and can be safely discarded without sacriﬁcing a lot of tracking performance.

푔푚푈 deﬁnes the merging threshold of GMs. If multiple GMs are centered at about the same location, the summation of two GMs can be approximate by a single Gaussian Mixture.

푔푚퐽푚푎푥 sets the maximum number of Gaussian Mixture to track in the process.

2.1.4 Appendix

Derivation of Error Estimation

This section, we provide derivation of the constant velocity model state update and state error estimation. Here, we consider 2D and 3D models for convenience of demonstration. 푇 If the target moves in a 2D space, then the state space has a dimensionality of 4: [푥, 푦, 푣푥, 푣푦] . If the target moves in 푇 a 3D space, then the state space has a dimensionality of 6: [푥, 푦, 푧, 푣푥, 푣푦, 푣푧] .(푧 in this case represent a dimension in state space). The linear constant velocity model provides a linear relationship between the target states before and after a time step. Mathematically,

푥푘+1 = 퐹 · 푥푘

8 Chapter 2. Multi-Target Tracking pymrt Documentation, Release 1.0.0

Now, consider a 2D constant velocity model. At time step 푘 + 1 with time interval 푡 , 푣푥 , 푣푦 stays the same, but 푥 and 푦 are both increased by 푣푥 · 푡 and 푣푦 · 푡 respectively. Formally, ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ 푥푘+1 1 0 푡 0 푥푘 ⎢ 푦푘+1 ⎥ ⎢ 0 1 0 푡 ⎥ ⎢ 푦푘 ⎥ ⎢ ⎥ = ⎢ ⎥ · ⎢ ⎥ ⎣ 푣푥 ⎦ ⎣ 0 0 1 0 ⎦ ⎣ 푣푥 ⎦ 푣푦 0 0 0 1 푣푦

In another word,

⎡ 1 0 푡 0 ⎤ ⎢ 0 1 0 푡 ⎥ 퐹2퐷 = ⎢ ⎥ ⎣ 0 0 1 0 ⎦ 0 0 0 1

The equation above provides an ideal physical constant velocity model. However, in real-world applications, due to various environmental noise such as air perturbation, an additive error term needs to be introduced. 푇 In 4D CV model, we assume that there is a white noise w푘 = [푤푥, 푤푦] added on top of the velocity of the particle that causes deviation between the actual model and the ideal physical model. Thus, the following physical dynamic equation holds: ⎡ ⎤ ⎡ ⎤ 푥 푣푥 휕x(푡) 휕 ⎢ 푦 ⎥ ⎢ 푣푦 ⎥ = ⎢ ⎥ = ⎢ ⎥ 휕푡 휕푡 ⎣ 푣푥 ⎦ ⎣ 푎푥 ⎦ 푣푦 푎푦 ⎡ 0 0 1 0 ⎤ ⎡ 푥 ⎤ ⎡ 0 0 ⎤ [︂ ]︂ ⎢ 0 0 0 1 ⎥ ⎢ 푦 ⎥ ⎢ 0 0 ⎥ 푤푥 = ⎢ ⎥ · ⎢ ⎥ + ⎢ ⎥ · ⎣ 0 0 0 0 ⎦ ⎣ 푣푥 ⎦ ⎣ 1 0 ⎦ 푤푦 0 0 0 0 푣푦 0 1 = 퐴 · x(푡) + 퐵 · w(푡) where ⎡ 0 0 1 0 ⎤ ⎡ 0 0 ⎤ ⎢ 0 0 0 1 ⎥ ⎢ 0 0 ⎥ 퐴 = ⎢ ⎥ , and퐵 = ⎢ ⎥ ⎣ 0 0 0 0 ⎦ ⎣ 1 0 ⎦ 0 0 0 0 0 1

Solve the differential equation with Laplace transformation: 휕 x = 퐴 · + 퐵 · 휕푡 x w 푠x(푠) − x(0) = 퐴 · x(푠) + 퐵 · w(푠) (푠 · 퐼 − 퐴) x(푠) = x(0) + 퐵 · w(푠) −1 −1 x(푠) = (푠 · 퐼 − 퐴) x(0) + (푠 · 퐼 − 퐴) · 퐵 · w(푠)

⎡ ⎤−1 ⎡ 1 1 ⎤ 푠 0 −1 0 푠 0 푠2 0 1 1 −1 ⎢ 0 푠 0 −1 ⎥ ⎢ 0 푠 0 푠2 ⎥ (푠 · 퐼 − 퐴) = ⎢ ⎥ = ⎢ 1 ⎥ ⎣ 0 0 푠 0 ⎦ ⎣ 0 0 푠 0 ⎦ 1 0 0 0 푠 0 0 0 푠

2.1. Dynamic Models for Multi-target Tracking 9 pymrt Documentation, Release 1.0.0

Now, take inverse Laplace transformation, ⎡ 푢(푡) 0 푡 · 푢(푡) 0 ⎤ ⎢ 0 푢(푡) 0 푡 · 푢(푡) ⎥ x(푡) = ⎢ ⎥ · x(0) ⎣ 0 0 푢(푡) 0 ⎦ 0 0 0 푢(푡) ⎡ 푢(푡 − 휏) 0 (푡 − 휏) · 푢(푡 − 휏) 0 ⎤ ⎡ 0 0 ⎤ ∫︁ 푡 ⎢ 0 푢(푡 − 휏) 0 (푡 − 휏) · 푢(푡 − 휏) ⎥ ⎢ 0 0 ⎥ + ⎢ ⎥ · ⎢ ⎥ · w(휏)푑휏 0 ⎣ 0 0 푢(푡 − 휏) 0 ⎦ ⎣ 1 0 ⎦ 0 0 0 푢(푡 − 휏) 0 1 As we are deriving discrete-time motion model, let 푡 > 0 be the time interval of two continuous sample. Thus, the step function 푢(푡) = 1, and the 푢(푡 − 휏) = 1 as 휏 < 푡. Moreover, assume that with in the time step, w(휏) = 푤 is constant. ⎡ 1 0 푡 0 ⎤ ⎡ 1 0 푡 − 휏 0 ⎤ ⎡ 0 ⎤ ∫︁ 푡 ⎢ 0 1 0 푡 ⎥ ⎢ 0 1 0 푡 − 휏 ⎥ ⎢ 0 ⎥ x(푡) = ⎢ ⎥ · x(0) + ⎢ ⎥ · ⎢ ⎥ ⎣ 0 0 1 0 ⎦ 0 ⎣ 0 0 1 0 ⎦ ⎣ 푤푥(휏)푑휏 ⎦ 0 0 0 1 0 0 0 1 푤푦(휏)푑휏 ⎡ ⎤ ⎡ ⎤ 1 0 푡 0 (푡 − 휏)푤푥(휏)푑휏 ∫︁ 푡 ⎢ 0 1 0 푡 ⎥ ⎢ (푡 − 휏)푤푦(휏)푑휏 ⎥ = ⎢ ⎥ · x(0) + ⎢ ⎥ ⎣ 0 0 1 0 ⎦ 0 ⎣ 푤푥(휏)푑휏 ⎦ 0 0 0 1 푤푦(휏)푑휏 ⎡ ⎤ ⎡ 푡2 ⎤ 1 0 푡 0 2 0 2 ⎢ 0 1 0 푡 ⎥ ⎢ 0 푡 ⎥ = ⎢ ⎥ · x(0) + ⎢ 2 ⎥ · w ⎣ 0 0 1 0 ⎦ ⎣ 푡 0 ⎦ 0 0 0 1 0 푡 Alternatively,

x푘+1 = 퐹 · x푘 + 퐺 · w푘 where ⎡ ⎤ ⎡ 푡2 ⎤ 1 0 푡 0 2 0 2 ⎢ 0 1 0 푡 ⎥ ⎢ 0 푡 ⎥ 퐹2퐷 = ⎢ ⎥ and퐺2퐷 = ⎢ 2 ⎥ ⎣ 0 0 1 0 ⎦ ⎣ 푡 0 ⎦ 0 0 0 1 0 푡

w푘 is a Gaussian noise with standard deviation of 휎. Similarly, for 3D space, ⎡ ⎤ ⎡ 푡2 ⎤ 1 0 0 푡 0 0 2 0 0 0 1 0 0 푡 0 ⎢ 푡2 ⎥ ⎢ ⎥ ⎢ 0 2 0 ⎥ ⎢ ⎥ ⎢ 푡2 ⎥ ⎢ 0 0 1 0 0 푡 ⎥ ⎢ 0 0 ⎥ 퐹3퐷 = ⎢ ⎥ and퐺3퐷 = ⎢ 2 ⎥ ⎢ 0 0 0 1 0 0 ⎥ ⎢ 푡 0 0 ⎥ ⎢ 0 0 0 0 1 0 ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ 0 푡 0 ⎦ 0 0 0 0 0 1 0 0 푡

2.2 Gaussian Component

pymrt.tracking.utils.GaussianComponent class implements a object class for Gaussian Component used in Guassian Mixture propagation calculation. Each Gaussian Component is deﬁned with weight, mean vector, and covariant matrix. Methods in the class provides calculation for merging, and propagating Gaussian mixtures. The following shows the proper usage and test case in verifying those functions.

10 Chapter 2. Multi-Target Tracking pymrt Documentation, Release 1.0.0

""" This file tests Gaussian Component Calculation """ import numpy as np import scipy.stats from pymrt.tracking.utils import GaussianComponent from mayavi import mlab def plot_2d_gm(): """ This function plot two 2D Gaussian distribution density side by side centralized at (-500, 0) and (500, 0) for comparison.

In this example, the Gaussian density has a covariance matrix of .. math:: cov = \left[ \begin{matrix} 10000, 0 0, 10000 \end{matrix} \right]

The peak of the density distribution is .. math:: max(\mathcal{N}) = \frac{1}{\sqrt{2\pi 10000}} = 1.59\times 10^{-5}

You can verify the value according to the plotted graph. """ figure= mlab.figure('GMGaussian') X, Y= np.mgrid[-1000:1000:10,-500:500:10] Z_GM= np.zeros(X.shape, dtype=np.float) Z_PDF= np.zeros(X.shape, dtype=np.float)

GM_mean= np.array([[-500], [0.]]) PDF_mean= np.array([[500], [0.]])

# Covariance matrix - with std to be 100 on both dimensions cov= np.eye(2) * 10000

gm= GaussianComponent(n=2, weight=1., mean=GM_mean, cov=cov) mvnormal= scipy.stats.multivariate_normal(mean=PDF_mean.flatten(), cov=cov)

for i in range(X.shape[0]): for j in range(X.shape[1]): eval_x= np.array([[X[i, j]], [Y[i, j]]]) Z_GM[i, j]= gm.dmvnormcomp(eval_x) Z_PDF[i, j]= mvnormal.pdf(eval_x.flatten())

print(Z_GM- Z_PDF)

scale_factor= max(np.max(Z_GM), np.max(Z_PDF)) # mlab.surf(X, Y, f, opacity=.3, color=(1., 0, 0)) mlab.surf(X, Y, Z_GM * (2000/ scale_factor), opacity=.3, color=(1.,0,0)) mlab.surf(X, Y, Z_PDF * (2000/ scale_factor), opacity=.3, color=(0., 1.,0))

mlab.outline(None, color=(.7,.7,.7), extent=[-1000, 1000,-500, 500, 0, 2000])

(continues on next page)

2.2. Gaussian Component 11 pymrt Documentation, Release 1.0.0

(continued from previous page) mlab.axes(None, color=(.7,.7,.7), extent=[-1000, 1000,-500, 500,0, 2000], ranges=[-1000, 1000,-500, 500,0, scale_factor], nb_labels=6) mlab.show() if __name__ =='__main__': plot_2d_gm()

Function plot_2d_gm() plot two 2D Gaussian distribution density side by side centralized at (-500, 0) and (500, 0) for comparison. In this example, the Gaussian density has a covariance matrix of [︂10000 0 ]︂ 푐표푣 = 0 10000 The peak of the density distribution is 1 푚푎푥(풩 ) = √ = 1.59 × 10−5 2휋10000 You can verify the value according to the plotted graph.

2.3 Gaussian-Mixture Probability Hypothesis Density Filter

GM-PHD, proposed by Vo et. al. in 2006 [Vo2006], is one of the close-form implementation of PHD ﬁlter for multi-target tracking. GM-PHD is developed with following assumptions: 1. Target Independence: each target evolves and generates observations independently of one another 2. Poisson Clutter: clutter porcess is Poisson and independent of target-originated measurements.

3. Poisson RFS: the predicted multi-target RFS governed by 푝푘|푘−1 is Poisson (See [Mahler2003]). 4. Linear Gaussian Dynamic Model: each target follows a linear Gaussian dynamic model represented by

푓푘|푘−1 (푥|휁) = 풩 (푥; 퐹푘−1휁, 푄푘−1)

5. Target Birth: the PHD of the target birth of RFS are gaussian mixtures of the form

퐽훾,푘 ∑︁ (푗) (푗) (푗) 퐷훾,푘(푥) = 푤훾,푘풩 (푥; 푚훾,푘, 푃훾,푘) 푗=1

2.3.1 Predictor

If the posterior PHD of multi-target RFS at time 푘 − 1 is represented in forms of Gaussian Mixtures:

퐽푘−1 ∑︁ (푗) (푗) (푗) 퐷푘−1(푥) = 푤푘−1풩 (푥; 푚푘−1, 푃푘−1) 푗=1

The predicted intesity 퐷푘|푘−1 at time 푘 is composed of three terms: target birth 퐷훾,푘(푥), target persistence 퐷푆,푘|푘−1(푥) and target spawning 퐷훽,푘|푘−1(푥).

퐷푘|푘−1(푥) = 퐷푆,푘|푘−1(푥) + 퐷훾,푘(푥) + 퐷훽,푘|푘−1(푥)

12 Chapter 2. Multi-Target Tracking pymrt Documentation, Release 1.0.0

2.3. Gaussian-Mixture Probability Hypothesis Density Filter 13 pymrt Documentation, Release 1.0.0

In the application of multi-resident tracking, there is no target spawning, so term 퐷훽,푘|푘−1(푥) is ignored in this implementation. The persistence term updates each existing Gaussian Components according to Kalman update equation (for proof, see Kalman Update for Gaussian Components).

퐽푘−1 ∑︁ (푗) (푗) (푗) 퐷푆,푘|푘−1(푥) = 푝푆,푘 푤푘−1풩 (푥; 푚푆,푘|푘−1, 푃푆,푘|푘−1) 푗=1 where (푗) (푗) 푚푆,푘|푘−1 = 퐹푘푚푘−1 and

(푗) (푗) 푇 푃푆,푘|푘−1 = 퐹푘푃푘−1퐹푘 + 푄푘 This calculation is implemented in gmphd_predictor().

2.3.2 Corrector

Assume that the predicted intensity (i.e. the output of predictor) for time 푘 is a Gaussian mixture of the form

푧∈푍푘

푝퐷,푘 is the probability that the target will be observed. The term (1 − 푝퐷,푘)퐷푘|푘−1(푥) represents the portion that is not observed at time step 푘 - and thus do not need to be corrected by the set of measurement 푍푘 at time 푘. The term ∑︀ 퐷 (푥; 푧) calculated the posterior PHD corrected according to the measurement set at current time 푧∈푍푘 퐷,푘 step. In the equation,

퐽푘|푘−1 ∑︁ (푗) (푗) (푗) 퐷퐷,푘(푥; 푧) = 푤푘 (푧)풩 (푥; 푚푘 (푧), 푃푘 ) 푗=1 The updated weight is (푗) (푗) (푗) 푝퐷,푘푤푘|푘−1푞푘 (푧) 푤푘 (푧) = ∑︀퐽푘|푘−1 (푙) (푙) 휅푘(푧) + 푝퐷,푘 푙=1 푤푘|푘−1푞푘 (푧) where

휅푘(푧) = 휆푐푐(푧)

14 Chapter 2. Multi-Target Tracking pymrt Documentation, Release 1.0.0

2.3.3 Appendix

Kalman Update for Gaussian Components

Assume that each target follows a linear Gaussian dynamic model, i.e.

푓푘|푘−1 (푥|휁) = 풩 (푥; 퐹푘−1휁, 푄푘−1)

where 휁 is the mean state vector of the target at time 푘 − 1, 퐹푘−1 is the state linear multiplier of dynamic model, 푄푘−1 is the covariance matrix of error estimation in dynamic model. If the posterior intensity of a target is represented by a Gaussian Component

푓푘−1(푥) = 푤푘−1풩 (푥; 푚푘−1, 푃푘−1) where 푚푘−1 is the mean vector and 푃푘−1 is the covariance matrix. According to the linear Gaussian dynamic model, at time 푘, the updated probability density of the target will be

푓푘(푥) = 푤푘−1풩 (푥; 푚푘, 푃푘)

where

푚푘 = 퐹푘−1푚푘−1

and

푇 푃푘 = 푄푘−1 + 퐹푘−1푃푘−1퐹푘−1

This calculation is implemented by kalman_update(). Exemple 2.1. A target is modeled by a dynamic model

푥푘+1 = 퐹푘푥푘 + 푤푘

where the error term 푤푘 has a mean of zero and covariance of 푄푘.

At time 푘, random variable 푥푘 follows Gaussian distribution 풩 (푥푘; 푚푘, 푃푘).

The mean of variable 푥푘+1 is

퐸[푥푘+1] = 퐸[퐹푘푥푘 + 푤푘]

= 퐹푘퐸[푥푘] + 퐸[푤푘]

= 퐹푘푚푘 + 0

= 퐹푘푚푘

The covariance of variable 푥푘+1 is

푇 푇 푐표푣[푥푘+1] = 퐸[푥푘+1푥푘+1] − 퐸[푥푘+1]퐸[푥푘+1] 푇 푇 푇 푇 푇 = 퐸[퐹푘푥푘푥푘 퐹푘 + 푤푘푤푘 ] − 퐹푘푚푘푚푘 퐹푘 푇 푇 푇 푇 = 퐹푘퐸[푥푘푥푘 ]퐹푘 + 푄푘 − 퐹푘푚푘푚푘 퐹푘 푇 푇 푇 푇 = 퐹푘(푐표푣[푥] + 퐸[푥]퐸[푥 ])퐹푘 + 푄푘 − 퐹푘푚푘푚푘 퐹푘 푇 푇 푇 푇 = 퐹푘(푃푘 + 푚푘푚푘 )퐹푘 + 푄푘 − 퐹푘푚푘푚푘 퐹푘 푇 = 퐹푘푃푘퐹푘 + 푄푘

2.3. Gaussian-Mixture Probability Hypothesis Density Filter 15 pymrt Documentation, Release 1.0.0

Gaussian Mixture Corrector Proof

Assume that target measurement follows a linear Gaussian observation model

퐿(푧|푥푘) = 풩 (푧; 퐻푘푥푘, 푅푘)

According to Theorem 6 in [Mahler2003], the “Single-Sensor” Bayes Update formula for PHD is (︃ )︃ ∼ ∑︁ 푝퐷(푥)퐿푧(푥) 퐷푘(푥) = 1 − 푝퐷(푥) + 퐷푘|푘−1(푥) 휆푐(푧) + 퐷푘|푘−1[푝퐷퐿푧] 푧∈푍푘

Term 1 − 푝퐷(푥) is the portion of predicted PHD that does not need to be corrected (as the portion is not observed). You can view the nominator of the fraction is the weight to the predicted PHD based on the likelihood of getting the observation 푧. The denominator of the fraction can be viewed as a normalization factor [Mahler2004].

퐽푘|푘−1 ∑︁ (푖) (푖) (푖) 퐿(푧, 푥푘) = 풩 (푧; 퐻푘푥, 푅푘) 푤푘|푘−1풩 (푥; 푚푘|푘−1, 푃푘|푘−1) 푖=1

푝퐷(푥)퐿푧(푥) 휆푐(푧) + 퐷푘|푘−1[푝퐷퐿푧] ∑︀퐽푘|푘−1 (푖) (푖) (푖) 푝퐷,푘풩 (푧; 퐻푘푥, 푅푘) 푖=1 푤 풩 (푥; 푚 , 푃 ) = 푘|푘−1 푘|푘−1 푘|푘−1 ∫︀ ∑︀퐽푘|푘−1 (푖) (푖) (푖) 휆푐(푧) + 푝퐷,푘 풩 (푧; 퐻푘휁, 푅푘) 푖=1 푤푘|푘−1풩 (휁; 푚푘|푘−1, 푃푘|푘−1)푑휁 ∑︀퐽푘|푘−1 (푖) (푖) (푖) 푝퐷,푘 푖=1 푤 풩 (푧; 퐻푘푥, 푅푘)풩 (푥; 푚 , 푃 ) = 푘|푘−1 푘|푘−1 푘|푘−1 ∑︀퐽푘|푘−1 (푖) ∫︀ (푖) (푖) 휆푐(푧) + 푝퐷,푘 푖=1 푤푘|푘−1 풩 (푧; 퐻푘휁, 푅푘)풩 (휁; 푚푘|푘−1, 푃푘|푘−1)푑휁 Based on calculus,

2.4 API

2.4.1 Gaussian Component class pymrt.tracking.utils.GaussianComponent(n, weight, mean, cov) Gaussian Mixture Component A Gaussian mixture component is composed of weight (푤), mean vector (휇), and covariance matrix (Σ). A multivariate Gaussian distribution 풩 (휇, Σ) evaluated at position 푥 is given by

1 − 1 (푥−휇)푇 Σ−1(푥−휇) 푓(푥) = 푒 2 √︀(2휋)푘 |Σ|

A multivariate Gaussian Mixture Component, denoted 풩 (푤, 휇, Σ) is a equivalent to a multivariate Gaussian distribution with a weight factor, i.e. 푤풩 (휇, Σ). As a result, a GM evaluated at position 푥 is given by

푓퐺푀 (푥) = 푤푓(푥)

where 푓(푥) is the density of a multivariate Gaussian distribution evaluated at location 푥.

16 Chapter 2. Multi-Target Tracking pymrt Documentation, Release 1.0.0

n int – Dimensionality of the space the Gaussian component is evaluated on. weight float – Weight of this Gaussian Component. mean numpy.ndarray – Mean vector (column) 휇 of shape (n, 1). cov numpy.ndarray – Covariance matrix of shape (n, n). dmvnorm(x) Density of multivariate normal distribution evaluated at location x dmv_part1 calculates

1 − 1 (푥−휇)푇 Σ−1(푥−휇) 푒 2 √︀(2휋)푘 |Σ|

dmvnormcomp(x) Density of multivariate GM component evaluated at location x kalman_update(F, Q) Update a GM Component based on a linear prediction model. Assume each target follows a linear Gaussian dynamic model:

푓푘|푘=1(푥|휂) = 푁(푥; 퐹푘−1휂, 푄푘−1)

2.4.2 Tracking Models

Constant Velocity Model Base Class class pymrt.tracking.models.CVModel(n) Constant Velocity Model Assume that the target of interest is moving at a constant velocity in a n-dimensional space. The location in the n- dimensional space is observable. Thus the dimensionality of the measurement space is 푛 and the dimensionality of the state space is 2푛. Parameters n (int) – Dimensionality of space the target is moving in. n int – Dimensionality of the space the target is moving in. It is the same as observation space in constant velocity model. x_dim int – Dimensionality of state space. _t int – Time intervals for each sample. _F numpy.ndarray – Dynamic model linear motion multiplier. _G numpy.ndarray – Dynamic model linea error multiplier. w_generator RandomGenerator – Random generator for state update error estimation.

2.4. API 17 pymrt Documentation, Release 1.0.0

r_generator RandomGenerator – Random generator for measurement error estimation. generate_new_state(x_prime, noise=True, noise_array=None) Generate new state based on previous state vector $x’$. Parameters • x_prime (numpy.ndarray) – State vector of size (self.x_dim, ). • noise (bool) – Whether the generated state includes disturbance. • noise_array (numpy.ndarray) – First order disturbance to the target state. It needs to be a column vector of size 푛. If it is set to None, the disturbance is generated by w_generator. Returns Next state vector of size (self.x_dim, ). Return type x (numpy.ndarray) generate_observation(x_prime, noise=True, noise_array=None) Generate Observation based on current state vector. Parameters • x_prime (numpy.ndarray) – State vector of size (self.x_dim, ). • noise (bool) – Whether the generated state includes disturbance. • noise_array (numpy.ndarray) – First order disturbance to the target state. It needs to be a column vector of size 푛. If it is set to None, the disturbance is generated by d_generator. Returns Measurement vector of size (self.z_dim, ). Return type z (numpy.ndarray)

Constant Velocity Model Base Class with Track ID class pymrt.tracking.models.CVIDModel(n) Constant Velocity Model with Track ID Assume that the target of interest is moving at a constant velocity in a n-dimensional space. The location in the n-dimensional space is observable. However, in order to associate observation with a speciﬁc target, a track_id ﬁeld needs to be added to the state vector. Thus the dimensionality of the measurement space is 푛 and the dimensionality of the state space is 2푛 + 1 Parameters n (int) – Dimensionality of space the target is moving in. n int – Dimensionality of the space the target is moving in. It is the same as observation space in constant velocity model. x_dim int – Dimensionality of state space. _t int – Time intervals for each sample.

18 Chapter 2. Multi-Target Tracking pymrt Documentation, Release 1.0.0

_F numpy.ndarray – Dynamic model linear motion multiplier. _G numpy.ndarray – Dynamic model linea error multiplier. generate_new_state(x_prime, noise=True, noise_array=None) Generate new state based on previous state vector $x’$. Parameters • x_prime (numpy.ndarray) – State vector of size (self.x_dim, ). • noise (bool) – Whether the generated state includes disturbance. • noise_array (numpy.ndarray) – First order disturbance to the target state. It needs to be a column vector of size 푛. If it is set to None, the disturbance is generated by w_generator. Returns Next state vector of size (self.x_dim, ). Return type x (numpy.ndarray) generate_observation(x_prime, noise=True, noise_array=None) Generate Observation based on current state vector. Parameters • x_prime (numpy.ndarray) – State vector of size (self.x_dim, ). • noise (bool) – Whether the generated state includes disturbance. • noise_array (numpy.ndarray) – First order disturbance to the target state. It needs to be a column vector of size 푛. If it is set to None, the disturbance is generated by r_generator. Returns Measurement vector of size (self.z_dim, ). Return type z (numpy.ndarray)

Base Model for GM-PHD class pymrt.tracking.models.GmPhdModel Deﬁnes the parameters needed for the model to be used with Gaussian-Mixture Probability Hypothesis Density Multi-target Tracking Filter. birth list – List of GMs deﬁning the birth probability across the state space. ps float – Persistence probability (between 0 to 1) pd float – Detection probability (between 0 to 1) lambda_c float – Poisson distribution parameter for clutter process (Assume it is a Poission Point Process). gm_T float – GM truncate threshold.

2.4. API 19 pymrt Documentation, Release 1.0.0

gm_U float – GM merge threshold. gm_Jmax int – Max number of GM to track in calculation.

Constant Velocity Model for GM-PHD class pymrt.tracking.models.GmPhdCvModel(n) Constant Velocity Model for Gaussian-Mixture Probability Hypothesis Density Filter

20 Chapter 2. Multi-Target Tracking CHAPTER 3

Reference

3.1 Multi-Target Tracking

21 pymrt Documentation, Release 1.0.0

22 Chapter 3. Reference CHAPTER 4

Indices and tables

• genindex • modindex • search

23 pymrt Documentation, Release 1.0.0

24 Chapter 4. Indices and tables Bibliography

[Mahler2003] Mahler, Ronald PS. Multitarget Bayes filtering via first-order multitarget moments. IEEE Transactions on Aerospace and Electronic systems 39, no. 4 (2003): 1152-1178. [Mahler2004] Mahler, R.P., 2004. “Statistics 101” for multisensor, multitarget data fusion. IEEE Aerospace and Elec- tronic Systems Magazine, 19 (1), pp.53-64. [Vo2006] Vo, B.N. and Ma, W.K., 2006. The Gaussian mixture probability hypothesis density filter. IEEE Transac- tions on signal processing, 54(11), pp.4091-4104.

25 pymrt Documentation, Release 1.0.0

26 Bibliography Index

Symbols GmPhdCvModel (class in pymrt.tracking.models), 20 _F (pymrt.tracking.models.CVIDModel attribute), 18 GmPhdModel (class in pymrt.tracking.models), 19 _F (pymrt.tracking.models.CVModel attribute), 17 _G (pymrt.tracking.models.CVIDModel attribute), 19 K _G (pymrt.tracking.models.CVModel attribute), 17 kalman_update() (pymrt.tracking.utils.GaussianComponent _t (pymrt.tracking.models.CVIDModel attribute), 18 method), 17 _t (pymrt.tracking.models.CVModel attribute), 17 L B lambda_c (pymrt.tracking.models.GmPhdModel at- birth (pymrt.tracking.models.GmPhdModel attribute), 19 tribute), 19 C M cov (pymrt.tracking.utils.GaussianComponent attribute), mean (pymrt.tracking.utils.GaussianComponent at- 17 tribute), 17 CVIDModel (class in pymrt.tracking.models), 18 CVModel (class in pymrt.tracking.models), 17 N n (pymrt.tracking.models.CVIDModel attribute), 18 D n (pymrt.tracking.models.CVModel attribute), 17 dmvnorm() (pymrt.tracking.utils.GaussianComponent n (pymrt.tracking.utils.GaussianComponent attribute), 16 method), 17 dmvnormcomp() (pymrt.tracking.utils.GaussianComponent P method), 17 pd (pymrt.tracking.models.GmPhdModel attribute), 19 ps (pymrt.tracking.models.GmPhdModel attribute), 19 G GaussianComponent (class in pymrt.tracking.utils), 16 R generate_new_state() (pymrt.tracking.models.CVIDModel r_generator (pymrt.tracking.models.CVModel attribute), method), 19 18 generate_new_state() (pymrt.tracking.models.CVModel method), 18 W generate_observation() (pymrt.tracking.models.CVIDModelw_generator (pymrt.tracking.models.CVModel attribute), method), 19 17 generate_observation() (pymrt.tracking.models.CVModel weight (pymrt.tracking.utils.GaussianComponent at- method), 18 tribute), 17 gm_Jmax (pymrt.tracking.models.GmPhdModel attribute), 20 X gm_T (pymrt.tracking.models.GmPhdModel attribute), x_dim (pymrt.tracking.models.CVIDModel attribute), 18 19 x_dim (pymrt.tracking.models.CVModel attribute), 17 gm_U (pymrt.tracking.models.GmPhdModel attribute), 19