Efficient Data Structures for Large Scale Tracking

R. O. Lane M. Briers T. M. Cooper S. R. Maskell QinetiQ QinetiQ Igence Radar Liverpool University Malvern, UK Malvern, UK Malvern, UK Liverpool, UK

Abstract—This paper describes a set of data structures that more than 100,000 vessels. To enable real-time tracking of this enables the tracking of large scale data sets. Although well- large scale data set a suite of efficient data structures has been known procedures exist for speeding up tracking performance, developed and implemented. such as gating, they are typically not sufficient for situations where it is required to simultaneously track tens or hundreds of The remainder of this paper is organized as follows. Section thousands of targets where even the gating calculations II gives an overview of related work aimed at speeding up themselves take a significant proportion of time. We describe tracking algorithms. Section III describes a set of efficient data dynamic spatiotemporal binary tree-based structures, a box structures. Section IV explains how these structures can be forest and cone forest, for storing measurements and tracks, and applied to the tracking problem. Finally, section V provides a string trie for text information. Efficient pruning of the concluding remarks. structures allows for a vast reduction in the number of gating calculations. Performance of a real-time tracking algorithm that II. RELATED WORK uses the data structures is demonstrated on a real-world maritime data set of more than 100,000 targets. A standard assumption in the tracking literature, known as the mutual exclusion constraint, is that each measurement can Keywords—box forest; cone forest; data association; string trie; only be assigned to one track and each track can have at most ubiquitous sensing one measurement per scan is assigned it. Joint probabilistic data association (JPDA) examines all possible associations that I. INTRODUCTION satisfy the above constraint and assigns a probability to each association. The target state can then be modeled by a Gaussian Over the last few decades a variety of techniques for mixture model with a certain number of components for tracking multiple targets with multiple sensors has been Kalman filter processing [1]. Standard JPDA is generally proposed. Three of the main tasks involved in a tracking considered to be computationally prohibitive for more than a system are prediction of target states at measurement times, few targets. However, the efficient hypothesis management data association – the assignment of measurements to tracks, (EHM) algorithm exploits redundancy in the calculations to and filtering – updating tracks that have measurements produce a mathematically equivalent answer with a substantial assigned to them. In a basic implementation of a tracking computational saving, allowing hundreds of targets to be algorithm, the state of every track must be predicted before tracked with this approach [2]. An alternative efficient data association can take place. During association, the implementation of JPDA is mentioned in [3]. A simpler computational load is usually reduced using a fine gating alternative to JPDA is the global nearest neighbor (GNN) procedure whereby measurements that are beyond a threshold algorithm, which picks the best single joint assignment of distance from a track are immediately rejected as not belonging measurements to tracks that satisfies the constraints. GNN is to that track. However, for asynchronous sensor reports the faster than JPDA but can have difficulties recovering if an number of prediction steps increases linearly with the number incorrect assignment is made at a particular point in time. A of targets, and the number of fine gating calculations is potential difficulty with these and other data association proportional to the square of the number of targets, assuming algorithms is the track swapping problem where measurements the number of measurements is proportional to the number of from nearby targets are assigned to the wrong track. A more targets. With the advent of ubiquitous sensing and large scale accurate multiple target tracker that considers the dependence data sets of thousands of targets, standard tracking algorithms between target states caused by unknown measurement-to- become computationally infeasible. target associations, and is implemented efficiently for large The work presented in this paper has been motivated by the numbers of targets, is described in [4]. need to track large numbers of ships. Maritime domain A number of algorithms have been proposed to further awareness is recognized internationally as a vital component in improve tracking efficiency. A coarse gating procedure is the fight against illegal activity such as terrorism, smuggling described in [5]. A track search area is defined as the set of and piracy. The amount of information available to this task positions that can be reached assuming a certain maximum has dramatically increased with the introduction of the speed, taking into account the effect of state and measurement automatic identification system (AIS). Ships with AIS uncertainty. Tracks with no measurements in the search area do transponders transmit their location, course, speed, and other not have to perform state predictions or be included in the data details. However, this information needs to be validated against association process. Tracks with at least one measurement in non-cooperative sensors such as radar. The data fusion task is the search area undergo the fine gating procedure as normal. computationally complex as the world’s shipping consists of Since coarse gating is much faster than fine gating, the computation time is reduced. An algorithm in [6] stores measurements from each scan in a kd-tree structure. The structure can then be queried to quickly retrieve measurements near the predicted position of a track before fine gating. Reference [7] processes all data in one batch to group Fig. 1. First three trees of a binary log forest. measurements into potential tracks without forcing the mutual exclusion constraint. One kd-tree is constructed per time step • Delete: When a leaf node is deleted from the tree, its and the trees are used to prune measurement sets that do not sibling is promoted to its parent’s position to maintain conform to the motion model. Note that since batch processing the full binary structure. is used, prediction calculations do not need to be carried out. An advance in the algorithm is reported in [8]. Cluster-based Note that there is no method for adding a single leaf to the tree data association approaches that are linear in the number of – all data must be loaded in one batch. The rationale for this is detections and tracked objects are described in [9]. Several explained in the following section. Efficient searching for a distributed tracking algorithms have been proposed. For data point can be carried out. The combination of the binary example, [10] uses consensus filters for fusion of the sensor structure, summary data at internal nodes, and pruning function data and covariance information, [11] uses the drain-and- enables searching for a data point in O(log n) time as opposed balance method for constructing efficient tracking hierarchies, to exhaustive searching, which is O(n). and [12] describes a hierarchy where lower tracking nodes in a tree are only activated when instructed by higher nodes. The B. Binary Log Forest aim of these algorithms is to distribute the computing among For real-time performance, the data structure used to store several processing nodes to reduce the per-node computational information must be able to be queried at a faster rate on requirement. average than the arrival of new queries. Although a standard Although the above approaches are an advance on standard binary tree is able to execute queries quickly for data loaded in tracking practice, each of them has some shortcoming that has a single batch, its performance degrades with the insertion and prevented general application to very large scale real-time deletion of points over time. When the tree is loaded in a batch, tracking problems. For example, although some efficient data all leaves exist in either the bottom or second-to-bottom layer structures have been applied to problems relating to tracking of the tree. If a single point is added or deleted, this property no ballistic objects in the atmosphere [6] and asteroids [7], the longer necessarily holds. If many points are added in one part trajectories of these objects are near-deterministic. The novelty of the tree, then the depth of the leaves in that region is much of this paper is to describe a combination of data structures that greater than the depth of leaves in other parts of the tree. This has the ability to be efficient while considering large numbers unbalanced tree structure can result in a much longer search of non-deterministic trajectories. time than the optimum for the total number of data points in the tree. To re-balance the tree it would be necessary to re-organize large parts of the tree. The quality of most indexing structures, III. EFFICIENT DATA STRUCTURES such as kd-trees or K-D-B-trees, deteriorates when a large Here we describe in detail a number of data structures that number of updates are performed on them [13]. will be used later in the paper. Some of these may be familiar To avoid these problems we use the logarithmic method to to readers with a computer science background but are not construct a binary log forest [13]. The forest consists of a finite necessarily commonly used in the tracking community. sequence of unconnected binary trees. The ith tree contains at Properties of the structures are presented for tracking purposes; most 2i-1 leaves. When a data point is deleted from the forest, it other applications may require modified versions. is simply deleted from whichever tree it is contained in. When a data point is inserted to the forest, the first k trees are A. Binary Tree destroyed and the kth tree is rebuilt from the new element A full binary tree is a directed tree data structure where together with all the elements that were stored in the destroyed each node of the tree is either a leaf or has two children, known trees. The parameter k is chosen to be the smallest possible, as the left and right nodes. The root node has no parents and subject to the reconstructed tree obeying the size constraints. leaf nodes have no children. For the particular type of binary The forest is queried by searching each tree in sequence and tree required for our tracking application, every node within the returning the list of data points that match the query in any tree. tree contains a summary of its descendant nodes. In addition, Note that querying multiple trees can be executed in parallel. leaf nodes contain track data of interest, specifically a location vector and track ID. The forest, as illustrated in Fig. 1, is a sequence of binary trees of exponentially increasing size. The log forest offers the Various functions on the tree and nodes are defined as same construction, searching, and removal functionality as the follows. binary tree but additionally offers the ability to add new • individual elements. The advantage of the log forest structure is Construct Tree: A set of objects can be inserted to the that repeated adding and removing of elements does not tree in one batch. degrade the structure in the same way that it would for a binary • Query: A pruning function returns all leaf nodes that tree and searches are always efficient. For our tracking meet a certain criterion. application, the computational effort required to occasionally reconstruct some trees is more than offset by the enhanced search speeds. It is noted that other self-balancing data structures exist, such as the Bkd-tree [13] or red-black tree 11 12 [14]. However, the red-black data structure has no knowledge 5 of the geometry of the underlying points. When it re-balances, arbitrary points would be associated together at low levels in the tree, and the internal node summaries would have significant overlap, leading to inefficient pruning.

C. Box Forest The box forest is a specialized form of binary log forest 2 6 where leaf nodes contain a vector that defines the target state 1 and a ‘box’ is stored at each internal node of each tree. The individual trees are known as box trees, which are a type of bounding volume hierarchy [15]. The box is a hyper-cuboid Fig. 2. Spatial representation of a box tree. A search query is indicated by the with the property that all of its edges are aligned with the dashed line. standard axes. In other words, it is the Cartesian product of a set of intervals. The size of the box is such that it contains all the vectors of leaves that are descendants of the node. The trees 1 2 are constructed by finding the smallest hyper-cuboid that contains the set of descendent vectors. When splitting internal 5 6 nodes with more than one vector, an axis-aligned hyper-plane is chosen such that approximately half of the vectors are each 11 12 side of the plane. The hyper-plane is chosen to be orthogonal to the axis along which these vectors have maximum spread. Once this hyper-plane has been found for a particular node, sub-trees are constructed recursively for each of the two Fig. 3. Tree representation of the box tree. Internal nodes selected by the children branches based on the two resultant subsets of the set search query are highlighted with dashed circles. of vectors. The box associated with a leaf node of a tree is the box all of whose sides have length zero that is at the same Let the set of points that we wish to divide be P, and let Q position as the point stored at the leaf. To search the tree, a be the subset of P that is chosen for the first of the two simple depth-first search is used with pruning determined by children. In the first case mentioned above, we choose Q to the hyper-cuboid. A single tree from the box forest is illustrated minimize V(Q) + V(P\Q), where V gives the volume. In the from two different perspectives in Fig. 2 and Fig. 3. second case, we choose Q to minimize ln(|Q|) + ln(|P\Q|). A compromise between these two strategies is to choose Q to We discuss in more detail the algorithm for determining the minimize f(Q) + f(P\Q) where f(R) = ln(|R|)V(R). Unless the exact location of hyper-planes to partition the data. The optimal data points are fractally distributed, a tree built using this strategy to use depends on the queries that will be performed compromise will tend to be more balanced at deep nodes in the on the final tree structure, and so to determine a good strategy, tree than near the root and should have good querying the efficiency of some queries must be analyzed. 1 performance . It is most easy to determine the efficiency of a query of the Fig. 2 shows the spatial position of 2D vectors, stored in the type ‘Does this point occur in the data structure?’. This query leaves of the tree, and box summaries that are stored at internal can be answered by a depth-first or breadth-first search of the nodes. An example search query that returns three data points tree, pruning branches that cannot contain the query point. The is shown as a dashed circle. Fig. 3 shows the tree representation number of branches examined in such a search is of the order of the data structure and more clearly illustrates the search of the number of nodes whose associated summary includes the procedure of the query. At the first two levels of the tree, box 2 query point. In the first case, where query points are distributed and 5 are the only summaries that match the query. At the third evenly in space, the expected time taken for the search is level both boxes 11 and 12 match the query. The contents of proportional to the total volume of all the boxes in the tree. A these two boxes are examined and the three data points in the useful method for approximating this structure is to choose leaves that match the query are returned. The box tree concept each partition to minimize the sum of the volumes of the two is similar to that of R-trees [16]. child shapes. However, sometimes a tree that is optimal in the sense of minimizing the total volume is very unbalanced. In the second case, where query points are chosen at random from 1 amongst the data points, then the expected time taken for the An efficient approximately optimal way of splitting the data in two search is proportional to the average depth of the leaves of the is to select a point C at random, find the point A the furthest from C tree. A tree that is optimal in the sense of minimizing this depth and the point B furthest from A. Each point is then allocated to the is as close to being balanced as possible. This structure can be closest of either A or B. This is an O(n) process but results in either obtained by choosing each partition so that each part contains overlapping or non-axis aligned boxes. Such structures are slower to search in 2D than non-overlapping axis-aligned structures, but can exactly half of the descendant data points. have utility in higher dimensions. D. Cone Forest t A cone forest is defined as a specialized form of the box forest relevant to tracking and is designed to enable efficient lookup in a database of tracks. Tracks are stored in the forest structure, and provision is made for the fact that each track may have been updated at a different time. The underlying space for the box tree is the product of the space part of the state space with time. A universal maximum rate of change of space with x time (i.e. speed smax) is assumed known and is used to scale the time dimension. When a new measurement is made, this yields a cone shaped volume of space-time in which to search. Each track is stored in a box forest according to the time of its last y update and location at that time. Given a measurement location and time, a search returns all tracks from the database whose last update lies within the cone projecting back in time, with apex the measurement, and slope determined by the maximum Fig. 4. Internal summary node from a box forest containing track updates and speed. The search volumes involved are illustrated in Fig. 4. cone search volume projecting backward in time from a measurement. Dashed lines indicate the projection of shapes onto the location plane at t=0. To enable efficient searching of the box forest a pruning algorithm is required. The simplest useful algorithm is a cone pruner. This accepts boxes that intersect the solid cone defined by those points able to reach apex of the cone while travelling at a normalized speed of less than 1. Here, one dimension represents time and the others are spatial. A box intersects such a cone if the face of the box obtained by minimizing the time dimension intersects the corresponding circle obtained from the cone at that time. Note that the box forest containing track data does not need to be updated each time a search is carried out Fig. 5. Cone-like search areas with a circular or elliptical base. The top face using different measurement time stamps. Thus the of the shapes is the measurement uncertainty. computational load of track prediction and update is constant with respect to the number of tracks, and is O(M) for M This just represents a stretch of the original ellipse along its measurements. This is a huge advantage compared to standard principal axes. The expanded ellipse is an approximation to the tracking practice where prediction and update computation true uncertainty shape, which ranges between a circle, in the time increases with the number of tracks. limit of isotropic uncertainty, and a race track shape for long thin measurement uncertainty ellipses. The elliptical search In practice, measurements have some uncertainty volume is illustrated on the right of Fig. 5. associated with them. If the uncertainty in the space dimension is isotropic, this can be represented by an error circle. The E. String Trie search volume is then a cone whose apex is at a time later than that of the measurement, such that a slice of the cone at the Some data sources include text-based information measurement time is the error circle. This search volume is associated with targets, such as a vessel name. It is required to illustrated on the left Fig. 5. Anisotropic measurement quickly determine whether or not a particular string is currently uncertainties in location are represented by an error ellipse, and associated with a track, including approximate matches. This an elliptical cone pruner is required. This algorithm accepts can be achieved through use of a string trie (pronounced “tree” boxes that intersect the solid cone-like shape defined and also known as a prefix tree), which is a tree-based structure approximately by those points able to reach the inside of the that is able to efficiently store and search for sequences of ellipse by the measurement time while travelling at less than symbols/letters. The root of the tree is the start of the word. the maximum speed. A box intersects such a shape if the Children of the root denote the first letter of the word and spatial face of the box at the earliest time intersects an descending the tree further appends a letter at each level. Each expanded ellipse calculated from the measurement ellipse and node has a Boolean flag to state whether or not that letter the maximum speed. The expanded ellipse is calculated as indicates the end of a word and this flag is set to true for all leaf follows. Let the eigenvalue decomposition of the measurement nodes. In addition to the letters, each node can also store a covariance matrix P be pointer to other information. For example, if each track has a 0 name associated with it, the pointer could refer to a track ID. P= VΛV-1, (1) 0 Fig. 6 illustrates the string trie data structure for a particular where V is the square matrix whose ith column is an set of words, referred to as a dictionary. In alphabetical order, eigenvector of P0 and Λ is the diagonal matrix whose diagonal the first few words stored are CAGE, CAR, CART, MAR, and elements are the corresponding eigenvalues. The covariance MARE. To determine whether a word is stored in the string matrix of the expanded ellipse after time t is then trie, the tree is descended one level at a time, selecting the branch that has the appropriate letter from the word. Eventually P= V (Λ1/2+s tI)2 V-1. (2) t max either the whole word is spelt out or there are no available

C M

A A O

G R R S A O

E T E Y T T N R

Fig. 6. A string trie where descending the tree spells out words stored in the Fig. 7. Comparison of per-measurement tracker processing times for a data structure. Shaded circles represent the last letter of the word. standard tracker and one using the cone forest. branches that contain the required letter. This tree-based search measurements per second or number of tracks. The tree-based is much faster than having to check every word in the data structure is more efficient than the coarse gating procedure dictionary until a match is found. In addition to exact matches described in [5], which calculates a coarse gate for every track. it is possible to use the string trie to find all matches where the two words are within a certain edit distance of each other. The B. Simulated data scenario edit distance is the number of edit operations required to The computational advantage of the cone forest structure is change from one word to the other. Typical operations demonstrated here with a simulated data scenario. Each target available are insertion, deletion, substitution of single letters, or moves in 2D with a state vector x = [x, x′, y, y′ ]T that varies swapping two adjacent letters. If insertion and deletion are the according to a nearly constant velocity model x Δ = Fx + w. only allowed operations then the edit distance between MARE t+ T t The transition matrix F and covariance Q of the zero-mean and MAR, MARY, and CAR is 1, 2, and 3, respectively. The Gaussian process noise w are as follows [18]. operation sequences for each example are: {delete E}, {delete E, insert Y}, and {delete M, delete E, insert C}. Efficient approximate string matching algorithms are discussed in detail in [17].

IV. APPLICATION TO TRACKING The target locations are initially distributed according to A. Introduction the uniform distribution with a density of 10-8 targets/m2 and The above data structures can be used in combination with target velocities in x and y are independently distributed a standard tracking architecture to enable large data sets to be uniformly from –10 m/s to 10 m/s. The scalar process noise σ2 2 efficiently processed. The forest structure is used to store the parameter is 1 (m/s) /s and the maximum target velocity is mean position of each track the last time it was updated by a 100 m/s. Measurements of x and y are received every ΔT = 10 s measurement. The time part of the vector used when a track is for 1000 s with zero mean and standard deviation 10 m. The stored has an offset subtracted from it to simulate the estimated number of targets was varied from 100 to 30,000 and a Kalman uncertainty in the tracker’s knowledge of the target position. filter applied using either no efficient data structures or the Whenever a new measurement or scan of measurements is cone forest. A comparison of the per-measurement processing received, a query on the cone forest using either the circular or times for each tracker is shown in Fig. 7. For less than 300 elliptical pruning algorithm returns all tracks that could targets the overhead of creating and searching the cone forest potentially have given rise to the measurement(s). Each of means that using this data structure is slightly slower than a these tracks is then predicted forward from the last update to standard tracker. However, above 300 targets, the computation the current time. A fine gating procedure is then used reject any load of the standard tracker increases rapidly and it is infeasible tracks that have a sufficiently low likelihood of generating the to use it for large numbers targets. In contrast, the per- measurements. A standard data association algorithm, such as measurement processing load of the cone forest tracker GNN or JPDA, is applied to the remaining tracks and increases only slightly with the number of targets. measurements, and the tracks are updated as appropriate. Note that if a track is not returned by the pruning algorithm no C. String Likelihood Calclations processing is carried out for that track – there is no need to If at least one field of the target state is a string then it is update it to the latest measurement time. With this tracking necessary to define probability distributions over strings and architecture, computational cost is dictated by the number perform some string-based likelihood calculations. The of track updates per second rather than the number of probability of a single error in a string can be defined as PE = exp[–k], (3) P(xt | z1:t) ∝ exp[–kz d(zt, xt) – kx d(xt, wB)] . (11) where k is known as the certainty parameter. A string If zt = wB, then this posterior distribution is a standard string probability mass function (PMF) is then defined according to distribution with best string wB and updated certainty parameter the geometric distribution kx(t) = kx(t–1) + kz(t). (12) P({x} | wB, k) = (1–PE) exp[–k d(x, wB)], (4) If zt ≠ wB, then simplification of (11) is not possible. To where wB is the “best” word/string and d is the edit distance maintain a consistent form of probability distribution we adopt between x and wB, with each operation having unit cost. The the following practical procedure. For each string field being notation {x} denotes the set of strings that have the same edit tracked and each track, two string PMFs are stored: one for the distance from wB. It is assumed that if there are several ways of most likely string and one for the most recently observed transforming wB to x only operations of the minimal edit string. If during the data association stage the measurement zt is distance need be considered. The probability of an individual associated with a track and is identical to the most likely string string depends on the sequence of edit operations required to then the PMF of the state estimate is modified according to obtain it from wB and is of the form (12). If zt is identical to the most recent string then the certainty parameter of that string is updated using the same method. If z P(x | w , k) = P({x}|w , k) n ∏ (p /n ). (5) t B B s i=1:d(x, wB) i i does not match either string then the most recent string For the ith operation in the sequence, pi is the probability of a distribution is replaced with parameters zt and kz. If at any time particular type of operation and ni is the number of words that the certainty of the most recent string is higher than that of the can be produced from wB by that type of operation. The number most likely string, then the most likely string is replaced by the of different operation sequences that result in the same final most recent one. Storing this extra “most recent” string string is denoted by ns. For the maritime tracking scenario prevents early incorrect association decisions (often made the discussed in the following section it is assumed that the only first time a track is associated with a data source containing a operations allowed are insertion and deletion and that these are string measurement) from negatively affecting future decisions. equally likely (p = 0.5). Other operations such as substitution i The likelihood of a measurement-to-track association is and adjacent letter swapping could easily be included in the used during the data association stage. Regardless of the same framework. If there are A letters in the alphabet and the association algorithm used, if the string likelihood is too low length of w is W, then for a string obtained by a single B then an association will not be made. If the track string insertion n = A(W+1) and for a string obtained by a single 1 information is stored in a string trie, then the trie can be used to deletion n = W. Similar expressions can be obtained for 1 prune the data so that only tracks within a certain edit distance operations that are part of a larger sequence. It is noted that an of the measurements are updated. This is the string equivalent alternative formulation of (5) could incorporate the parameters of spatial gating, and results in a vast reduction in the number relating to the sequence of operations into a modified definition of track updates that need to be made. of edit distance with non-unit costs. In tracking frameworks a procedure is required to update D. Maritime Scenario the distribution of the state xt-1 from a previous time step to An application of large scale tracking is the monitoring of include information provided by a new measurement zt at the the world’s commercial shipping. Almost all AIS-based ship current time step. This can be derived for strings as follows. By tracking services currently in operation associate measurements Bayes’ rule using the supposedly unique Mobile Maritime Service Identity

P(xt | z1:t) = P(zt | xt)P(xt | z1:t-1)/P(zt | z1:t-1). (6) (MMSI) and/or International Maritime Organization (IMO) fields of the AIS data. Unfortunately, these fields are not The predictive prior can be written as necessarily unique on a global scale. One issue arising from such association methods is the rapid divergence of tracks – a P(xt | z1:t-1) = Σx(t-1)P(xt | xt-1,z1:t-1)P(xt-1 | z1:t-1). (7) track “flips” between one vessel’s actual location and the Since it is assumed that string fields don’t change, location of another vessel, which would usually be spatially P(x | z ) = P(x | z ). (8) well separated from the first. Such flipped tracks make the t 1:t-1 t-1 1:t-1 situation awareness picture difficult to interpret. Another Substituting this into (6), the recursive form of Bayes rule is difficulty with AIS data is that it is a self-reporting system. found: Errors in the transponders aboard ships or deliberate attempts to spoof the location (i.e. for a ship to report an incorrect P(x | z ) = P(z | x )P(x | z )/P(z | z ). (9) t 1:t t t t-1 1:t-1 t 1:t-1 location) mean that the information cannot necessarily be relied Combining (4) and (5), the likelihood of a measurement zt upon in isolation. By using non-cooperative sensors such as given the state estimate xt and certainty parameter for radar it is possible to validate the AIS information and increase measurements kz is the confidence in the tracks produced.

P(zt | xt) = (1–PE(kz)) exp[–kz d(zt, xt)] ns ∏i=1:d(pi/ni). (10) QinetiQ has implemented a real-time tracking system that takes data from a variety of sensors, such as AIS and radar, and Let the parameters of the predictive prior in (8) be wB and kx. fuses the signals together to output validated tracks. Since Then the approximate analytic form of (9) is

Fig. 8. Example output of the tracker: a sample of 1000 tracks in Western Europe from a database of more than 100,000 tracks spread over the globe. tracking is required over the whole of the Earth it is not as defined in (3). The cone forest and string tries are used for possible to use a local Cartesian coordinate system. The target efficient pruning of irrelevant data so that only a small number motion part of the state space therefore consists of the latitude of tracks need to be updated. Thus the string fields are used to and longitude of ship locations, and the rate of change of those aid data association but a standard fine gating procedure is used variables. These coordinates are assumed to exist on a uniform to separate spatially removed ships with the same IDs. 2D grid. This approximation is good at the equator but slowly degrades towards the poles because lines of longitude are The tracking system has been applied to a three-week data set containing coastal AIS and radar data. The AIS data source closer to each other near the poles than at the equator. In practice, the approximation has been found to be sufficient for was provided with measurements for each ship occurring at good tracking performance in the data examined. Independent approximately 10 minute intervals while the ship was nearly constant velocity motion models have separately been detectable by the coastal AIS receiver. Radar detections, where used for latitude and longitude, with scalar process noise available, were supplied at a higher rate. In total, more than 100,000 tracks were produced by the system. Ground truth was parameter 5×10-12 (degree/s)2/s. The measurement noise of AIS not available for this data set, but manual examination of a reports was assumed to have a variance of 10-8 degree2, which large number of tracks suggests that the vast majority were is equivalent to a standard deviation of about 11 m at the valid tracks for separate ships, and individual ships generally equator. Tracking directly in latitude and longitude creates had only one track associated with them. A 1000-track subset difficulties for the integration of radar measurements made in in Western Europe of the whole data set is shown in Fig. 8. polar coordinates relative to the sensor location. The solution to Despite the fact that only coastal sensors were available, tracks this has been to use an unscented Kalman filter as described in have correctly been formed across large bodies of water, such [19]. The association algorithm used was GNN. as the North Sea. Some cross-land tracks that are visible have When measurements are received, the efficient data been created by ships that were detected at inland ports but did structures described in the previous section are used to decide not produce any measurements for a long period of time before which tracks should be updated. Track updates are stored in the being detected by another sensor. The visualization simply cone forest. The MMSI, IMO ID, and ship name are stored as draws a straight line between two such track points. There are strings in the string trie, each string field being an independent no “flipped” tracks that connect two widely separated ships component of the target’s state. In some data sources, these with the same ID. strings are typed manually once and then transmitted as part of an automated system. Due to typographical errors it is possible E. Other applications that the same ship could have similar but different In urban scenarios it may be necessary to track crime or instantiations of each of these fields in different data sources. terror suspects via CCTV, for example. However, even when it In some sources, the information is considered “correct”. The is possible to reliably carry out the signal processing to detect fidelity of the data source is captured by its certainty parameter and recognize the appropriate people and their actions [20], it would be difficult to do so among a large population, without REFERENCES some efficient form of track processing. Reference [21] [1] D. J. Salmond, Mixture reduction algorithms for target tracking in proposes a multipath exploitation radar system for use in urban clutter, Proc. SPIE 1305:434-445, October 1990. environments containing vehicles that takes advantage of [2] P. Horridge, S. Maskell, Real-Time Tracking of Hundreds of Targets reflections from buildings to see around the corner and detect with Efficient Exact JPDAF Implementation, Int. Conf. on Information moving targets. Tracking all the possible “ghost” reflections Fusion, Florence , July 2006. until the true location becomes unambiguous is a large scale [3] M. R. Chummun, T. Kirubarajan, K. R. Pattipati, Y. Bar-Shalom, Fast tracking problem that may require use of the techniques data association using multidimensional assignment with clustering, IEEE Trans. Aerospace and Electronic Systems 37(3):898-913, July described in this paper. 2001. Persistent wide area surveillance systems, such as the [4] P. Horridge, S. Maskell, A scalable method of tracking targets with Autonomous Real-time Ground Ubiquitous Surveillance dependent distributions, Int. Conf. on Information Fusion, July 2009. Imaging System (ARGUS-IS), are beginning to be developed [5] H. Wang, T. Kirubarajan, Y Bar-Shalom, Precision Large Scale Air Traffic Surveillance Using an IMM Estimator with Assignment, IEEE from research concepts to deployed systems. These systems Trans. Aerospace and Electronic Systems 35(1):255-266, January 1999. produce imagery with in excess of 1 gigapixels, and with [6] J. K. Uhlmann, Algorithms for multiple-target tracking, American appropriate processing are able to detect large numbers of Scientist, 80(2):128–141, 1992. objects moving in the scene. However, track processing of the [7] J. Kubica, A. Connolly, A. Moore, R. Jedicke, A Multiple Tree entire surveyed area for airborne systems has not been possible Algorithm for the Efficient Association of Asteroid Observations, to-date due to limited on-board processing power and it is not KDD’05, Chicago, Illinois, USA, August 2005. possible to transmit the whole image for ground-based [8] J. Kubica, L. Denneau, T. Grav, J. Heasley, R. Jedicke, J. Masiero, et. al, processing due to bandwidth limitations [22]. The techniques Efficient intra- and inter-night linking of asteroid detections using kd- described in this paper could be used to reduce the trees, Icarus189(1): 151-168, July 2007. computational load of the tracker, enabling a comprehensive [9] M. Betke, D.E. Hirsh, A. Bagchi, N.I. Hristov, N.C. Makris, T.H. Kunz, Tracking Large Variable Numbers of Objects in Clutter, IEEE Conf. on surveillance solution where image chips of all moving objects Computer Vision and Pattern Recognition pp.1-8, June 2007. are sent to a ground-based station for further analysis. [10] R. Olfati-Saber, N. F. Sandell, Distributed Tracking in Sensor Networks There are in excess of 300,000 known asteroids, and this with Limited Sensing Range, American Control Conference, Seattle, Washington, USA, June 2008. number is likely to increase significantly over the next few [11] H. T. Kung, D. Vlah, Efficient Location Tracking Using Sensor years with the advent of gigapixel resolution camera systems Networks, IEEE Wireless Communications and Networking, WCNC designed to detect objects in space that were not previously 2003, New Orleans, LA, USA, March 2003. visible [8]. Tracking this many objects will require the use of [12] P. Scheffel, R. Fish, R. Knobler, T. Plummer, Distributed multi-sensor efficient data structures. fusion, Proc. SPIE 6974, 2008. [13] O. Procopiuc, P. K. Agarwal, L. Arge, J. S. Vitter, Bkd-Tree: A The ability to track large numbers of targets enables other Dynamic Scalable kd-Tree, SSTD 2003, LNCS 2750:46-65, 2003. possibilities for data exploitation. For example, in [23] and [24] [14] R. Hinze, Constructing red-black trees, Workshop on Algorithmic a higher-level pattern-of-life analysis of tracks is performed. Aspects of Advanced Programming Languages, WAAAPL’99, Paris, Statistical models for behavior are built based on historical France, September 1999. track data and any anomalous departure from the model is [15] P.K. Agarwal, M. de Berg, J. Gudmundsson, M. Hammar, H.J. marked for further manual inspection by an analyst. This Haverkort, Box-Trees and R-trees with Near-Optimal Query Time, allows huge amounts of data to automatically be processed, Discrete & Computational Geometry, 28(3):291-312, August 2002. only requiring human analysis for the most interesting and [16] A. Guttman, R-trees: a dynamic index structure for spatial searching, unusual data. Proc. ACM SIGMOD Int. Conf. on Management of Data, 1984. [17] H. Shang, T. H. Merrettal, Tries for approximate string matching, IEEE Trans. Knowledge and Data Engineering, 8(4):540-547. V. CONCLUSIONS [18] S. Maskell, Sequentially Structured Bayesian Solutions, PhD Thesis, Several data structures for the efficient tracking of large- Cambridge University Engineering Department, 2004. scale data sets have been proposed. The complication in [19] P. Horridge, Fusing Geodetic and Radar measurements for tracking designing such data structures arises from the fact that ground‐based targets, Mathematics in Defence 2013, Malvern, UK, 24th geometric estimates such as position are subject to uncertainty October 2013. and are evolving temporally. This means that the data structure [20] N. Robertson, I. Reid, A general method for human activity recognition in video, Computer Vision and Image Understanding 104:232–248, needs to be able to handle searches with explicit consideration 2006. of the uncertainty, which is not undertaken in standard database [21] R.O. Lane, Target detection in urban scenarios using multipath netted representations. The structure also needs to be dynamically radar, Mathematics in Defence 2009, Farnborough, Hampshire, UK, reconfigurable, since it has to represent target locations that 19th November 2009. change over time. The data structures reported in this paper are [22] B. Leininger, J. Edwards, J. Antoniades, D. Chester, D. Haas, E. Liu, et able to handle both of these issues and have been demonstrated al., Autonomous Real-time Ground Ubiquitous Surveillance - Imaging to work in real time with a global-scale maritime data set. System (ARGUS-IS), Proc. SPIE 6981 , April 2008. [23] R.O. Lane, D.A. Nevell, S.D. Hayward, T.W. Beaney, Maritime anomaly detection and threat assessment, 13th International Conference ACKNOWLEDGMENT on Information Fusion, Edinburgh, UK, 26-29 July 2010. The majority of this work was carried out at QinetiQ from [24] R.O. Lane, K.D. Copsey, Track Anomaly Detection with Rhythm of Life and Bulk Activity Modeling, 15th International Conference on 2008-2009 under the Maritime Wide Area Surveillance Information Fusion, Singapore, 9-12 July 2012. programme of the UK Ministry of Defence.