FAMNet: Joint Learning of Feature, Affinity and Multi-dimensional Assignment for Online Multiple Object Tracking Peng Chu and Haibin Ling Temple University Philadelphia, PA USA fpchu,
[email protected] Abstract other, which results in a complex method design and exten- sive tuning parameters to adapt different target categories Data association-based multiple object tracking (MOT) and tracking scenarios. involves multiple separated modules processed or optimized Recently, deep neural network (DNN) has been investi- differently, which results in complex method design and re- gated intensively to learn the association cost function in a quires non-trivial tuning of parameters. In this paper, we unified architecture combining both feature extraction and present an end-to-end model, named FAMNet, where Fea- affinity metric [10, 26, 42]. Through training, the task and ture extraction, Affinity estimation and Multi-dimensional scenario prior can be automatically adapted by the candi- assignment are refined in a single network. All layers in date representation and estimation metric without manu- FAMNet are designed differentiable thus can be optimized ally tuning the hyper-parameters. However, the associa- jointly to learn the discriminative features and higher-order tion algorithm still stands outside the network, which re- affinity model for robust MOT, which is supervised by the quires dedicated affinity samples to be manually fabricated loss directly from the assignment ground truth. We also from ground truth association for the training process. It integrate single object tracking technique and a dedicated is not guaranteed that training and inference phases share target management scheme into the FAMNet-based tracking the same data distribution; consequently it may lead to the system to further recover false negatives and inhibit noisy degraded generalizability of the trained model.