sensors Article Head Pose Estimation through Keypoints Matching between Reconstructed 3D Face Model and 2D Image Leyuan Liu 1,2 , Zeran Ke 1 , Jiao Huo 1 and Jingying Chen 1,2,* 1 National Engineering Research Center for E-Learning, Central China Normal University, Wuhan 430079, China;
[email protected] (L.L.);
[email protected] (Z.K.);
[email protected] (J.H.) 2 National Engineering Laboratory for Educational Big Data, Central China Normal University, Wuhan 430079, China * Correspondence:
[email protected]; Tel.: +86-135-1721-9631 Abstract: Mainstream methods treat head pose estimation as a supervised classification/regression problem, whose performance heavily depends on the accuracy of ground-truth labels of training data. However, it is rather difficult to obtain accurate head pose labels in practice, due to the lack of effective equipment and reasonable approaches for head pose labeling. In this paper, we propose a method which does not need to be trained with head pose labels, but matches the keypoints between a reconstructed 3D face model and the 2D input image, for head pose estimation. The proposed head pose estimation method consists of two components: the 3D face reconstruction and the 3D–2D matching keypoints. At the 3D face reconstruction phase, a personalized 3D face model is recon- structed from the input head image using convolutional neural networks, which are jointly optimized by an asymmetric Euclidean loss and a keypoint loss. At the 3D–2D keypoints matching phase, an iterative optimization algorithm is proposed to match the keypoints between the reconstructed 3D face model and the 2D input image efficiently under the constraint of perspective transformation.