Abstract：In order to improve the performance of facial information recognition under complex conditions, a novel deep convolutional neural network (DCNN) method is proposed for the task coordination of facial landmark localization and head pose estimation.First, the facial information is detected from video images.Secondly, a DCNN model is designed for synergistic optimization of both facial landmark localization and head pose estimation tasks, and then used to simultaneously estimate the coordinates of facial landmarks and the angles of head pose,which are then fused to generate the human-computer interaction information.Finally, the proposed method is tested on both public datasets and actual scene data, and then compared with other state of the art methods.Experimental results show that the proposed approach performs better in facial landmark localization and pose estimation,and also achieves good accuracy and robustness in the HCI applications under complex conditions of illumination variations, expression changes and partial collusions, with the average speed at 16 frames per second, which demonstrates its efficiency and practicality.
［1］Xu X,Zhang Y,Zhang S,et al. 3D hand gesture tracking and recognition for controlling an intelligent wheelchair［J］. International Journal of Computer Applications in Technology,2014,49(2):104-112.
［2］Veneri G,Federighi P,Rosini F,et al. Influences of data filtering on human-computer interaction by gaze-contingent display and eye-tracking applications［J］. Computers in Human Behavior,2010,26(6):1 555-1 563.
［3］Murphy-Chutorian E,Trivedi M M.Head pose estimation in computer vision:a survey［J］.IEEE Transactions on Pattern Analysis & Machine Intelligence,2009,31(4):607-626.
［4］Asthana A,Zafeiriou S,Cheng S,et al. Robust discriminative response map fitting with constrained local models［C］// Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition.Portland,OR,USA,2013,3444-3451.
［5］Tzimiropoulos G,Pantic M.Gauss-newton deformable part models for face alignment in-the-wild［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus,OH,USA,2014,1851-1858.
［6］Baltruaitis T,Robinson P,Morency L P. Continuous conditional neural fields for structured regression［C］// Proceedings of Computer Vision–ECCV2014. Zurich,Switzerland,2014,593-608.
［7］Ren S,Cao X,Wei Y,et al. Face alignment at 3000 fps via regressing local binary features［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Columbus,OH,USA, 2014,1685-1692.
［8］Sun Y,Wang X,Tang X. Deep convolutional network cascade for facial point detection［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Portland,OR,USA,2013,3476-3483.
［9］程淑红,刘洁.基于多尺度韦伯脸和梯度脸的复杂光照下人脸识别研究［J］.计量学报,2017,38(1):60-64.
Cheng S,Liu J. Face Recognition under Complex ILLumination Based on Multi-scale Weber-face and Gradient-face［J］.Acta Metrologica Sinica,2017,38(1):60-64.
［10］唐云祁,孙哲南,谭铁牛.头部姿势估计研究综述［J］.模式识别与人工智能,2014,27(3):213-225.
Tang Y,Sun Z,Tan T. A Survey on Head Pose Estimation［J］.Pattern Recognition and Artificial Intelligence,2014,27(3):213-225.
［11］Ma B,Chai X,Wang T. A novel feature descriptor based on biologically inspired feature for head pose estimation［J］.Neurocomputing,2013,115:1-10.
［12］Foytik J,Asari V K. A Two-Layer framework for piecewise linear manifold-based head pose estimation［J］.International Journal of Computer Vision,2013,101(2):270-287.
［13］Lu J,Tan Y P. Ordinary preserving manifold analysis for human age and head pose estimation［J］.IEEE Transactions on Human-Machine Systems,2013,43(2):249-258.
［14］Lécun Y,Bottou L,Bengio Y,et al. Gradient-based learning applied to document recognition［J］.Proceedings of the IEEE,1998,86(11):2278-2324.
［15］LeCun Y,Bengio Y,Hinton G. Deep learning［J］.Nature,2015,521(7553):436-444.
［16］Krizhevsky A,Sutskever I,Hinton G E. ImageNet classification with deep convolutional neural networks［C］// Proceedings of Advances in Neural Information Processing Systems,.Lake Tahoe,Nevada,USA,2012:1097-1105.
［17］Sermanet P,Eigen D,Zhang X,et al.Overfeat:Integrated recognition,localization and detection using convolutional networks［J］.Eprint Arxiv,2013.
［18］Sun Y,Wang X,Tang X.Deep Convolutional network cascade for facial point detection［C］// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,Portland,OR,USA,2013:3476-3483.
［19］赵一中,刘文波.基于深度信念网络的非限制性人脸识别算法研究［J］.计量学报,2017,38(1):65-68.
Zhao Y,Liu W.Research on Unconstrained Face Recognition Based on DBNs Network［J］.Acta Metrologica Sinica,2017,38(1):65-68.
［20］Margarita Osadchy Y L C,Miller M L. Synergistic face detection and pose estimation with energy-based models［J］.Journal of Machine Learning Research,2007,8(1):1017-1024.
［21］Zhang Z,Luo P,Chen C L,et al. Facial Landmark Detection by Deep Multi-task Learning［C］// Proceedings of European Conference on Computer Vision,Columbus,OH,USA,2014:94-108.
［22］Simonyan K,Zisserman A. Very deep convolutional networks for large-scale image recognition［J/OL］.(2015-04-10)［2017-07-01］.https://arxiv.org/abs/1409.1556
［23］Sagonas C,Tzimiropoulos G,Zafeiriou S,et al. 300 faces in-the-wild challenge:The first facial landmark localization challenge［C］// Proceedings of the IEEE International Conference on Computer Vision Workshops.Sydney,Nsw,Australia,2013:397-403.
［24］Wei L,Lee E J. Improved Multi-pose 2D Face Recognition Using 3D Face Model with Camera Pose Estimation Approach and nD-PCA Recognition Algorithm［C］// Proceedings of the Third IEEE International Conference on Convergence and Hybrid Information Technology.Busan,South,Korea,2008,2:728-736.
［25］Jain V,Learned-Miller E. FDDB:A benchmark for face detection in unconstrained settings［R］.UMass Amherst Technical Report, 2010.
［26］Jia Y,Shelhamer E,Donahue J,et al. Caffe:Convolutional architecture for fast feature embedding［C］// Proceedings of the ACM International Conference on Multimedia.Orlando,Florida,USA,2014:675-678.
［27］Yang H,Jia X,Loy C C,et al. An empirical study of recent face alignment methods［J/OL］.(2015-11-16)［2017-07-01］.https://arxiv.org/abs/1511.05049 .
［28］Yang X X,Torre F D L. Supervised Descent Method and Its Applications to Face Alignment［C］// IEEE Conference on Computer Vision and Pattern Recognition.Portland,OR,USA,2013:532-539.
2019, Vol. 40 
