Research on Gesture Recognition Algorithm under Multi-channel Fusion and Design of Ship Virtual Interaction Platform
CHENG Shu-hong1,YANG Zhen-hao1,WANG Chang2
1. Institute of Electrical Engineering, Yanshan University, Qinhuangdao, Hebei 066004, China
2. Institute of Mechanical Engineering, Yanshan University, Qinhuangdao, Hebei 066004, China
Abstract：A gesture recognition algorithm based on multi-channel fusion is proposed to promote the intelligent human-computer interaction of ship virtual interaction platform.It collects gesture images from different perspectives and establishes gesture data sets.Based on the YOLOV4 target detection algorithm, a gesture recognition framework is constructed to recognize the multi-angle gesture images, and the final result is determined by a multi-data fusion algorithm.At the same time, the virtual scene of ship navigation based on Unity 3D is designed to realize the operation of virtual ship stability platform by gesture recognition.Experiments were conducted to test the robustness of the gesture recognition algorithm in complex environment, and compare the one-view gesture identification method based on YOLOV4 with the accuracy of this algorithm.Experimental results show that this gesture recognition algorithm can eliminate false candidate results and retain maximum confidence by multi-data fusion.The average recognition accuracy of gesture is 95.06 % in complicated environments.
［1］Perabo F, Park D, Zadeh M K, et al. Digital Twin Modelling of Ship Power and Propulsion Systems: Applic-ation of the Open Simulation Platform (OSP)［C］// 2020 IEEE 29th International Symposium on Industrial Elect-ronics (ISIE).Delft, Netherlands, 2020:1265-1270
［2］应翰海, 关宏旭. 福姜沙中水道船舶操纵模拟试验研究 ［J］. 水运工程, 2020 (9): 126-131.
Ying H K, Guan H X. Ship maneuvering simulation test of navigation in Fujiangsha middle channel ［J］. Port & Waterway Engineering, 2020 (9): 126-131.
［3］曾庆亮. 基于VR技术的船舶驾驶模拟仿真系统开发 ［J］. 舰船科学技术, 2018, 40 (24): 190-192.
Zeng Q L. Development of ship driving simulation system based on VR technology ［J］. Ship Science And Technology. 2018, 40 (24): 190-192.
［4］符斌, 任鸿翔, 王德龙. 语音交互在三维虚拟船舶仿真平台中的应用 ［J］. 船海工程, 2018, 47 (3): 133-136.
Fu B, Ren H X, Wang D L. Application of Speech Interaction in 3D Virtual Ship Simulation Platform. Ship & Ocean Engineering, 2018, 47 (3): 133-136.
［5］李秀艳, 刘宗毓, 王琦, 等. 基于电学测量的智能手势识别方法研究 ［J］. 仪器仪表学报, 2019, 40 (10): 192-199.
Li X Y, Liu Z S, Wang Q, et al. Research on the intelligent gesture recognition method based on electrical measurement ［J］. Chinese Journal of Scientific Instru-ment, 2019, 40 (10): 192-199.
［6］李露, 胡乃瑞, 高闯, 等. 基于多项式拟合的电容式传感器手势识别系统 ［J］. 电子器件, 2020, 43 (4): 873-877.
Li L, Hu N R, Gao C, et al. Capacitive Sensor Gesture Recognition System Based on Polynomial Fitting ［J］. Chinese Journal of Electron Devices, 2020, 43 (4): 873-877.
［7］Zhang X L, Yang Z, Chen T, et al. Cooperative sensing and wearable computing for sequential hand gesture recognition ［J］. IEEE Sensors Journal, 2019, 19 (14): 5775-5783.
［8］Shin S O, Kim D, Seo Y H. Controlling Mobile Robot Using IMU and EMG Sensor-Based Gesture Recognition［C］// Ninth International Conference on Broadband & Wireless Computing. Guangzhou, China, 2014:554-557.
［9］Hazra S, Santta A.
Robust Gesture Recognition using Millimetric-Wave Radar System ［J］. IEEE Sensors Letters, 2018, 2(4): 2882642.
［10］Smith K A, Csech C, Murdoch D, et al.
Gesture Recognition Using mm-Wave Sensor for Human-Car Interface［J］. IEEE Sensors Letters, 2018, 2(2): 2810093.
［11］Ahuja M K, Singh A. Static vision based Hand Gesture recognition using principal component analysis［C］// IEEE International Conference on Moocs. Madurai, India,2015: 402-406
［12］时梦丽, 李健, 沈添润, 等. 基于超像素的手势识别算法 ［J］. 计算机工程与设计, 2019, 40 (12): 3545-3550.
Shi M L, Li J, Shen T R, et al. Superpixel-based hand gesture recognition algroithm ［J］. Computer Engineering And Design, 2019, 40 (12): 3545-3550.
［13］宋一凡, 张鹏, 刘立波. 基于视觉手势识别的人机交互系统 ［J］. 计算机科学, 2019, 46 (S2): 570-574.
Song Y F, Zhang P, Liu L B. Human-machine Interaction System with Vision-based Gesture Recognition ［J］. Computer Science, 2019, 46 (S2): 570-574.
［14］程淑红, 程彦龙. 融合批量再标准化和YOLOv3的手势识别研究 ［J］. 计量学报, 2021, 42 (1): 29-34.
Cheng S H, Cheng Y L. Hand Gesture Recognition Algorithm Combined with Batch Renormalization and YOLOv3 ［J］. Acta Metrologica Sinica, 2021, 42 (1): 29-34.
［15］谭芳, 穆平安, 马忠雪. 基于YOLOv3检测和特征点匹配的多目标跟踪算法 ［J］. 计量学报, 2021, 42 (2): 157-162.
Tan F, Mu P A, Ma Z X. Multi-target Tracking Algorithm Based on YOLOv3 Detection and Feature Point Matching ［J］. Acta Metrologica Sinica, 2021, 42 (2): 157-162.
［16］张玉燕,任腾飞,温银堂. 一种基于YOLOv3算法的3D打印点阵结构缺陷识别方法［J］. 计量学报, 2022, 43 (1): 7-13.
Zhang Y Y, Ren T F, Wen Y T. A Method for Detecting Internal Defects of Metal Lattice Structure Based on YOLOv3 Algorithm［J］. Acta Metrologica Sinica, 2022, 43 (1): 7-13.
［17］翁建军, 周阳. 海上光污染对船舶夜航安全的影响及对策分析 ［J］. 武汉理工大学学报 (交通科学与工程版), 2013, 37 (3): 549-552.
Wong J J, Zhong Y. Analysis of Impact of Light Pollution at Sea upon Ships Navigation Safety at Sea and Countermeasures. Journal of Wuhan University of Technology (Trainsportation Science & Engineering), 2013, 37 (3): 549-552.
2022, Vol. 43 
