多通道融合下的手势识别算法研究及船舶虚拟交互平台设计

doi:10.3969/j.issn.1000-1158.2022.07.04

摘要
图/表
参考文献(17)
相关文章 (15)

全文: PDF (3664 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要为促进船舶虚拟交互平台的人机交互方式向智能化转变,提出了一种多通道融合下的手势识别算法。在不同视角下采集手势图像并建立手势数据集;搭建基于YOLOv4目标检测算法的手势识别框架,分别对多视角手势图像进行识别,将多个识别结果通过一种多数据融合算法决策出最终手势识别结果。同时,设计了基于Unity3D的船舶航行虚拟场景,通过手势识别输出的指令实现对虚拟船舶稳定度平台的操作。实验测试了手势在复杂环境下该手势识别算法的鲁棒性,与基于YOLOv4的单视角手势识别方法的准确率进行了对比。实验结果显示:该手势识别算法能通过多数据融合剔除错误候选结果并保留最大置信度,在复杂环境下对手势的平均识别准确率达到95.06%。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	程淑红
	杨镇豪
	王唱

关键词 ：计量学, 手势识别, 船舶虚拟交互平台, YOLOv4目标检测算法, 多通道融合

Abstract：A gesture recognition algorithm based on multi-channel fusion is proposed to promote the intelligent human-computer interaction of ship virtual interaction platform.It collects gesture images from different perspectives and establishes gesture data sets.Based on the YOLOV4 target detection algorithm, a gesture recognition framework is constructed to recognize the multi-angle gesture images, and the final result is determined by a multi-data fusion algorithm.At the same time, the virtual scene of ship navigation based on Unity 3D is designed to realize the operation of virtual ship stability platform by gesture recognition.Experiments were conducted to test the robustness of the gesture recognition algorithm in complex environment, and compare the one-view gesture identification method based on YOLOV4 with the accuracy of this algorithm.Experimental results show that this gesture recognition algorithm can eliminate false candidate results and retain maximum confidence by multi-data fusion.The average recognition accuracy of gesture is 95.06 % in complicated environments.

Key words： metrology gesture recognition virtual ship interaction platform YOLOV4 target detection algorithm;multi-channel fusion

收稿日期: 2021-02-02 发布日期: 2022-07-18

PACS:	TP96
	TB973

基金资助:十三五”装备预研共用技术项目(41412040302);河北省重点研发计划(20371801D); 国家重点研发子课题(2021YFB3202303); 河北省教育厅在读研究生创新能力培养资助项目(CXZZBS2022145)

作者简介: 程淑红(1978-),女,河北秦皇岛人,燕山大学教授,博士,主要从事视觉检测、图像处理、水质检测方面的研究。Email:shhcheng@ysu.edu.cn

引用本文:

程淑红,杨镇豪,王唱. 多通道融合下的手势识别算法研究及船舶虚拟交互平台设计[J]. 计量学报, 2022, 43(7): 856-862.
CHENG Shu-hong,YANG Zhen-hao,WANG Chang. Research on Gesture Recognition Algorithm under Multi-channel Fusion and Design of Ship Virtual Interaction Platform. Acta Metrologica Sinica, 2022, 43(7): 856-862.

链接本文:

http://jlxb.china-csm.org:81/Jwk_jlxb/CN/10.3969/j.issn.1000-1158.2022.07.04 或 http://jlxb.china-csm.org:81/Jwk_jlxb/CN/Y2022/V43/I7/856

［1］Perabo F, Park D, Zadeh M K, et al. Digital Twin Modelling of Ship Power and Propulsion Systems: Applic-ation of the Open Simulation Platform (OSP)［C］// 2020 IEEE 29th International Symposium on Industrial Elect-ronics (ISIE).Delft, Netherlands, 2020:1265-1270
［2］应翰海, 关宏旭. 福姜沙中水道船舶操纵模拟试验研究［J］. 水运工程, 2020 (9): 126-131.
Ying H K, Guan H X. Ship maneuvering simulation test of navigation in Fujiangsha middle channel ［J］. Port & Waterway Engineering, 2020 (9): 126-131.
［3］曾庆亮. 基于VR技术的船舶驾驶模拟仿真系统开发［J］. 舰船科学技术, 2018, 40 (24): 190-192.
Zeng Q L. Development of ship driving simulation system based on VR technology ［J］. Ship Science And Technology. 2018, 40 (24): 190-192.
［4］符斌, 任鸿翔, 王德龙. 语音交互在三维虚拟船舶仿真平台中的应用［J］. 船海工程, 2018, 47 (3): 133-136.
Fu B, Ren H X, Wang D L. Application of Speech Interaction in 3D Virtual Ship Simulation Platform. Ship & Ocean Engineering, 2018, 47 (3): 133-136.
［5］李秀艳, 刘宗毓, 王琦, 等. 基于电学测量的智能手势识别方法研究［J］. 仪器仪表学报, 2019, 40 (10): 192-199.
Li X Y, Liu Z S, Wang Q, et al. Research on the intelligent gesture recognition method based on electrical measurement ［J］. Chinese Journal of Scientific Instru-ment, 2019, 40 (10): 192-199.
［6］李露, 胡乃瑞, 高闯, 等. 基于多项式拟合的电容式传感器手势识别系统［J］. 电子器件, 2020, 43 (4): 873-877.
Li L, Hu N R, Gao C, et al. Capacitive Sensor Gesture Recognition System Based on Polynomial Fitting ［J］. Chinese Journal of Electron Devices, 2020, 43 (4): 873-877.
［7］Zhang X L, Yang Z, Chen T, et al. Cooperative sensing and wearable computing for sequential hand gesture recognition ［J］. IEEE Sensors Journal, 2019, 19 (14): 5775-5783.
［8］Shin S O, Kim D, Seo Y H. Controlling Mobile Robot Using IMU and EMG Sensor-Based Gesture Recognition［C］// Ninth International Conference on Broadband & Wireless Computing. Guangzhou, China, 2014:554-557.
［9］Hazra S, Santta A.
Robust Gesture Recognition using Millimetric-Wave Radar System ［J］. IEEE Sensors Letters, 2018, 2(4): 2882642.
［10］Smith K A, Csech C, Murdoch D, et al.
Gesture Recognition Using mm-Wave Sensor for Human-Car Interface［J］. IEEE Sensors Letters, 2018, 2(2): 2810093.
［11］Ahuja M K, Singh A. Static vision based Hand Gesture recognition using principal component analysis［C］// IEEE International Conference on Moocs. Madurai, India,2015: 402-406
［12］时梦丽, 李健, 沈添润, 等. 基于超像素的手势识别算法［J］. 计算机工程与设计, 2019, 40 (12): 3545-3550.
Shi M L, Li J, Shen T R, et al. Superpixel-based hand gesture recognition algroithm ［J］. Computer Engineering And Design, 2019, 40 (12): 3545-3550.
［13］宋一凡, 张鹏, 刘立波. 基于视觉手势识别的人机交互系统［J］. 计算机科学, 2019, 46 (S2): 570-574.
Song Y F, Zhang P, Liu L B. Human-machine Interaction System with Vision-based Gesture Recognition ［J］. Computer Science, 2019, 46 (S2): 570-574.
［14］程淑红, 程彦龙. 融合批量再标准化和YOLOv3的手势识别研究［J］. 计量学报, 2021, 42 (1): 29-34.
Cheng S H, Cheng Y L. Hand Gesture Recognition Algorithm Combined with Batch Renormalization and YOLOv3 ［J］. Acta Metrologica Sinica, 2021, 42 (1): 29-34.
［15］谭芳, 穆平安, 马忠雪. 基于YOLOv3检测和特征点匹配的多目标跟踪算法［J］. 计量学报, 2021, 42 (2): 157-162.
Tan F, Mu P A, Ma Z X. Multi-target Tracking Algorithm Based on YOLOv3 Detection and Feature Point Matching ［J］. Acta Metrologica Sinica, 2021, 42 (2): 157-162.
［16］张玉燕,任腾飞,温银堂. 一种基于YOLOv3算法的3D打印点阵结构缺陷识别方法［J］. 计量学报, 2022, 43 (1): 7-13.
Zhang Y Y, Ren T F, Wen Y T. A Method for Detecting Internal Defects of Metal Lattice Structure Based on YOLOv3 Algorithm［J］. Acta Metrologica Sinica, 2022, 43 (1): 7-13.
［17］翁建军, 周阳. 海上光污染对船舶夜航安全的影响及对策分析［J］. 武汉理工大学学报 (交通科学与工程版), 2013, 37 (3): 549-552.
Wong J J, Zhong Y. Analysis of Impact of Light Pollution at Sea upon Ships Navigation Safety at Sea and Countermeasures. Journal of Wuhan University of Technology (Trainsportation Science & Engineering), 2013, 37 (3): 549-552.