|
|
|
| Dynamic Gesture Recognition Method Based on DSConvBiGRU Network and Thermopile Array |
| GU Liang,YU Lianzhi |
| School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China |
|
|
|
|
Abstract The DSConvBiGRU network model which is suitable for embedded systems and combines depthwise separable convolutional neural networks and bidirectional gated recurrent units for the classification of dynamic gesture sequences is proposed. A dynamic gesture recognition solution which utilizes a low-resolution thermopile array sensor is designed and implemented. An experimental dataset comprising various dynamic gestures has been constructed and publicated on open website. The deployment of the pre-trained network model on the Raspberry Pi edge device has been accomplished. The system preprocesses 20 consecutive temperature matrices exported by the sensor through interval mapping, background subtraction, Lanczos interpolation, and Otsu thresholding to obtain a single dynamic gesture sequence. Subsequently, the pre-trained DSConvBiGRU network is employed for the classification. Experimental results demonstrate that the network model achieves an accuracy of 99.291% on test dataset. The time comsunption of preprocess and inference on the edge device is 5.513ms and 8.231ms respectively. The system meets the design requirements for low-power consumption, high precision, and real-time performance.
|
|
Received: 02 August 2023
Published: 06 June 2024
|
|
|
|
|
|
| ［6］ |
胡德勇. 基于数据手套的动态手势识别研究［D］. 大连:大连海事大学,2020.
|
|
TIAN Q H, YANG H M, LIANG Q L, et al. Overview on vision-based dynamic gesture recognition［J］. Journal of Zhejiang Sci-Tech University(Natural Sciences), 2020, 43(4):557-569.
|
|
CHENG S H, YANG Z H, WANG C. Research on Gesture Recognition Algorithm under Multi-channel Fusion and Design of Ship Virtual Interaction Platform［J］. Acta Metrologica Sinica, 2022, 43(7): 856-862.
|
|
LIU X H, DENG B S, PEI Y, et al. Research on Wearable Gesture Interaction System and Recognition Algorithm［J］. Journal of Chinese Computer Systems, 2020, 41(11):2241-2248.
|
| ［5］ |
王剑波. 基于Kinect的动态手势交互控制技术研究［D］. 西安:西安工程大学,2021.
|
| ［7］ |
屈乐乐,吴涵旭,杨天虹,等. 基于双通道CNN的FMCW雷达手势识别方法［J］.电子器件,2022,45(6):1469-1475.
|
| ［8］ |
金展翌. 基于红外摄像头的手势识别［D］. 南京:东南大学,2020.
|
| ［10］ |
BINNIE T D, ARMITAGE A F, WOJTCZUK P A. A passive infrared gesture recognition system［C］//IEEE SENSORS. Glasgow, UK, 2017:1-3.
|
| ［12］ |
李少荣. 基于改进直方图均衡化的红外图像增强技术的研究［J］. 工业控制计算机,2022,35(12):52-53,56.
|
| ［1］ |
田秋红,杨慧敏,梁庆龙,等. 视觉动态手势识别综述［J］. 浙江理工大学学报(自然科学版),2020,43(4):557-569.
|
| ［3］ |
刘璇恒,邓宝松,裴育,等. 穿戴式手势交互系统与识别算法研究［J］. 小型微型计算机系统,2020,41(11):2241-2248.
|
|
QU L L, WU H X, YANG T H, et al. Hand Gesture Recognition Method Using FMCW Radar Based on Dual Channel CNN［J］. Chinese Journal of Electron Devices, 2022, 45(6):1469-1475.
|
| ［11］ |
TATENO S, ZHU Y W, MENG F X. Hand Gesture Recognition System for In-car Device Control Based on Infrared Array Sensor［C］//58th Annual Conference of the Society of Instrument and Control Engineers of Japan. Hiroshima, Japan, 2019:701-706.
|
| ［13］ |
余丹. 基于热像仪的手势识别研究［D］. 南京:东南大学,2019.
|
| ［15］ |
KEN T. Filters for common resampling tasks［J］. Graphics gems, 1990: 147-165.
|
| ［16］ |
丁杨. 实时低复杂度增强视频编码研究［D］. 重庆:重庆邮电大学,2022.
|
| ［17］ |
王智超,杨喜旺,黄晋英,等. 基于ITD-SDP图像特征和DSCNN的道岔转辙机故障诊断［J］.铁道学报,2023,45(5):65-71.
|
| ［18］ |
CHOLLET F. Xception:Deep learning with depthwise separable convolutions［C］//IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA, 2017: 1251-1258.
|
| ［19］ |
张立峰,王智,吴思橙. 基于卷积神经网络与门控循环单元的气液两相流流型识别方法［J］.计量学报,2022,43(10):1306-1312.
|
| ［20］ |
TRAN D, BOURDEV L, FERGUS R, et al. Learning spatiotemporal features with 3d convolutional networks［C］//IEEE International Conference on Computer Vision(ICCV). Santiago, Chile,2015: 4489-4497.
|
| ［23］ |
HOWARD A, SANDLER M, CHU G, et al. Searching for MobileNetV3［C］// IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea, 2019: 1314-1324.
|
| ［2］ |
程淑红,杨镇豪,王唱. 多通道融合下的手势识别算法研究及船舶虚拟交互平台设计［J］. 计量学报,2022,43(7):856-862.
|
| ［9］ |
VANDERSTEEGEN M, REUSEN W, VAN BEECK K, et al. Low-latency hand gesture recognition with a low resolution thermal imager［C］//IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle, USA, 2020:98-99.
|
| ［22］ |
MUTHUKUMAR K A, BOUAZIZI M, OHTSUKI T. A novel hybrid deep learning model for activity detection using wide-angle low-resolution infrared array sensor［J］. IEEE Access, 2021, 9: 82563-82576.
|
| ［4］ |
JACOB A, KOSHY M, NISHA K K, et al. Real Time Static and Dynamic Hand Gestures Cognizance for Human Computer Interaction［C］//International Conference on Advances in Computing and Communications (ICACC). Kochi, Kakkanad, India, 2021:1-6.
|
|
LI S R. Infrared Image Rnhancement Technology Based on Improved Histogram Equalization［J］. Industrial Control Computer, 2022, 35(12):52-53, 56.
|
| ［14］ |
LIANG Q S, YU L, ZHAI X P, et al. Activity Recognition Based on Thermopile Imaging Array Sensor［C］//IEEE International Conference on Electro/Information Technology(EIT). Rochester, USA, 2018: 770-773.
|
|
ZHANG L F, WANG Z, WU S C. Flow Pattern Recognition of Gas-Liquid Two-phase Flow Based on Convolutional Neural Network and Gated Recurrent Unit［J］. Acta Metrologica Sinica, 2022, 43(10):1306-1312.
|
| ［21］ |
FAN X, ZHANG H, LEUNG C, et al. Robust unobtrusive fall detection using infrared array sensors［C］//IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).Daegu, Korea, 2017: 194-199.
|
|
WANG Z C, YANG X W, HUANG J Y, et al. Fault Diagnosis of Turnout Switch Machine Based on ITD-SDP Image Features and DSCNN［J］. Journal of the China Railway Society, 2023, 45(5): 65-71.
|
|
|
|
2024, Vol. 45 
