International Journal of Neural Network, 2021, 2(3); doi: 10.38007/NN.2021.020303.
Wei Dong
Xizang Minzu University, Xizang, China
As a communication method widely used among hearing-impaired people, sign language is a natural language that transmits information through the three-dimensional space of the "manual-visual" channel, and solves the communication problem of deaf people through human-computer interaction. The problem is a qualitative leap, and human-computer interaction effectively solves the communication problem between deaf people and ordinary people. Therefore, this paper relies on the convolutional recurrent neural network (CRNN) to study the sign language recognition (SLR) method. This paper first describes the two concepts of SLR and image processing, and then builds the CNN-LSTM model through the CRNN structure, network training and parameter settings, and finally analyzes the CNN-LSTM model. Model analysis shows that the CNN-LSTM model has high SLR accuracy and good recognition performance.
Convolutional Neural Network, Recurrent Neural Network, Sign Language Recognition, CNN-LSTM Model
Wei Dong. A Sign Language Recognition Method Relying On Convolutional Recurrent Neural Networkk. International Journal of Neural Network (2021), Vol. 2, Issue 3: 19-26. https://doi.org/10.38007/NN.2021.020303.
[1] Lee M, Lee S, Lee J, et al. Predicting Maximum Wind Speed of Typhoons based on Convolutional Recurrent Neural Network via COMS Satellite Data. Journal of Korean Institute of Industrial Engineers. (2019) 45(4):349-360. https://doi.org/10.7232/JKIIE.2019.45.4.349
[2] Park M, Kim H, Park S. End-to-End Autonomous Driving Based on Convolutional Recurrent Neural Network. Journal of Korean institute of intelligent systems. (2019) 29(4):297-301. https://doi.org/10.5391/JKIIS.2019.29.4.297
[3] Kumar E K, Kishore P, Kumar M, et al. Three-Dimensional Sign Language Recognition With Angular Velocity Maps and Connived Feature ResNet. IEEE Signal Processing Letters. (2018) 25(12):1860-1864. https://doi.org/10.1109/LSP.2018.2877891
[4] Savant R, Ajay A. Indian Sign Language Recognition System for Deaf and Dumb Using Image Processing and Fingerspelling: a Technical Review. National Journal of System and Information Technology. (2018) 11(1):23-34.
[5] Gndz C, Polat H. Turkish Sign Language Recognition Based on Multistream Data Fusion. Turkish Journal of Electrical Engineering and Computer Sciences. (2021) 29(2):1171-1186. https://doi.org/10.3906/elk-2005-156
[6] Al-Shamayleh A S, Ahmad R, Jomhari N, et al. Automatic Arabic Sign Language Recognition: A Review, Taxonomy, Open Challenges, Research Roadmap And Future Directions. Malaysian Journal of Computer Science. (2020) 33(4):306-343. https://doi.org/10.22452/mjcs.vol33no4.5
[7] Saleh B M, Al-Beshr R I, Tariq M U. D-Talk: Sign Language Recognition System for People with Disability using Machine Learning and Image Processing. International Journal of Advanced Trends in Computer Science and Engineering. (2020) 9(4):4374-4382. https://doi.org/10.30534/ijatcse/2020/29942020
[8] Tabassum T, Mahmud I, Uddin M P, et al. Enhancement of Single-Handed Bengali Sign Language Recognition Based on Hog Features. Journal of Theoretical and Applied Information Technology. (2020) 98(5):743-756.
[9] Kasmin F. A Comparative Analysis of Filters towards Sign Language Recognition. International Journal of Advanced Trends in Computer Science and Engineering. (2020) 9(4):4772-4782. https://doi.org/10.30534/ijatcse/2020/84942020
[10] Kaushik N, Rahul V. A Survey of Approaches for Sign Language Recognition System. International Journal of Psychosocial Rehabilitation. (2020) 24(1):1775-1783. https://doi.org/10.37200/IJPR/V24I1/PR200278
[11] Mallick T, Balaprakash P, Rask E, et al. Graph-Partitioning-Based Diffusion Convolutional Recurrent Neural Network for Large-Scale Traffic Forecasting: Transportation Research Record. (2020) 2674(9):473-488. https://doi.org/10.1177/0361198120930010
[12] Hoshi I, Shimobaba T, Kakue T, et al. Single-Pixel Imaging Using a Recurrent Neural Network Combined with Convolutional Layers. Optics express. (2020) 28(23):34069-34078. https://doi.org/10.1364/OE.410191
[13] Rokade P, Sali N, Shinde D, et al. Indian Sign Language Recognition System in Marathi Language Text. International Journal of Computer Sciences and Engineering. (2019) 7(5):881-885. https://doi.org/10.26438/ijcse/v7i5.881885
[14] Elakkiya R, Vanitha V. Interactive Real Time Fuzzy Class Level Gesture Similarity Measure based Sign Language Recognition Using Artificial Neural Networks. Journal of Intelligent & Fuzzy Systems. (2019) 37(5):6855-6864. https://doi.org/10.3233/JIFS-190707
[15] Ibrahim N B, Zayed H H, Selim M M. Advances, Challenges, and Opportunities in Continuous Sign Language Recognition. Journal of Engineering and Applied Sciences. (2019) 15(5):1205-1227. https://doi.org/10.36478/jeasci.2020.1205.1227
[16] Kotari M. Sign Language Recognition using Image-based Hand Gesture. Global Journal of Engineering Science and Research Management. (2019) 2(5):237-240.
[17] ME Martínez-Gutiérrez, JR Rojano-Cáceres, E Benítez-Guerrero, et al. Data Acquisition Software for Sign Language Recognition. Research in Computing Science. (2019) 148(3):205-211. https://doi.org/10.13053/rcs-148-3-17
[18] Ravi S, Maloji S, Polurie V, et al. Sign language recognition with multi feature fusion and ANN classifier. Turkish Journal of Electrical Engineering and Computer Sciences. (2018) 26(6):2872-2886. https://doi.org/10.3906/elk-1711-139