ASL Recognition Quality Analysis Based on Sensory Gloves and MLP Neural Network

Authors

  • Firas A. Raheem University of Technology
  • Hadeer A.Raheem

Keywords:

ASL, Artificial neural network, forward kinematics, inverse kinematics, deaf, DOF.

Abstract

A simulated human hand model has been built using a virtual reality program which converts printed letters into a human hand figure that represents American Sign Language (ASL), this program was built using forward and inverse kinematics equations of a human hand. The inputs to the simulation program are normal language letters and the outputs are the human hand figures that represent ASL letters. In this research, a hardware system was designed to recognize the human hand manual alphabet of the ASL utilizing a hardware glove sensor design and using artificial neural network for enhancing the recognition process of ASL and for converting the ASL manual alphabet into printed letters. The hardware system uses flex sensors which are positioned on gloves to obtain the finger joint angle data when shown each letter of ASL. In addition, the system uses DAQ 6212 to interface the sensors and the PC. We trained and tested our hardware system for (ASL) manual alphabet words and names recognition and the recognition results have the accuracy of 90.19% and the software system for converting printed English names and words into (ASL) have 100% accuracy.

References

[1] D. M. Perlmutter. “what is sign language,” Linguistic Society of America. Washington, DC 20036-6501. Available: https://www.lsadc.org/.
[2] Cobos, Salvador, et al. "Simplified human hand models for manipulation tasks." Cutting Edge Robotics 2010. InTech, 2010.
[3] Huenerfauth, Matt. "A multi-path architecture for machine translation of English text into American Sign language animation." Proceedings of the Student Research Workshop at HLT-NAACL 2004. Association for Computational Linguistics, 2004.
[4] Zhao, Liwei, et al. "A machine translation system from English to American Sign Language." Envisioning machine translation in the information future (2000): 191-193.
[5] Morrissey, Sara, and Andy Way. "An example-based approach to translating sign language." (2005).
[6] Kim, Taehwan, Karen Livescu, and Gregory Shakhnarovich. "American sign language fingerspelling recognition with phonological feature-based tandem models." Spoken Language Technology Workshop (SLT), 2012 IEEE. IEEE, 2012.
[7] Pugeault, Nicolas, and Richard Bowden. "Spelling it out: Real-time asl fingerspelling recognition." Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on. IEEE, 2011.
[8] Wysoski, Simei G., et al. "A rotation invariant approach on static-gesture recognition using boundary histograms and neural networks." Neural Information Processing, 2002. ICONIP'02. Proceedings of the 9th International Conference on. Vol. 4. IEEE, 2002.
[9] Pansare, Jayashree R., Shravan H. Gawande, and Maya Ingle. "Real-time static hand gesture recognition for American Sign Language (ASL) in complex background." Journal of Signal and Information Processing 3.03 (2012): 364.
[10] Dong, Cao, Ming C. Leu, and Zhaozheng Yin. "American sign language alphabet recognition using microsoft kinect." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2015.
[11] Allen, Jerome M., Pierre K. Asselin, and Richard Foulds. "American Sign Language finger spelling recognition system." Bioengineering Conference, 2003 IEEE 29th Annual, Proceedings of. IEEE, 2003.
[12] Shembade, Yateen P. Design and Simulation of a Mechanical Hand. Rochester Institute of Technology, 2012.
[13] De Marco, Robert Michael, and R. A. Foulds. "Data recording and analysis of American Sign Language." Bioengineering Conference, 2003 IEEE 29th Annual, Proceedings of. IEEE, 2003.
[14] Wang, Honggang, Ming C. Leu, and Cemil Oz. "American Sign Language Recognition Using Multi-dimensional Hidden Markov Models." Journal of Information Science and Engineering 22.5 (2006): 1109-1123.
[15] Butterworth, Rod R. The Perigee Visual Dictionary of Signing: An A-to-Z Guide to Over 1,350 Signs of American Sign Language. Penguin, 1995.
[16] Sternberg, Martin LA. American sign language dictionary. HarperPerennial, 1998.
[17] van der Hulst, Frank PJ, et al. "A functional anatomy based kinematic human hand model with simple size adaptation." Robotics and Automation (ICRA), 2012 IEEE International Conference on. IEEE, 2012.
[18] Lowe, Whitney. Orthopedic Assessment in Massage Therapy. Daviau Scott, 2006.
[19] Schilling, Robert J. Fundamentals of robotics: analysis and control. Simon & Schuster Trade, 1996.
[20] Hind Z. Khaleel, ''Inverse Kinematics Solution for Redundant Robot Manipulator using Combination of GA and NN,” Al-Khwarizmi Engineering Journal, Vol. 14, No. 1, pp. 136-144, 2018.
[21] Spong, Mark W., Seth Hutchinson, and Mathukumalli Vidyasagar. Robot modeling and control. Vol. 3. New York: Wiley, 2006.
[22] Firas A. Raheem, Azad R. Kareem and Amjad J. Humaidi, 'Inverse Kinematics Solution of Robot Manipulator End-Effector Position Using Multi-Neural Networks', Eng. &Tech.Journal, Vol.34,Part (A), No.7, 2016.

Downloads

Published

2018-09-24

How to Cite

Raheem, F. A., & A.Raheem, H. (2018). ASL Recognition Quality Analysis Based on Sensory Gloves and MLP Neural Network. American Scientific Research Journal for Engineering, Technology, and Sciences, 47(1), 1–20. Retrieved from https://asrjetsjournal.org/index.php/American_Scientific_Journal/article/view/4383

Issue

Section

Articles