Research Article Open Access

An Air-Written Real-Time Multilingual Numeral String Recognition System Using Deep Convolutional Neural Networks

Meenal Jabde1, Chandrashekhar Himmatrao Patil1, Amol D. Vibhute2 and Shankar Mali1
  • 1 Department of Computer Science, Dr. Vishwanath Karad MIT World Peace University, Pune, MH, India
  • 2 Symbiosis Institute of Computer Studies and Research (SICSR), Symbiosis International (Deemed University), Pune, MH, India

Abstract

Air-writing is a modern practice for free-space writing of characters or words with hand or finger movements. In the new era, hand gestures are commonly used for Human-Computer Interaction (HCI) or controlling machines in several applications. Writing with a finger or holding any object in hand and 3-D movement in the air is helpful for several applications. However, it is hard to build simulated environments due to the complex structure of the hand and regulate the joints. Palm detection is another way to reduce the complexity of hand detection. Deep learning-based techniques enhance object detection tasks with excellent performance in this task. Convolutional Neural Networks (CNN) are the powerful frameworks used in detecting palms facing several challenges in this task. Therefore, we propose a novel approach using hand landmark detection to identify palms. Our proposed CNN-based air-writing recognition model is intended to detect and recognize numbers in 2 different languages. We present the CNN-based 2-D model to detect air-writing in real-time video. The proposed models were implemented on our developed datasets for two languages, Devanagari and English, with 99.55% accuracy. Our contribution is focused on the air-writing interface using a fingertip to write the numbers instead of traditional input devices. In addition, different air-writing gestures are introduced to control the writing activity. Therefore, our proposed model integrates detection, recognition, and control of air-writing activity. Hence, we achieved precise detection and management of palm movements and accurate recognition of air-written numbers. The results are promising and valuable for real-time multilingual numeral string recognition.

Journal of Computer Science
Volume 20 No. 12, 2024, 1712-1722

DOI: https://doi.org/10.3844/jcssp.2024.1712.1722

Submitted On: 13 July 2024 Published On: 8 November 2024

How to Cite: Jabde, M., Patil, C. H., Vibhute, A. D. & Mali, S. (2024). An Air-Written Real-Time Multilingual Numeral String Recognition System Using Deep Convolutional Neural Networks. Journal of Computer Science, 20(12), 1712-1722. https://doi.org/10.3844/jcssp.2024.1712.1722

  • 556 Views
  • 280 Downloads
  • 0 Citations

Download

Keywords

  • Air-Writing
  • Palm Detection
  • Handwriting Recognition
  • HCI
  • CNN