A Novel Communication System For Deaf And Dumb People using gesture

Human Beings know each other and contact with themselves through thoughts and ideas.The best way to present our idea is through speech. Some people don’t have the power of speech; the only way they communicate with others is through sign language. Now a days technology has reduced the gap through systems which can be used to change the sign language used by these people to speech. Sign language recognition (SLR) and gesture-based control are two major applications used for hand gesture recognition technologies. On the other side the controller converts the sign language in to the text and speech which gets converted with the help of text to speech conversion and analog to digital conversion. A Dumb person throughout the world uses sign language for the communication.The best way to present our idea is through speech. Some people don’t have the power of speech; the only way they communicate with others is through sign language. Now a days technology has reduced the gap through systems which can be used to change the sign language used by these people to speech. Sign language recognition (SLR) and gesture-based control are two major applications used for hand gesture recognition technologies. On the other side the controller converts the sign language in to the text and speech which gets converted with the help of text to speech conversion and analog to digital conversion. A Dumb person throughout the world uses sign language for the communication.


Introduction
In today's world dumb and deaf people face a lot of difficulties in day to day life to communicate with the normal people. To ease their communication problem, we design and develop a communication system that will convert their sign language into the speech and text. As the name suggests, this system gives voice to voiceless i.e. voice is given to the person who is not able to speak. Dumb people use signs to communicate with the people. This sign includes different forms which are made with hand to understand what the person wants to say. This sign language includes different types of hand gestures such as, different facial movements which express the feelings of the persons. In our proposed system flex sensors (mounted on gloves) plays an important role. These flex sensors are attached to the gloves using a thread. Flex sensors are those which carry a certain amount of current which flows through the flex sensors. Further, AVR (Alf and Vegard's RISC processor) controller (mounted on gloves) takes the input signal in analog form from the flex sensors which is further converted into digital form by using inbuilt micro-controller. Further, digitally converted data is transferred to the android device from microcontroller to the android device wirelessly using Bluetooth chip. Hence, this digital signal (user instruction) is sensed by the android device and will speak according to the character generated.
The existing system dealt with similar problems has some drawbacks and used video devices such as camera for processing data [4]. But the key limitations of the existing system are lack of feasibility (not robust for the use), portability, and accuracy [5]. Further, in the existing system author used contact sensors, image processing are used which makes the project complex in nature. The other limitations of the existing project were limited number of letters can be implemented (American sign language only) [1], unable to display or recognize the sentences, special gestures/patterns are not included, and help message is not implemented.
The key motivation of the proposed IOT based device is to ease the day to day life of disable people who face problems in communication. They face many difficulties in their day to day life because of communication gap with other people. This project will help them to easily communicate with the other people with just their hand gestures. The overall idea of the proposed work in this paper is describe as in sensor-based system, the values which are stored in the microcontroller are transferred to the mobile device wirelessly. After the data stored in the device is matched with the corresponding values it displays the respected text. This text is also converted in to speech by text to speech conversion. Hence, the key contributions of the proposed work in this paper are described as follows: 1. To design and develop a robust and low-cost communication system for deaf and dumb people 2. To design a portable and high accuracy system. 3. To design and develop a low maintenance a low maintenance system using lithium battery.
The reminder of this paper is organised as follows: Literature survey with reference to novel communication system for deaf and dumb people is given in section II. Section III provides proposed work. Experimental setup, results and analysis are described in section IV. Section V presents the conclusion and future work.

Literature Survey
The existing work of the IOT based gesture recognition system is described as follows: W. Jingqiu et al [1], suggests design and develop a communication system through an fusing of manual as well as non-manual signals. Manual signals basically refer to hand posturing and hand motions, whereas non-manual signals resemble to an external appearance. The main limitation of the proposed system is lack of device mobility due to wired signal. A. Z. Shukor et al [2] authors used their own sign language in place of generalized sign language for example (American Sign Language (ASL) and Germany Sign Language (GSL)). However, there are similar sign languages for each country but slightly change version of sign languages in different regions of same country. Therefore, it is a challenge to implement a universal sign language used worldwide.
N. Sriram et al [3] study, used sign languages which is used for recognizing by using two different methods that is vision and non-vision approaches. But the key limitation of the proposed work is lots of miss interpretation of data.
To resolve the problem with vision approach S. V. Matiwade et al [4] designed and developed a system which uses a camera to observe the information received through finger and hand movement is the most widely adopted visual-based approach. But the key limitation of the proposed work was lack of portable in nature. A similar Approach was used by More and Sattar S et al [6] using SIFT algorithm, that further lowered dimensions of the feature vector by using Principal Component Analysis (PCA) algorithm to speed up the time of processing. Since the key limitation of the proposed work was author used storage space to store the database which requires lot of external storage space. To recognize dynamic hand movements used in Japanese Sign Language (JSL),Murakami and Taguchi [7] proposed utilizing the recurrent neural systems fit for perceiving the JSL finger letters in order, which comprise of 42 gestures. Neural network was used by author for the gesture reorganization which increase the complexity of the system.
In variation, Chowdary et al. [8] has used a basic checking approach to calculate the movement and fingers orientation in binary that transform images which are captured from a web camera. More sophisticated recognition of gesture system was proposed by Khan and Pathan [9] making use of digital images, that includes filtering of image(pre-processing), segmentation of image, color segmentation of, detection of skin(detecting hand and finger by using binary images), and matching the template. For the time being, an indirect method recognizes fingers and hand gestures on the basis of the RGB color spaces divided based on various hue for each finger by making use of a data glove.
An achievable division strategy utilizing RGB shading spaces utilizing a hand glove for seeing motion was proposed by Siby et al. [10]. This method utilizes estimations of RGB shading space drawn out from a hand signal picture caught from information glove, and contrasts those qualities and the one those put away in database.Further, Lamberti and Camastra [11] a continuous hand motion acknowledgment framework was proposed utilizing a learning vector quantization classifier for separating three shades of wool glove comparing to the fingers, palm and remaining parts of glove from image that are captured.
The limitation of a vision-based technique is enormously influenced by the processing of t images, likely background cancellation, image filtering, detection of boundary and color segmentation, for example, uncontrolled and diverse images of background can influence color of the skin segmentation or detection of movement. In fact, many researchers have failed to address such complications, and no strong solutions are yet been proposed. As a result, a non-vision-based technique is an alternative approach. This method typically utilizes motion and flex sensors to calculate the fingers flexion and the orientation of hand, separately.
The summary of existing work with their limitations is described in Table 1.

Proposed Work
In our proposed work in this paper we designed and develop a data glove for communication of dumb and deaf people. The flow chart of the proposed system is shown in figure 1. In the our proposed system we used Arduinoboard, Flex sensors, Accelerometer, Bluetooth module and Android Mobile. To recognize the hand movement first of all we design a glove by mounting the flex sensors, PCB circuit, components on PCB circuit such as (microcontroller, accelerometer, battery, Bluetooth module and voltage regulator). The Changes in hand movement will generate a voltage drop in flex sensors in analog form will be forwarded to microcontroller. After receiving the analog signal microcontroller converts the analog signal to digital signal. The digital signal via Bluetooth module send to android user. The second stage is to write and install the program on microcontroller for gesture reorganization. After that develop an android app for communication and we used Bluetooth for the interfacing. Once the values are send to the mobile device it maps the word or letter matching to that value and display that letter or word. Once the hand moment gets matched with the value in mobile device it prints the particular word or letter and also voice is generated in mobile device.  When we attach a accelerometer then we get a axis which can be used for every finger direction.
Micro controller : This is the heart of the complete system. Here AVR AT-mega16 microcontroller is used. It has in built ADC. The analogue input data from flex sensors is converted to digital form by micro controller.
Bluetooth Module : Microcontroller sends the digital data to the mobile device using Bluetooth module wirelessly.The Bluetooth module used in this system is HC-05.
Android Mobile : The digital signal which is transferred from the microprocessor is recorded in the android device.There is the fix algorithm for every input and according to the input the output is generated in the device.
Power Supply : An AVR microcontroller and flex sensor needs 5V DC voltage whereas operating voltage of Bluetooth module is 3.3V. Power supply consists of transformer, rectifier, filter and regulator. The rectifier used here is a Bridge Rectifier which converts 230V AC into desired DC voltage.

Result And Discussion
To check the performance of our proposed data glove we set the experimental setup. We used Bluetooth connectivity for transferring the data from microcontroller to android device. We designed an android app using basic for android software (B4A) which is based on c language. After setting the detailed hardware setup user generates the input signal by bending the wrist movement and finger movements. On the basis of all the movements of data gloves the out will generates in the form of digital signal. Then we map the voltage value with the alphabets implemented in the program. And then after it will display the recognized alphabets with its sound. Hence, by this way we can generate the word and sentences. Further, theses word and sentences will be used for the communication of deaf and dump people. Hence, by this way we can provide a strong communication media to deaf and dump people for their communication and makes their life easy.    the different voltage values. Hence, the voltage value is changing according to finger and wrist movements as we can see in figure5 the voltage is varying with reference to time. Figure 6 shows the plot when the glove is held as a fist. In this position, all the voltages are low as can be seen below. Below Figure 7 shows the hand gestures that represents 26 characters from A to Z and also some sentences which are used in daily life for deaf and dumb people.  Table 2 describes the voltage value of each character in the alphabet from a to z. These different voltage values we got from three flax sensors and theses values used for character reorganization.

Conclusions And Future Work
This paper highlights the to reduce the communication gap between deaf or mute community and normal people. The proposed system in this paper will improve dumb/ deaf person's lifestyle. Even it will be beneficial for the communication between the blind person and the dumb person. Overall System is effective and efficient because the use of AVR microcontroller and android phone. The system can be extended to support more number of signs and different languages mode.
In future work we will implement different day to day life activities with the help of different sensors