A Robust Home Alone Security System Using PIR Sensor and Face Recognition

. CCTV-based video monitoring technology is one of the fastest growing security technologies markets. The existing video monitoring systems are, however, still not in a position to be used to prevent crime. For public safety purposes, large networks of cameras are increasingly deployed in public places like Residential Buildings, College Campus, offices, airports, railway stations, and shopping malls. Such systems are primarily dependent on human observers and are therefore limited over long periods by factors such as exhaustion and monitoring. In order to overcome this constraint, "intelligent" systems are required, which can highlight the critical data and remove normal conditions that are not a safety hazard. We propose a model utilizing machine learning techniques in order to build these smart systems. This research aims to create an application in real time, which is necessary for labs, places of work or homes where human detection and Recognition will be done for human safety


Introduction
Globalization and societal liberalization and search of new opportunities makes a trend among young people to move to urban areas is increasing A 2015 research article published in Demographic Research, 'One-person households in India' by Premchand Dommaraju [1] places the increase in India at 9.04 million one-person households in 2011 from 6.8 million according to the 2001 census. This reflects a 0.1% increase in such households along with an overall propensity towards the nuclear family setup. On the other hand, the crime incidents targeting home alone senior citizens or kids with the intention to cause harm to the people staying is also increasing alarmingly. Because of many reshaping living arrangements, in majority cases, small kids, single girls, and elderly people stay alone at home and are vulnerable to this kind of attack. Most of these cases could be prevented from becoming a tragedy if timely help is provided. In this project, we propose to address this problem faced by all those who care for the loved ones staying alone in distant places. The remote connection of the people not only keeps them closure, but it also connects them emotionally. One can keep a watch on the dear ones as they get alert when any suspicious event occurs. A smart monitoring system as shown in Figure 1 can help in an investigation in case of problems.
The system intends to overcome the drawbacks of the past surveillance systems and to enhance security, adaptability, and efficiency. The main aim of this research is to automatically detect the humans using the PIR sensor mounted on the front door and activate the CCTV which should be active as long as the visitors are in the house. The system identifies and recognizes the visitors as family, friends, relatives, service provider and Unknown and continuously tracks the movement of the people to recognize the abnormality. * email: harsha.saxena@rait.ac.in **email: hodce@rait.ac.in The system is developed using PIR sensors to detect the presence of humans, Haar cascade for face detection and Local Binary Pattern Histograms (LBPH) for Face Recognition. The Setup of CCTV to capture the human movement at a door using the PIR Sensor. The system recognizes the person at the door and categorizes the people as a family, relative, friend and an unknown person. As soon as the system recognizes the unknown person and sends an alarm and picture to the registered email to prevent the tragedy from happening. There are few problems faced by the face recognition process are change in illumination, low-resolution camera, occlusion due to objects like eye-glasses, hair styling, and makeup is solved in the paper. Using this kind of solution, the monitoring is discrete based on the accuracy of the system. The monitoring person need not be observing the camera footage continuously. The alert message will trigger his attention to observe the live CCTV footage and take necessary action quickly like calling neighbors, police, etc. If missed, then the messages will give the time details and searching for abnormal events in the stored CCTV footage will be facilitated. The further organization of the paper is listed in this section as follows: The literature survey and the related work done before-in this field of research for our studyis covered in Section-2. Section-3 contains the analysis of performances of the various algorithm used in Face recognition. Further, Section-4 and Section-5 contain our proposed methodology to solve the problem and the expected results. Finally, we conclude the paper by stating the conclusion of the entire paper in Section-6. The references of papers we used for working have been annotated in Section-7.

Literature Survey
Security is one of the most challenging house requirements. The goal of this study is to identify a person using face recognition and alert them of safety in danger [2]. One of the applications for image processing is facial recognition. The image processing method involves transforming an image into a digital form and performing some image processing operations, producing an improved image, or collecting some useful data. In our CCTV cameras, we will benefit from image processing and face recognition. Video recording and review of the images collected is a process that requires a significant amount of memory. CCTV video tracking is used in every part of the world today [3]. However, there is no effective video surveillance implemented yet. Video surveillance is usually used to install a camera and review the captured video. However, we can do something better at the same cost [4]. Our device is fitted with a camera in the safe room. A PIR sensor is used along with the camera so that the camera is not turned on every time. If the sensor detects the human presence, the camera activates and begins to record the footage. Human facial characteristics are observed from the frames extracted from the captured video. The image is compared to the picture stored in the data set [2]. When the face is recognized [12] it will open the door for family members and relatives. When the face is not recognized as an alarm/doorbell. It will also track the person inside the house and give an alert message if any suspicious activity happens. There are many Door lock security systems [13] [14] are classified based on technology used GSM based [6][7], smart card-based, Password-based, Biometric based[8] [9], Social networking sites based, RFID based, Bluetooth based, Door phone-based, OTP based [10], Motion detector based using PIR [11], VB based, Combined system.

Analysis of Literature Survey
A literature survey shows that a lot of work is still going on face recognition in videos to improve its accuracy. Table 1 shows the surveys. Analyzing the literature, we came to know some limitations and scope to work on the face detection problem as follows: Even though still image face recognition results are excellent, face recognition in video frames is still an unsolved problem as the frames are influenced by real-time effects like the motion, lighting, no of people, and occlusion [2]. The conventional face recognition such as Eigen and Fisher facing problems in the detection of side views. Besides, face recognition on Fisherfaces is not ideal for the shift in lighting in different real conditions [5].
Facial recognition based upon LBPH, consisting of an array of histograms and blocks measuring distance. The scheme detects the right faces, even in slanting pictures, by showing a rectangular region on the face [5]. With the use of a locally normalized histogram with a gradient orientation similar to SIFT descriptors [17], it produces very good results in the detection of persons, which reduces false-positive rates.
The Authors in the paper [23] proposed the method in which the aim was to detect and recognize the face in Real-time. They achieved an overall system accuracy of 96.8% by using MTCNN for face detection and Inception-Res Net network for Face Recognition. The author created the database of 100 faces, if the face matched with the face in the database, then it is considered as a known person else it will be recognized as unknown. They detected face under various circumstances and achieved average confidence value under illumination 93%, Head Pose 94.15%, Occlusion by hand 89%, Face Expression 91% and Makeup 95%. But the technology used was time-consuming, it takes more time for recognition when multiple faces occur in the environment. Few faces were not tracked when video moves faster. In this paper face recognition is done by using CNN, the accuracy obtained is very accurate but it needs very high computation cost as it passes through many hidden layers. There is a need to look into the simplest and fastest technology to find an efficient and less time-consuming recognition algorithm.
By considering limitations that is the speed of computation and accuracy for recognizing a face. we have proposed a Robust Home Security System Using PIR Sensor and LBPH Face Recognition which requires less computation cost and provides good accuracy is explained further in the next section.

Proposed Methodology
This is interdisciplinary research to address the remote monitoring of the dear ones using smart ubiquitous environment. The different components and the technology required to address the problem are microcontroller, sensors, signal processing, communication, image processing, database management, log keeping and soft computing techniques to build a ubiquitous environment.
The flow of the proposed research is as shown in Figure 2 followed by the explanation of different important technology/components.

Passive Infrared (PIR) Sensor
A PIR detector is a motion sensor that detects a living body's heat. The sensor is passive because it is sensitive to the infrared energy generated by every living thing, instead of transmitting a light beam or microwave energy disturbed for a passing person to detect another person. When an intruder enters the field of vision of the detector, it senses the sudden increase in energy level. The machine is installed at the front door, as shown in Figure 3. The sensors are designed to distinguish between intrusion movement and vegetation oscillation. The program focuses on the smallest of creatures like dogs, wolves, leopards, and tigers compared to humans. The machine can distinguish between humans and animals by measuring the intruders ' height. To do this, the system would conduct two classifications for each signal detected: firstly, the distinction between vegetation and non-vegetation, and then human and animal classifications. Once people are spotted, the camera turned on and start recording. This saves the energy of the system.

Face Recognition and Classification
A pattern recognition function primarily performed on faces is called facial recognition. It can be described as classifying a face as Family Member, Relative, Friend or unknown, after matching it with stored known individuals as a database. Upon confirmation that the family member or relative is the individual, the door is opened immediately. It is also important to have a program that can learn to recognize unfamiliar faces.
Five key functional blocks have their roles in Figure 4, as shown below.

Figure 4: Flow chart of Face Recognition and Classification
A. The acquisition module The user sets the face image as the input for the face recognition device within this module. This is the entry point of the face recognition process. The face is captured from the real-time input stream.
B. Pre-Processing The images are normalized in this module to improve machine recognition. Pre-processing steps are normalization of image size, background elimination, translation and rotational normalization, normalization of illumination.
C. Face Detection In our system, we only need the frontal faces that are normalized in scale from the input images. To reduce the computation for feature extraction, it is important to localize and extract the facial region from an image. We are using Haar cascade for face detection.
The haar features are used for face detection and are of a rectangular type which is determined by an integral image. Figure 5 shows different types of haar features that are similar to a few properties common to human faces. The eye region is the one which is darker than the upper cheeks so the second type of haar feature in figure 6 is used to detect that facial region and haar feature for the nose bridge region which is brighter than the cheeks as shown in the figure 6. Here, using these features we can find the locations of eyes, bridge of nose and mouth by calculating Feature Value = ⅀ the sum of pixels in the black area -⅀ the sum of pixels in the white area It is used for facial edge detection and hence the output is a horizontal high-value line. In haar, we use 24 X 24-pixel sub-window on the image window to find the edge therefore many possible features can be extracted which are further used for facial region detection. We use this window size 24 X 24 as we are ignoring the face which are smaller than window size. We create a kernel using haar features to extract this line. Then apply the kernel to the whole image and it has a high output only where the image value matches the kernel that is our expected output. In Cascade classifier, the term cascade means several filters on the resultant, that's why cascade is used to combine many features efficiently. It on discarding non-faces images to avoid the unnecessary work and spend more time on images with probable face regions. Therefore, a cascade classifier issued which is composed of stages containing strong classifiers. So that with the output from each we can discard non-facial images. Although training to create new Haar-cascade is important, OpenCV has a robust collection of Haarcascades that were used for the project. Figure 7 displays the flow diagram of the detection system. D. Recognition and Classification To recognize a person, we have to train our machine for certain images of that person. Here training is done using Haar cascades and Local Binary Pattern Histograms (LBPH). Haar features are used to detect faces in video frames, now each detected face is treated as a dataset to train our machine and LBPH features corresponding to each face are saved in a file. The extracted features of the face image are compared with the ones stored in the face database. It is used to match the test face image is then classified as either Family Member, Relative, Friend or unknown. There are three stages for face recognition as follows: Stage 1: Feature encoding The image is split into 3x 3 pixels (cells). The centers, as shown in Figure 8, are compared with the centers in a clockwise or anti-clockwise direction. The neighbor is equal to the middle pixel with the frequency or luminosity. A 1 or 0 is allocated to the position based on the difference being higher or lower than 0. This gives the cell value of 8-bit. The benefit of this approach is even if there is a variation in the luminosity the result will remain the same.   Figure 10. Histogram intersection is used to measure the similarity between two histograms. It is a distance matching algorithm that measures the dissimilarity strength between two entities using Chi-square as a dissimilarity measure.
The same can be used for weighted histograms generated using LBPH. The less the distance, the more similar are the two histograms. The extracted feature histogram of the new sample images is compared with the one stored in the Template Database and check for the extent of match between the two. F. Alarm Generation After detecting the face of the person, he will classify as Family Member, Relative, friend or unknown. If an unknown person is detected an alert message through the wireless signal is sent to the Owner mobile.

Results
The proposed methodology was implemented on a system that had an Intel i3 processor running with 4GB RAM, on windows 10. For facial recognition, It only takes face as an entry, if the camera doesn't find a face then it stays paused for any face. Our research has been checked on 75 people in which 5 faces of Family members, 30 faces of Friends, 30 of relative and 10 unknown faces are taken. LBPH is trained for 65 faces, if a face is not from the training set then it will be labeled as unknown.
We used the confusion matrix table for calculating the accuracy of our techniques as shown in Table 2. A live implementation of the project on a real-time scenario is depicted below. Figure 11 shows the result of face detection and recognition which shows that the system is used to recognize the real-time face and classified as either Family, Relative, Friend or unknown.  and Multiple Face Recognition We achieved 78.67% accuracy in Real-Time scenarios under many variations and unconstrained environments like change in illumination, low-resolution camera, occlusion due to objects like eye-glasses, hair styling, and makeup. The accuracy of the system is less as compared to the accuracy achieved by Roshni Singh [23] that is 96.8% by using MTCNN for face detection and Inception-Res Net network for Face Recognition, but the system need more computational time for training and testing. The proposed system takes 112.95 seconds time for training and 0.1259 seconds time for testing the model. The system can detect, recognize and classify a face as Family, Relative, Friend or unknown and generate alert accordingly in Real-Time.

Conclusion
This is a research problem with huge scope to improve the performance of the system with respect to sensing, tracking, and controlling the home access by the visitors. In this paper we discussed the advantages and limitation of various techniques. To overcome the limitation, we develop the system that senses the humans and activates the camera as long as the visitors are in the house. It works on Real-Time and identify the visitors as family, friend, relative and an unknown with good accuracy of 78.67 percent and will raise an alarm in case of an emergency.