A Review on Fatigue Driving Detection

: The socialization of automobile development has brought great convenience to people's travel. However, the rapid increase in the number of vehicles has also caused a series of problems. The increase in traffic accidents has brought great social casualties and economic losses. Fatigue driving, which is an important factor in the traffic accident, has aroused people's attention. This paper reviews all kinds of fatigue driving detection methods at present; compares various fatigue driving detection methods in terms of accuracy, real-time and cost; analyses the advantages and disadvantages of various methods; introduces the application of fatigue detection system in automobile; summarizes the current deficiencies and future development trends in the field of fatigue driving detection. The future research of this field will be more to the data fusion, computer vision and deep learning.


Introduction
The number of cars is increasing rapidly in recent years, which has led to an increase in the number of traffic accidents.Frequent traffic accidents have brought a lot of casualties and economic losses and become a serious problem around the world [1][2][3].It is an important task for people to determine the causes of traffic accidents and to find suitable methods to reduce the incidence of accidents.Fatigue driving is one of the most important factors of traffic accidents.According to the statistical data of the China National Bureau of statistics [4], there were 1029735 traffic accident in the past 5 years, causing 304671 deaths and direct economic loss of 5 billion 294 million yuan.The traffic accidents caused by fatigue driving accounted for 20% of the total number of accidents and more than 30% of highway traffic accidents.Fatigue driving refers to the driver's mental or physical disorders due to lacking of sleep or long time driving, which results in the decreased control of the vehicles [5].Therefore, it will be able to effectively reduce the probability of traffic accidents by finding out the causes and characteristics of fatigue driving and proposing an effective fatigue driving detection method.At present, more and more researchers focused on fatigue driving detection method [6].
The main methods of fatigue driving detection can be divided into subjective detection methods and objective detection methods [7][8].The subjective detection methods require the driver to determine their degree of fatigue by checking the driving record sheet and their own physiological response.The objective detection methods extract the characteristic parameters related to driving with special equipment, and then evaluate the driver's fatigue degree based on the model build with the extracted parameters.
Subjective fatigue detection methods are simple and easy.But their accuracy and real-time performance is very poor because these methods require driver to test their own fatigue level.This approach will be affected by driver's individual differences.The objective detection methods are more reliable and accurate because they use special model or algorithm to analyze the driver's fatigue degree with the characteristic parameters related to driving.The driver only needs to focus on driving the vehicle.However, the current fatigue driving detection systems have not been widely used because they need special equipment to collect data, which need high cost.In addition, these current algorithms still have room for improvement in accuracy.
Although current methods have various drawbacks, they have great potential in reducing the incidence of traffic accidents.So in this paper, all kinds of popular objective fatigue driving detection technology are summarized and sorted out, which can lay a solid foundation for the future research of fatigue driving detection technology.

Method
Objective detection methods use various sensors or image acquisition equipment to collect physiological parameters, driver behavior parameters, vehicle parameters and other kinds of parameters.The collected parameters are used as input of the analysis model to evaluate the fatigue degree of driver.This kind of method is more objective and reliable than the subjective detection methods.Therefore, the objective fatigue detection methods have been referred to as the research focus in the field of scientific research and engineering application.This paper focuses on objective detection methods.As shown in Figure 1, the objective detection method can be divided into three types: driver physiological parameters based detection method, driver facial features based detection method, and the vehicle driving parameters based detection method.

Driver Physiological Parameters Based Detection Method
Related studies showed that [9], when a driver is in the state of fatigue, his physiological response will become slow, his body's response to stimulation appears to be delay, his physiological indicators will deviate from the normal state.Therefore, driver's physiological parameters collected by physiological sensors can be used to determine whether the driver is in the fatigue state.At present, there are many mature detection methods, such as the methods based on the detection of EEG (electroencephalogram), EMG(electromyography), ECG(electrocardiogram), pulse beat and respiratory frequency [10][11][12].

EEG Detection
Brain waves are fluctuations in the potential of synchronous changes in nerve cells in the brain.The change of EEG reflects the change of human brain.There are four kinds of wave in the human brain, delta wave, theta wave, alpha wave and beta wave.The delta wave and theta wave increase while the alpha wave and beta wave decrease when the driver changes from a state of consciousness to a state of fatigue [13].Based on the relationship between the change of band signal and degree of driver fatigue, detection algorithm is able to use the real-time EEG signals to make accurate judgments on the degree of driver fatigue.EEG signals is the most important and reliable index for fatigue detection [13].
Mohammedabubasim developed an auxiliary driving system based on the EEG signals [14].They first collected the real-time EEG signals by sensors.Then they processed the signals by Gobar filter and the linear SVM classifier and get the analysis result.At last they send alarm messages to drivers falling into fatigue state.
EEG signal detection has high reliability and sensitivity, strong anti-interference ability.But its cost is high because the structure of the instrument is complex and sophisticated.In addition, it may affect the driver's operation because the sensors directly contact with the driver.

EMG Detection
EMG is the superposition of action potential of motor unit in time and space.SEMG is a comprehensive effect of superficial muscle electrical signals and electrical activity on the skin surface, which can reflect the neuromuscular activity to some extent [15].When the driver's fatigue degree deepens, the frequency of EMG signal will continue to decline, the amplitude gradually increased [16].Therefore, the analysis of EMG signals can achieve the purpose of the driver's fatigue level.In general, the evoked potential method is used to measure the EMG signal, it uses an electrode fixed on the muscle surface of the driver to collect the signals.
HONG et al. [15] judged the driver's fatigue based on the driver's EMG signals.They used sensors to collect the EMG signal of the driver, processed the original signals with the FastICA algorithm and digital filter.The processed results were classified using the Mahalanobis distance and classification results were used to judge the degree of driver fatigue.
The detection method based on EMG signal is objective and true, but it has some invasion to the driver's skin, which is not conducive to the driver's safe driving.

ECG Detection
The ECG is produced by the weak current in the process of cardiac excitation.When the driver is tired, the ECG signal will show a significant regularity decline [17].The heart rate (HR) index and heart rate variability (HRV) index are the important indexes to judge the degree of fatigue.
Sangeetha et al. [18] developed a vehicle mounted fatigue driving detection system.The system collected the driver's ECG signal and compared it with the stored fatigue driving ECG signal.The system sent driver an alarm message when the similarity between the two signals reached the threshold.
ECG detection method has the characteristics of easy to carry and easy to operate.However, its sensitivity is poor.And because of the individual differences of drivers, their heart rate change is different, which results in the high false positive rate of this method.

Pulse Beat Detection
Pulse beat can also reflect the degree of driver fatigue.When the driver is tired, the intensity and frequency of the pulse gradually decrease.Japanese Convenience Computer Company [19], collected real-time driver pulse signal parameters by sensors mounted on the steering wheel, and used algorithm model to analysis the signal parameters.The analysis result was used to judge the fatigue degree of the driver.This method is simple and easy.However, the direct contact of the sensor to the driver will make the driver feel uncomfortable.Besides, due to the different pulse changes of different drivers, this method is not accurate enough.Therefore this method is best used as an adjunct to other methods.

Breathing Frequency Detection
Similar to the pulse signal, when the driver enters the fatigue driving state, the breathing frequency tends to be gentle.In this way, the driver needs to wear the respiratory frequency sensor, which will affect the driver's normal driving.Therefore, the method is also suitable for use as an auxiliary method for other methods.

Driver Facial Features Based Detection Method
When the driver is in a state of fatigue, the facial features of the driver will be different from that of the conscious state.Therefore, it is an effective method to detect fatigue driving in real time by continuously collecting and analysing the driver's facial features data [20].The changes in the head and face features are more obvious and most easily detected, thus the research hotspot at present also gathered in this part.The major methods of detection include PERCLOS, head position detection, gaze direction detection, blink frequency detection and mouth state detection [21][22][23].

Perclos
Carnegie Mellon University Driving Research Center developed PERCLOS system [24] in 1998.This system was based on the theory that the reflection ability of retina to infrared light of different wavelengths is different in the same illumination.It located the retina according to the difference of image formed by different wavelength, analysed the size and position of the eyes, and finally determined the eyes closed according to the image of eye region.
The experiment shows that there is a close relationship between the length of the eyes closed and the degree of fatigue.The longer the eyes are closed, the more serious the degree of fatigue.The PERCLOS method is used to evaluate the driver's fatigue degree by comparing the percentage of eye closure time per unit time.
The PERCLOS method has the advantages of high accuracy, real-time and non-contact.It is the most widely used detection method in the market currently.However, this method is easy to be affected by light, occlusion, driver's glasses and other factors.It still has room for improvement in robustness.

Head Position Detection
In the course of driving, the position of the head is different when the driver is awake and tired.Therefore, it is feasible to determine whether the driver is in a state of fatigue by detecting the driver's head position.Reference [25] proposed a method to detect fatigue by the change of head acceleration.It collected the head acceleration parameters by setting the three axis acceleration sensor ADXL330 on the driver's seat behind his head, which was based on the Micro-Electro-Mechanical Systems (MEMS).
The advantages of this method are real-time and reliable.The disadvantages are high cost, low sensitivity and immunity.So it is not suitable as the main judging parameter, but as an auxiliary parameter.

Gaze Direction Detection
The sight line of awake driver will be in front of the vehicle and move fast.The sight line of fatigued driver will deviate from the normal direction and move slow.Inho Choi et al. [26] used the CDF analysis method to locate the driver's pupil position, and then to detect the gaze direction of the driver's sight line and the changing frequency of the sight line.These two indicators were used to estimate the driver's fatigue level.
The equipment of gaze direction detection is simple and cheap.But the reliability and sensitivity of this method is poor.It is suitable to be used in combination with other detection methods.

Blink Frequency Detection
Blink frequency refers to the number of blink times per unit time.When the driver is tired, the blink frequency will be higher than the normal value.Sigari et al. [27] used the computer vision technology to extract the driver's eye information and calculated the blink frequency.The calculated blink frequency was used to analyse the driver's fatigue.
Blink frequency detection method has the characteristics of real-time and high accuracy.Its robustness is poor because it is easy to be affected by occlusion, driver's glasses and other factors.With the development of computer vision technology, this problem can be overcome [28].Therefore, blink frequency is an important fatigue evaluation index.

Yawning Detection
Yawning is one of the most important and obvious facial features of fatigue driving.Many researchers have detected the fatigue degree by extracting the characteristics of the driver's mouth [29][30].Based on the image recognition technology, researchers extracted and analysed the mouth features of driver to judge whether the driver was yawning.They evaluated driver's fatigue degree based on the number of driver yawn per unit time and the mouth opening degree when driver yawns.This method is easy to go wrong when the driver is just talking rather than yawning.In order to solve this problem, the researchers [31] used hidden Markov model to analyse the driver's mouth characteristics, and successfully separated the two characteristics of talk and yawning.The method of yawning detection has the advantages of ITA 2017 real-time and high accuracy, which is suitable to be used as an auxiliary method for fatigue detection.

Vehicle Driving Parameters Based Detection Method
With the deepening of fatigue, the driver's ability to perceive the surrounding environment, judge the situation and control the vehicle will decline.This change makes driver's control precision reduced, which can be reflected by the abnormal fluctuation of vehicle driving parameters.
Compared to the acquisition of various parameters for the driver, the acquisition of vehicle driving parameters is simpler.Current vehicles are equipped with various types of sensors, which are used to collect the real-time parameters of vehicle speed, fuel consumption and engine speed [32].It is possible to indirectly detect the driver's fatigue degree through the modeling of these data.Therefore, the detection methods of fatigue driving based on the vehicle driving parameters also receive a lot of attention in research field.The popular methods include steering wheel angle detection method, steering wheel grip detection method, vehicle speed detection method, vehicle offset detection method, brake pedal force detection method and the accelerator pedal force detection method [33][34][35].

Steering Wheel Angle Detection
When the driver tired, his ability to control the car will fall.The amplitude of the steering wheel will increase, and then the value of steering wheel angle does not change significantly in the next period of time, the operation of the steering frequency decreases.It is a common method to determine the fatigue driving by collecting and analysing the parameters of the steering wheel angle in time, amplitude and frequency from steering wheel sensors.The researchers used method of sample entropy to analyse steering wheel rotation characteristics [34].They found the sample entropy under the normal state is greater than the sample entropy under fatigue state.So they developed the method to determine driver fatigue based on rotation angle of steering wheel detection.
The steering wheel angle detection method can easily collect the precise data from sensors without affecting the driver's normal driving.But its accuracy is not enough because the driver's driving habits, cars and driving environment are different.

Steering Wheel Grip Detection
When the driver is in the state of fatigue, the steering wheel grip will gradually decrease.Based on this characteristic, researchers proposed an algorithm to analyze the change rule of the driving steering wheel grip [35].The data was collected by the force sensor installed on the steering wheel.
Just like the steering wheel angle detection method, the detection signal parameters are easy to obtain by the sensors embedded in the steering wheel.And this method has no effect on the driver.But the accuracy is still not enough.

Vehicle Speed Detection
The vehicle speed detection method mainly uses the CCD camera and the vehicle mounted sensor to detect the vehicle's speed, lateral acceleration, lateral displacement and other parameters.In reference [36], the parameters of vehicle driving were obtained by simulation driving.After being denoised, the processed parameters were used to train a classifier for distinguishing driver's awake state and fatigue state.
The vehicle speed detection method can easily collect the precise speed parameters of vehicle with low cost.However, the change of vehicle speed is difficult to accurately reflect the driver's fatigue degree.This method can only be used as an auxiliary parameter.

Vehicle Offset Detection
The vehicle offset detection method mainly monitors the vehicle's travel path by using the computer vision technology to identify the lane.It sends the driver alarm messages when the vehicle is found to deviate from the lane.AutoVue system [37] developed by American Iteris, detects the driver's track by the image data collected by a front facing CCD camera in the car.The system will alarm when it finds vehicle lane departure caused by fatigue driving.
The advantages of vehicle offset detection include high accuracy and non-contact detection.The problem is that it is easily affected by weather, road conditions and other factors.In addition, it is too late for fatigue driving detection when the vehicle is out of control.

Summary
These common fatigue detection methods have their own advantages and disadvantages.As shown in Table 1, the three kinds of fatigue detection methods are compared in terms of accuracy, sensitivity, contact, anti-jamming and cost.On the whole, the fatigue detection method based on driver facial features is a promising method.objectively and accurately reflect the degree of driver fatigue.The disadvantages are that data acquisition equipments are complex and expensive; the equipment contact the driver's body directly, which affects the normal operation of the driver and the traffic safety.Therefore, the application of this method to the actual driver fatigue detection is limited.
The advantage of the fatigue detection method based on driver facial features is intuitive and obvious.It can more accurately determine the degree of driver fatigue.Besides, it realizes the feature extraction of non-contact, which does not affect the normal driving of the driver.The disadvantages are that the recognition algorithm is complex, the feature extraction is difficult, and the detection results are easily affected by the illumination and occlusion factors.But in recent years, image recognition, machine learning techniques have made great progress.Especially the deep learning technology has made remarkable achievements in the field of computer vision.Method based on convolutional neural network (CNN) can quickly and accurately extract useful features from the images for analysis.Therefore, the driver fatigue detection method based on driver facial features is the mainstream direction in the future development of this field.
Fatigue detection method based on vehicle driving parameters has the advantage of simple and real-time data acquisition.The disadvantages are that its analysis result is easily influenced by personal driving habits, the weather, traffic conditions and other external factors, the accuracy is enough; moreover, the method can detect the abnormality only when the driver is about to have an accident, which is not safe for driver.Therefore, the analysis result of this method is better to be used as an auxiliary detection indicator rather than the main detection indicators.

Application of Fatigue Driving Detection Technology
After a long period of development, the fatigue driving detection technology has been gradually transferred from the field of research to the field of industrial applications.However, due to the low detection accuracy and high cost, fatigue detection products are far from universal but only appear in some high-end vehicles.The representative products are as follows.
Attention assist system equipped with Benz [38], which uses 71 sensors to collect parameters of vehicle steering wheel angle, acceleration, accelerator pedal, brake pedal and various external environmental parameters.These parameters are used to conduct a scientific analysis of the driver's driving behavior.The system alerts when the driver is in a state of fatigue.
The European Union has launched a comprehensive monitoring system for driving behavior called "AWAKE" project [39].The system collects driving data with a variety of sensors of image, pressure and speed.The collected data include parameters like eye movement, gaze direction, steering wheel grip, and driving speed, which are used to analyse the driver and vehicle behavior.The driver's fatigue degree is divided into wakefulness, fatigue and deep fatigue.The system improves the driver's alertness by acoustic optic stimulation and seat belt dithering.
Volvo automobile is equipped with a driver safety warning system (DAC) [40].The system uses a camera to collect driver's facial feature data and use sensors to collect the parameters of the vehicle movement.The comprehensive analysis result of these two kinds of data is used judge the fatigue state of driver.The system warns the driver who is in a state of fatigue by displaying the warning text on the dashboard and making a sound in the vehicle.
Most of the existing fatigue driving detection productions judge the fatigue state of driver based on multi characteristic parameters, which guarantees the accuracy of analysis result.At present, the production cost of this kind of detection equipment is high, which leads to the failure of wide application of the fatigue detection system.The market now needs low cost and high accuracy products.Fatigue detection method based on driver facial feature will be promising.It only needs special camera to collect the facial features of driver, the cost is low.With the development of deep learning, the ability of the machine to extract features from a picture will be enhanced and the accuracy of fatigue analysis by machine will be improved.

Future Works
The driver fatigue detection system needs to detect the driver's fatigue accurately and in real time without affecting the driver's normal driving and comfort.This section analyses the current problems and future development trend of fatigue driving detection technology.The future works will be focused on two aspects: detection technology research and engineering implementation.

Detection Technology Research
Fatigue detection technology research includes research fields of the data sources selection, data collection and data analysis.Problems existing in these fields can be summarized as follows.

Data Sources Selection
Current fatigue detection methods are based on one single kind of parameters like driver physiological parameters, driver facial features and vehicle driving parameters.The driving behavior of driver cannot be represented completely by a single kind of feature parameters, which only contain limited information.So the present methods need to be improved both in accuracy and robustness.Method based on multiple kinds of parameters is able to obtain more credible analysis result.Therefore, how to integrate multiple kinds of parameters for fatigue detection is a problem need to be solved.
From this point of view, the data fusion technology is helpful to detect fatigue driving, which integrates a ITA 2017 variety of parameters of driver and vehicle to judge the fatigue driving.Any single detection method has its own disadvantages, so the accuracy of method which is based on one single parameter is not enough.With the development of information fusion technology, many researchers try to integrate the driver's physiological parameters, driver facial features and the running parameters of the vehicle to judge the fatigue state of the driver.Considering all kinds of factors, this method can effectively improve the accuracy of the results, and reduce the probability of false alarm.
From the present point of view, fatigue detection method based on driver facial features works best overall.By means of data fusion, a new method with higher accuracy and stronger robustness can be obtained by analysing the driver facial features and vehicle related parameters.

Data Collection
Driver's facial image data and Vehicle driving parameters are able to be collected by camera and vehicle sensor easily.Driver physiology parameters are hard to be collected because current sensors need to be in contact with the human body, which will affect the normal driving.This is the reason why the method based on driver physiological parameters has not been widely used, although the driver physiological parameters are the most reliable parameters to measure the fatigue degree.
There are two ways to improve the sensor, noncontact sensor and wearable sensor.Non-contact sensor can be deployed to the vehicle and collect various kinds of driver's physiological parameters.With these data, computer can analyse and obtain most reliable fatigue state of the driver.But the point is the manufacturing cost, and if the costs of sensors are too high, then this method will be meaningless.In recent years, wearable mobile device like Apple Watch and Mi Band have emerged.These kinds of devices are able to collect the wearer's physiological parameters without making him uncomfortable.Wearable sensor is a better choice for fatigue detection.Compared to the traditional sensor, wearable sensor will not affect the normal driving.

Data Analysis
Most fatigue detection methods are based on traditional machine learning algorithms for data analysis.Traditional machine learning algorithms require manually designed features.The abstraction degree of these kinds of features is low, so their representation power is weak.Therefore, the analysis results of current fatigue detection methods are easily affected by light, shadow, driving habits, weather environment and other factors.Getting a feature with high abstraction degree and strong representation power is the work to be done.
Combining the deep learning technology, the more robust feature extraction models are able to be trained.In recent years, the deep learning develops rapidly, which has achieved good results in research fields like image analysis, video analysis and NLP Natural Language Processing .It is able to learn and extract more abstract and robust features from the input, which has stronger representation power.Therefore, driving fatigue detection method based on deep learning is also a hot research field in the future.

Engineering Implementation
Engineering implementation refers to the method of applying technology to the industrial field.In the field of fatigue driving detection, the current approach is to deploy data acquisition devices, data analysis equipment, and alarm devices to the vehicle.The advantage is that the vehicle has become an independent system and it does not need to communicate with other systems.But the problem is that the data analysis requires on-board embedded system with high processing capacity because real-time performance is essential.This has led to a high cost of the entire fatigue driving detection system, which is the reason why the current fatigue detection system only appeared in high-end vehicles.Therefore, how to apply the fatigue detection technology to the vehicle at lower cost is also a problem to be considered.
"Internet of Vehicles" based on big data processing technology is a solution that can be considered [41][42].
In the system of "Internet of Vehicles", the vehicle is responsible for data acquisition, transmission and analysis results reception while the big data processing system is responsible for data processing and analysis [43][44].In this way, the purpose of real time can be achieved under the premise of satisfying accuracy.This solution reduces the cost of on-board embedded system because the most onerous work is done by the "Cloud".It should be noted that this solution requires bandwidth and stability of mobile networks.But with the development of mobile networks, solutions based on "Internet of Vehicles" are the future trend.

Conclusions
The increase in the number of cars will lead to an increase in the number of traffic accidents.Fatigue driving is one of the most important factors in traffic accidents.Therefore, it is of great significance to study the fatigue detection methods with high accuracy, realtime and strong anti-interference.One single kind of parameters is not accurate enough to determine the fatigue state of driver.Fatigue detection methods based on data fusion technology judge the fatigue driving by the fusion data of parameters related to vehicle and driver.This is an important research direction; In addition, fatigue detection methods analysing fatigue state by deep learning are able to get more robust detection result.This is a hot research field; finally, the "Internet of Vehicles" is the development trend of intelligent transportation.It is a good solution to combine "Internet of Vehicles" and driver fatigue detection.The strong data transmission and analytical capabilities will make a great contribution to the accuracy and real-time performance of fatigue driving detection.

Table 1 .
Method comparison