Lane Detection Techniques using Image Processing

Lane detection is a developing technology that is implemented in vehicles to enable autonomous navigation. Most lane detection systems are designed for roads with proper structure relying on the existence of markings. The main shortcoming of these approaches is that they might give inaccurate results or not work at all in situations involving unclear markings or the absence of them. In this study one such approach for detecting lanes on an unmarked road is reviewed followed by an improved approach. Both the approaches are based on digital image processing techniques and purely work on vision or camera data. The main aim is to obtain a real time curve value to assist the driver/autonomous vehicle for taking required turns and not go off the road


I. INTRODUCTION
Autonomous vehicles are the next big step in the transportation sector due to their efficiency and safety benefits. A self-driven car needs to understand its surroundings so it can navigate its way in streets with minimal human assistance. Driver and passenger safety systems are one of the rising topics in automotive development. Lane keep and Steering assist are some of the systems that allow vehicles to detect lanes and help in keeping the car in an ideal position with respect to the road and assist in turning.
There are several works presented on the well structured roads having proper lane markings, but fewer on unmarked roads. Two of such approaches are reviewed in this paper. These are based on Digital Image processing techniques. [5] It involves using various computer algorithms for manipulating images. This is performed to obtain an image that is according to desired requirements which can be with improved clarity, sharpness, details, reduced noise for efficient extraction and analysis.
The common steps in both approaches are Thresholding, Warping and Region of Interest(ROI). Then further processing includes Pixel Summation(Histogram), Gaussian Blur, Image dilation,Canny Edge Detection, and Sliding Window algorithm which are subject to specific methods. Using Gaussian Based road model [2] lane could be detected using a multi stage algorithm. The processes involved in these stages are histogram generation, fitting and normalisation. Different ordered polynomial functions are used for lane fitting.

II. LITERATURE REVIEW
In the approach used in [3], curvature is detected using image processing. The captured image is converted into binary space and segmented into three sections. The curvature direction is determined on the basis of the section having the most number of non-zero pixels. Approach I follows similar logic, but for calculating curve values average values [8] of the histogram of the lower part of the binary image are used instead of performing image segmentation.
The approach [4] presents an efficient version of sliding window algorithm for lane curve fitting. One of the advantages of this improved version is that it allows sliding window to be applied even on uneven lane markings. In the second approach, this method is applied onto the road edges from the binary image. The coordinates of the road edges are then processed to obtain the curve value and the vehicle offset.
Few of the popular algorithms includes GOLD system developed by Broggi which uses an edge-based lane boundary detection algorithm [9]. Remapping of acquired image is done to represent in bird's eye view of the road. Extraction of quasi vertical bright lines that concatenated into specific larger segments is done using specific adaptive filtering.
The RALPH system [10], used to control the lateral position of the vehicle. Lane's curvature and lateral offsets are determined by a template to the averaged scan line intensity profile which is adaptively adjusted and aligned using a matching technique. The same university developed another system called AURORA which tracks the lane markers present on structured road using a color camera mounted on the side of a car pointed downwards toward the road [11].

A. Benefits of Lane Detection
• Lane Detection System can provide assistance and certain details to the drivers and as well as pedestrians. • Such systems can allow drivers to drive safely and also help them to stay on their particular lane by providing some kind of indicators.
• According to research presented by WHO, [6] about 1.35 million human deaths were caused due to road traffic injuries. Autonomous vehicles could bring this number down by some extent. • According to McKinsey's report in 2015, the mass adoption of autonomous vehicles could possibly reduce road traffic accidents by 90%. • Since software will drive the car, [7] the modern vehicle can now be programmed to reduce emissions up to 60% according to Ohio University.

III. LANE DETECTION TECHNIQUES IN DETAIL
For detection of lane and getting curve values, among the two approaches, initial steps are essentially the same i.e Thresholding and Warping. All the processing will be done on the video frame captured by the camera which is supposed to be mounted on the vehicle.
A. Approach I 1) Thresholding: : Since the approach is for the unmarked roads, Colour thresholding is done in order to detect the colour of the road instead of detecting the lane markings. The thresholding is not applied directly on the image obtained from video feed. Images obtained from video feed are in BGR (Blue Green Red) format, and are converted into HSV(Hue Saturation Value) [12] since it is better for object or colour detection. HSV color space represents how colors behave under light. Mathematically HSV consists of three matrices for 'Hue', 'Saturation', and 'Value' with ranges lying between 0-179, 0-255 and 0-255 respectively. The main application for HSV is color based image segmentation which is used in both of the approaches for separating the road from its surroundings. Thresholding will convert the image into a binary image i.e the area where the color of the path is detected will be white and the rest of the part of that image will be kept black. 2) Warping: : The image obtained via the camera module has an angled perspective. The lane edges cannot be precisely located using this image as is. In order to fix this image warping concept is used. Image warping is used to get a birds eye / top view of the path by changing the perspective of the image. In Perspective Transformation [14], the perspective of an image or a video frame is changed in order to get certain insights in a better way. For warping a set of points is provided over the image. The values of these points will determine the Region of Interest (ROI). With these two sets of points the perspective of the image or a video frame is transformed. It is done by passing these coordinates to OpenCV library functions. The resultant would be an image which would be cropped as per the ROI and having a bird's eye perspective. Warping is one of the most common preprocessing techniques used in lane detection systems as it provides an image upon which further calculations would be easier as compared to original image obtained through camera. The basic idea is to get a rectangle shape when the road is straight.

B. Approach II
In this approach, instead of using the entire road pixels only the edges of the path are considered. The initial steps of Capturing the image, Warping and Thresholding are the same as the previous approach. A few image pre-processing procedures are performed to get a clearer view of the path edges. The extracted edge geometry is then used to determine the curvature using sliding window algorithm.
1) Gaussian Blurring: : Due to uneven road conditions, the edges of the thresholded image are too sharp. This may Various methods are available like averaging, bilateral filtering, Gaussian blurring and median blurring [13]. For the test case in consideration, Gaussian blurring provided the best balance in reducing noise and smoothening out the sharp edge imperfections.
This algorithm scans the entire image pixels, followed by recalculating the pixel values. The intensity of this operation is adjusted by a kernel. The greater the kernel size greater is the blur effect but at the same time can lead to decreased details.
Gaussian blur mixes the values of the neighbouring pixels depending on the distance to the reference pixel. The pixels right next to the reference pixel get more of the colours from the reference pixel while the pixels farther away get less of the colours. This can be viewed graphically wherein the values closer to 0 on the x-axis have higher magnitude on the y-axis which gradually decreases as we go further away from the centre.

Fig. 4: Gaussian Curve
Thus the uneven edges get smoothened out as the pixels don't have a too sharp change in their values after blurring.
2) Canny Edge Detection: : The next step is to extract the lane boundaries from the thresholded binary image. For straight lines, this can be achieved by Hough transform. However, the proposed test case needs to detect curved edges as well so Canny edge detection algorithm is used. The Canny edge detector uses a multi-stage algorithm [15] to detect edges in images by identifying large changes in gradients of the pixels. The steps involved are: • The noise present in the image is reduced with a 5x5 Gaussian filter. • The intensity gradient of the image is found by Sobel kernel which detects horizontal, vertical and diagonal edges. • A scan of the complete image is performed followed by removing any unwanted pixels which may not represent the edge; this is also known as Non-maximum Suppression. • The final stage i.e Hysteresis Thresholding decides which all edges are really edges and which are not. For this, two threshold values -min and max are used. Any edges having intensity gradient more than the maximum value are accepted as edges and those below minimum value are are discarded. The remaining which lie between these two values are classified based on their connectivity. 3) Image dilation: : The result of the canny detection function is an image with just the edge geometry of the path with the remaining image features removed. This needs to be passed to a searching algorithm for detection of this geometry. But before doing so image dilation is performed which makes the edges more clear and filled with less breaks. Dilation expands the image pixels and adds pixels to object boundaries.  A. Approach I: • As it provides the approximate turning values without using the edge positions, offset value can't be calculated.
• This can be used as a fallback method for traditional methods wherein the edges are too disoriented.

B. Approach II
• It can be applied for both situations -lane markings or no lane markings (using edges of road).
• It is more accurate than the 1st approach as using the edge geometry gives a precise value of the lane position.
• Other important measures such as vehicle offset can be measured which are not possible using approach 1 • The downside of this method is if the edges are too uneven or obscured by vegetation the curve values may get disturbed.

V. CONCLUSION
The overall purpose of this project is to enable lane detection in poor road conditions wherein the lane markings are not clearly visible or worn out. This project approaches for the implementation of lane detection with the help of image warping, thresholding and approaches such as pixel summation and sliding window algorithm for an intelligent autonomous vehicle. Finally, the advantages and shortcomings of the above methods along with their ideal use case scenarios have been discussed.