Automated Pavement Distress Detection Using Image Processing Techniques

Pavement crack and pothole identification are important tasks in transportation maintenance and road safety. This study offers a novel technique for automatic asphalt pavement crack and pothole detection which is based on image processing. Different types of cracks (transverse, longitudinal, alligator-type, and potholes) can be identified with such techniques. The goal of this research is to evaluate road surface damage by extracting cracks and potholes, categorizing them from images and videos, and comparing the manual and the automated methods. The proposed method was tested on 50 images. The results obtained from image processing showed that the proposed method can detect cracks and potholes and identify their severity levels with a medium validity of 76%. There are two kinds of methods, manual and automated, for distress evaluation that are used to assess pavement condition. A committee of three expert engineers in the maintenance department of the Mayoralty of Baghdad did the manual assessment of a highway in Baghdad city by using a Pavement Condition Index (PCI). The automated method was assessed by processing the videos of the road. By comparing the automated with the manual method, the accuracy percentage for this case study was 88.44%. The suggested method proved to be an encouraging solution for identifying cracks and potholes in asphalt pavements and sorting their severity. This technique can replace manual road damage assessment. Keywords-pavement distress; AEOP; python code; image processing

INTRODUCTION Road traffic accidents are globally becoming a big issue [1]. According to recent World Health Organization reports, more than 1.25 million people are killed, and 50 million are injured in road traffic accidents annually [2]. Road defects play a significant role in the occurrence of accidents [3]. Road networks are an essential part of our daily life. Pavements deteriorate over time for a variety of causes. The load of the travelling vehicles may cause a pavement to deteriorate. Transverse cracks, longitudinal cracks, block cracks, and alligator cracks are the four types of cracks found in early pavement deterioration. If these early deteriorations are not addressed, potholes emerge, making the road more unsafe. Rehabilitation procedures, such as fixing potholes, will cost 10 to 20 times as much as resealing cracks. Pavement detection and rating are important to maintain the cost of repairing road deterioration low [4]. Traditionally, the cracks in the pavement are manually inspected. Manual inspection is not only timeconsuming, inefficient, and prone to errors, but it can also lead to security incidents. Most researches aim to locate and fix pavement cracks as fast as possible [5]. Several studies have been carried out to create a system for automated detection of pavement cracks that solves the restrictions of the manual technique [6].
According to threshold values obtained from histograms of an image, the authors in [7] did an entropy and image dynamic thresholding to automatically divide pavement crack pixels into crack and non-crack pixels. Every binary image achieved by using the threshold value was classified into non-overlapping blocks in which the occurrence of cracks was confirmed using the entropy function and the thresholding operation although the findings are not consistent with every examined image, because of the various lighting circumstances while utilizing a single threshold. Authors in [8] presented another use of image processing on crack identification, focusing and getting an extra specific crack image free of noise. The given approach appears hopeful, but it is time-consuming because it analyzes a one-lane 10m road for a few minutes. Authors in [9] suggested a mixed method for automatic crack identification based on 3D ultra laser-imaging pavement data. Matched filtering was applied to show the cracks, while tensor voting was used to find the main directions of the cracks, and the minimal spanning tree was utilized to identify the crack path. Meanwhile, authors in [10] developed an approach for detecting and segmenting cracks in pavement images by utilizing a steerable filter with two shifted and rotatable tails to achieve satisfactory crack segmentation accuracy. Authors in [11] recently made an automated algorithm-based image processing approach for segmenting and improving pavement crack identification utilizing 3D pavement images.
Although image processing techniques have been used effectively in crack finding, there are yet certain concerns regarding the accuracy and efficiency that must be addressed. Authors in [12] gave an image processing algorithm, special for the rapid evaluation of cracking in the pavement surface. Authors in [13] invented a photogrammetric method to categorize and enumerate the number of pavement cracks automatically. Authors in [14] proposed a novel automatic crack discovery approach depending on a segment extending for sophisticated pavement movie frames. Interrelated segments are connected to produce a crack by analyzing the relationship between connected domains, and the character of crack trend could be best employed in crack differentiation. Real pavement surface images were utilized to investigate the method's execution, and the results demonstrated that the surface pavement crack could be accurately and automatically classified. Authors in [15] proposed a way to detect and categorize the defects on road pavement surfaces automatically by using an anisotropy measure. Authors in [16] designed a 3stage method for crack investigation through high-contrast images. The method is based on the detection of cracks in the pavement through the evaluation of curves and the mathematical formation in the presence of a percentage of noise in the processed image. It was depended on the mathematical morphology and evaluation of curvature that detects the crack-like patterns in a noisy environment. Authors in [17] evaluated the achievement of 6 frequently applied segmentation methods. According to their findings, the dynamic optimization-based method surpasses the other algorithms. Authors in [18] state that the goal of incorporating the Shuffled Frog-Leaping Algorithm (SFLA) into the Electromagnetism-like Mechanism (EM) is to improve pavement crack properties in lighting invariant images. Authors in [19] created a pavement distress detection method that helps reduce noise and sharpen the linear characteristics of raw images. Authors in [20] utilized wavelet transform on image processing in order to automatically locate and highlight cracks. The suggested method has restrictions because it is hard to detect cracks due to sharp surface consistencies that create noise in the last image. Because crack pixels have more darkness than the surrounding pavement pixels, some thresholding approach has been used by many studies to locate cracks on the pavement.

II. STUDY OBJECTIVES
The essential goals of this study are: • To develop new algorithms for automatic detection and classifications of pavement cracks and potholes in images and videos.
• To compare the manual method (visual survey) and the automated method.
III. METHODOLOGY With the high-performance optical sensor technology and the increasing number of algorithms that rely on computer vision, the development of civil engineering-related applications increased. Most of these applications are dependent on image and video processing techniques by emphasizing the specific properties of the image and increasing the probability of correct detection [21]. Two python codes were used in this study. The first one, which dealt with images, attempted to categorize the types of 2D pavement distress and assess their severity. The second, which dealt with videos, attempted to evaluate sections of road pavement. A. Image Possessing Code (IPC)

1) Image Capturing and Reading
Images obtained from the road were used to detect cracks (horizontal, vertical, and alligator) and potholes and to assess their severity. The images were captured from a GoPro 8 hero camera that was positioned with an angle of 90 o and a height of 1.10m from the pavement surface as shown in Figures 2 and 3. A captured image must be in jpg format for the code to understand it. The reading of the image is an essential phase of the computer vision flow chart. It enables the framework to get the information. The angle between the camera and the surface of the pavement.

2) Apply Blur
Step 2 is used to reduce noise by Gaussian blur, which is the most commonly used smoothing technique to eliminate noises in images. In this technique, an image should be convolved with a Gaussian kernel to produce the smoothed image. It can be considered as a nonuniform low-pass filter that preserves low spatial frequency [22].

www.etasr.com
Abbas & Ismael: Automated Pavement Distress Detection Using Image Processing Techniques

3) Covert to Grayscale and Resize
For many image processing applications, color information doesn't help in identifying important edges or other features. So, the image will be converted from RGB to grayscale to neutralize colors. Also the image is resized to 200×200 pixels.

4) Apply Thresholding
It is a sort of image segmentation in which the pixels of an image are changed to make the image easier to analyze. Thresholding is a famous technique used in image processing algorithms to extract features. Thresholding is most commonly used to identify regions of interest in an image while disregarding areas of no concern [23]. For a given image, this work involves creating a histogram of grayscale values to find the vertices in the image. A threshold is then chosen according to the restricted area confined between two peaks. The failure areas of the pavement within the captured image are often marked by abrupt changes in the grayscale level of adjacent regions.

5) Apply Canny (Edge Detection)
Canny edge detection sets the edges of each pixel in the image to determine the color value after converting the image from grayscale to binary, ensuring that there are only two colors (black and white) in the image. The Canny edge detection algorithm process consists of 4 steps: • Apply a Gaussian filter to smooth the image and remove the noise because edge detection is prone to noise. A a 5×5 Gaussian filter is used to remove the noise from the image.
• Find the intensity gradients in the image: The image is then smoothed and filtered in both horizontal and vertical dimensions with a Sobel kernel to have the initial derivative in horizontal and vertical direction, G x and G y . The edge gradient and direction for each pixel are calculated from: The direction of the gradient is always perpendicular to the edges. It is rounded to one of these different angles: vertical, horizontal, and the two diagonal directions [24].
• Non-maximum suppression: after the identification of the extent and the direction of the gradient, the image is completely inspected to discard any unwanted pixels. Every pixel is assessed to look if it is a regional maximum in its neighbourhood in the gradient's direction [24,25].
• Hysteresis Thresholding: this step decides if the edges are real or not. This requires two thresholding values, minVal and maxVal. Pixels with intensity gradient of the edges greater than maxVal are guaranteed to be edges, while those with an intensity gradient less than minVal are guaranteed to be non-edges and are therefore eliminated. The ones located between the two criteria are sorted as edges or nonedges according to their connection. If they are linked to "sure-edge" pixels, they are regarded to be edges. If not, they will be removed as well [24].

6) Dilate Filter Application
A dilate filter is applied to fill the empty holes. A 5×5 kernel size is used. Dilation is applied to the binary images. The primary action of dilation on a binary image is that the boundaries of regions of foreground pixels are continuously increased (for example, white pixels). As a result, foreground pixel areas grow in size, while gaps within those regions shrink. Image dilation is used over erosion because erosion contracts the object. Since the noise is gone, it won't return, but the object area increases. It is useful in connecting the broken parts of an object back together.

7) Image Processing
The processing on the images consists of the following steps: • Detect all contours in the image (finding all objects): Contours can be defined as a curve linking all continuous points (along the border) of matching color or intensity. The contours are important tools for object recognition and shape evaluation [26]. A find-contours function retrieves all the contours in the image that it can detect. Contours can appear in an image in a variety of ways. Some might be grouped in other contours etc. This technique is used to simplify locating the contours of interest and to understand the hierarchy in which the contours are nested [26].
• Calculate contour area: Image moments assist in calculating some properties usually selected to have attractive features. It is used to define objects and to detect uncomplicated features of the image like area, intensity, centroid, orientation [26], etc. Image moment can be defined as a weighted average (moment) of the image pixels' intensities • Filter areas larger than the selected minimum crack area: Minimum crack size can be variant using the software UI, and changes will be seen in real-time for crack detection.
• Calculate arc length: arc length is used to calculate the perimeter of the contour. This is used to know more about the crack length and later to decide its type.
• Get the arc borders edges that approach a curve or a polygon with another curve such that the distance between them is less or equal to the particular accuracy, then calculate the number of curves/polygon endpoints to know how many endpoints are in the crack. For example, longitudinal and transverse have two endpoints and the alligator type has a minimum of 5 endpoints.

www.etasr.com Abbas & Ismael: Automated Pavement Distress Detection Using Image Processing Techniques
• Get the arc type, based on these conditions: If width < 100 and h > w × 2, it is a longitudinal.

If w > h × 2 and h < 100, it is a transverse.
If the endpoints of the edge are larger than 5 and the shape is adequate, it is an alligator crack.
Else, it is a pothole.
• Draw contours on the screen.
• Draw the arc path to detect crack areas and their width.
• Get the severity based on user-entered values for low, medium, and high severity.
• Draw result texts and border around the crack.

8) Show the Results
An example can be seen in Figure 4.

1) Import Video
The video is imported from the computer as an MP4 type. Video reader instructions are used by the code to read the movie.

2) Calculate the Total Number of Frames
The total number of frames in the movie is calculated by using a high-speed (240 fps) camera.

3) Select the Frames that will Be Processed
For surveying speed of 80km/hr or 22m/s and w=3m, L=2.20m frame dimensions, 1 frame must be selected from every 22 frames. For example, if the period of video is 1s (240 frames), the code will select 11 frames (1,23,45,67,89,111,133,155,177,199,221). These numbers apply only to the type of camera Hero 8 Black because they are calculated based on the characteristics of this camera.

4) Change RGB Frames to Greyscale and then to Binary
This step is essential in enabling the code to work with only two colors.

6) Measure the Standard Deviation of Each Frame by
Utilizing STD Orders.

8) Filtering
To decrease the frame noise, the image should be filtered. In MPC, and after experimentation with many kinds of filters, the most useful filter that can be utilized is the Gaussian filter. This kind of filter is dealing with almost all noises in the frame.

9) Color Conversion
In order to differentiate undamaged pavement from the impairment area in the same image, the colors are converted to conclude the involved areas in white and black for the normal pavement.

10) Edge Detection
Edge detection is conducted by the Canny algorithm.

11) Total Edge Values
Total edge values are found for every column and every row in the frame.

12) Finding the Distress Kind
The values of total edges were compared to find the distress kind in the frame. The damage in the pavement and its severity are found by measuring the white to black ratio in the frame.

IV. RESULTS AND DISCUSSION
The traditional method of assessment of pavement condition is to walk or drive down the road and collect the data manually. This way of road pavement surface detection depends on the experience level of the employees and it is time-consuming, hazardous, and subjective. Therefore, an effort has been made to fully automate the data collection process. To examine the possibility of the program to identify the types of failures and their severity from different images, 50 failure images were obtained and processed. The program correctly identified 38 of them. In the remaining images, the program either failed to detect the crack, identified it incorrectly, or gave a wrong severity of the crack. So, the examined images have an accuracy percentage of about 76%. This medium accuracy may be caused by environmental factors like shadows, dust, or weather conditions. The accuracy percentage of the distress detection may be enhanced by taking clearer images and having criteria for proper capture of the images with a steady source of light.
The proposed Automated Evaluation Of Pavement (AEOP) method detects the defects of roads automatically and classifies the most common distresses (fatigue, transversal cracks, longitudinal cracks, and potholes) by using computer vision techniques and image processing. The obtained classification results contain distress name/class and severity level as demonstrated in Figures 7-10. The manual method of assessment was conducted by the specialized department that is responsible for the evaluation and maintenance of city roads. The engineers did the manual assessment by using the www.etasr.com Abbas & Ismael: Automated Pavement Distress Detection Using Image Processing Techniques Pavement Condition Index (PCI) to quantify the condition of the road based on a scale from 0 to 100. They measured the Pavement's integrity and surface condition and rated the sections as very poor, poor, fair, satisfactory, good, or excellent according to [27]. By comparing the automated with the manual survey for the 8 sections, the percentage of accuracy results of distress detection for this case study is 88.44%. In Figure 5, we can see that the largest differences between the two methods occurred in sections 7 and 6. The smallest difference between the two methods occurred in section 3. When the two methods' results were compared, as shown in Table I, the results are nearly similar with a small variation. The discrepancy in the results can be attributed to several factors, including the type of pavement evaluation method, or maybe due to other issues like environmental factors such as stains, lane marks of the road, adjacent vehicles, shadows of side road trees, or texture differences among different pavement surfaces that affect the results of the automated detection of the crack. Among the most crucial comparisons to make is the difference in the time survey between the two techniques. The survey committee estimated the time required for the field survey and provided a report on the pavement's efficiency. The differences between the manual method and the automated approach are shown in Table II. The time required to do a manual survey vs the AEOP approach is significantly different. The AEOP took less time than the manual method, as shown in Figure 6. The time needed to finish the manual assessment of the 8 sections of the highway by the engineers of the maintenance department of the mayoralty of Baghdad was about 23hr. and the time needed for finishing the automated way was about 3hr. The cost of surveying is also an important issue to be evaluated. The manual method requires a committee with at least 3 experienced road engineers or technicians who work in road maintenance, and the use of these specialists means an augmented cost that will be spent on the evaluation of the road section. In addition, the evaluation of all the road sections requires a long time, and increasing time also means increased cost of the assessment of the whole road. On the other hand, the evaluation of the performance of the road's pavement using the AEOP method doesn't need committees or many workers. It requires just one person who knows how to use the code and a vehicle supplied with a camera. V. CONCLUSIONS The presented results support the following conclusions: • It is useful to utilize the image processing approach for identifying pavement failures. The presented algorithm has a lot of promise in terms of automated crack and pothole detection, since it can provide quick, nearly precise, and cost-effective results, which is important in pavement management systems.
• The suggested study is an encouraging solution for identifying cracks and potholes in asphalt pavements and sorting their severity by automated survey. By comparing the automated with the manual survey, the percentage of accuracy results of distress detection for this case study is 88.44%.
• There is a very large difference in the time needed to conduct the two methods. The AEOP required about 87% less than the manual method.
• Also, the automated method is much safer because it is generally conducted at prevailing traffic, it is easy to operate, it does not require committees, just one person to do it, and so it costs less than the manual method.
• The automated technique can replace the traditional road measurements.
Improving the automated crack detection can be done by using a vehicle with more developed equipment like multiple sensors, or a lighted camera to overcome the shadowing problem. There is a need to develop the code through cooperation with artificial software experts who can add new algorithms and thresholds to the current code in order to increase its performance and make it more efficient in exploring more types of failures on paved roads.