2.1 Flight planning and execution

The aircraft used to carry out the flight was the Phantom 4 Advanced model from the manufacturer DJI (Dà-Jiāng Innovations), with a battery life of 30 minutes. The camera attached to the aircraft has a lens with an 84º field of view and a focal length of 8.8 mm/24 mm, equipped with a 1” 20 Megapixel CMOS (Complementary Metal Oxide Semiconductor) sensor and RGB (Red, Red, Green, and Blue) (DJI Brazil, 2017). For aerial mapping, the study considered the following factors: sunny weather and appropriate light incidence without clouds; an ideal flight window from 11 a.m. to 1 p.m.; favorable wind speed and direction; landing and take-off locations selected according to local obstacles; and non-interference in airspace, as requested by the National Civil Aviation Agency (ANAC).

The evaluated stretch has a length of 600 m, with two segments perpendicular to each other, located at Rua Irineu Parzianello, municipality of Pato Branco, state of Paraná, Brazil, with initial coordinates 26º11'55.14”S and 52º41'19.15''W, and end 26º12'4.92”S and 52º41'34.75”W. The stretch had several surface pathologies, allowing a comprehensive analysis of the percentage correctness of pathology and non-pathology identification methods. The presence of vegetation bordering the road allowed the evaluation of elements in shade.The flight plan complied with the following parameters: lateral coverage of 65%, 75% longitudinal coverage, and a maximum speed of 15 m/s with a GSD (Ground Sample Distance) of 1.5 cm/px, resulting in an average altitude of 50 meters. The approximate flight time was 7 minutes. Ten control points (Ground Control Points, GCPs), determined along the chosen stretch randomly and interspersed between the sides of the track, optimize the image processing adjustment. The authors posted targets and their respective coordinates through the GNSS receiver (Global Navigation Satellite System), model Zenith 25, and the RTK (Real Time Kinematic) method. The processing to create the orthomosaic was carried out in the Agisoft PhotoScan program, used to reference the photos, correct positions to the GNSS coordinates, and carry out the orthorectification to obtain the final orthomosaic used in the manual and supervised classifications.

2.2 Manual image classification

From the orthomosaic, AutoCAD® 2018 software vectorized each element to manually classify the pathologies (Fig. 1).

Figure 1 Example of manual delimitation of pathology areas. 

To obtain reference data for the survey carried out with the RPA, pathologies were evaluated by path, identifying the type and extent of the pathologies to verify whether the existing pathologies on the road would be visible in the images obtained by the RPA.

2.3 Supervised classification of images

A complement to the QGIS platform (Quantum Geographic Information System), the SCP (Semi-Automatic Classification Plugin) allowed the application of the supervised classification of images. From the final orthoimage, the plugin created a set of bands equivalent to the pre-processing step. The first processing stage includes the following categories: asphalt pavement in good condition, cracks, potholes, patches, vegetation (overlapping the road), and areas with clayey material. Next, samples of each element of interest to the classification were selected, which may have a pixel or a polygon as a capture instrument. These samples feed the training file, obtaining a raster image as a result of the classification.

In order to estimate the most appropriate supervised classification method for the machine used, the tests included combinations of variables associated with the identification criteria per pixel or polygon of the classification algorithm (minimum distance, maximum distance, maximum likelihood, or spectral angle mapping) and the number of samples for machine learning.

Four samples of each element were randomly collected for the first test, eight for the second, and sixteen for the third, totaling 24, 48, and 96 samples, respectively. This stage allowed a choice of samples consistent with the machine's capacity. Using the SCP, samples were collected by pixel, where the area with pixels similar to the selected one is delimited, and by polygon, where the area is delimited manually (Fig. 2).

Figure 2 Examples of sampling by polygon (a) and by pixel (b) 

As for image classification algorithms, SCP provides three options: (1) Minimum Distance, which classifies pixels by the average spectral distance between categories, by Euclidean distance, according to Eq. (1) (^{Richards, 2013}):

dx,y=∑i=1n(xi-yi)² (1)

Where dx,y is the Euclidean Distance, xi : Spectral signature vector of the pixel image, yi : Spectral signature vector of the area of a sample, n: Number of bands in the image.

(2) Maximum Likelihood, which calculates the normal probability distribution of the class, according to Eq. (2) (^{Richards, 2013}):

gkx=ln⁡p(Ck) -  12ln⁡Σk -  12x-ykt Σk-1x-yk (2)

Where gkx is the Normal Probability Distribution of the class, 𝑥: Spectral signature vector of the pixel image, Ck : Land cover class k, p(Ck) : Probability that the correct class is Ck , | Σk |: Determinant of the covariance matrix of data in class Ck , Σk-1 : Inverse of the covariance matrix, yk : Spectral signature vector of class k.

(3) Spectral Angle Mapping, which determines the degree of similarity between spectral curves, according to Eq. (3) (^{Kruse et al., 1993}):

θx,y= cos-1⁡∑i=1nxiyi(∑i=1nxi2)12*(∑i=1nyi2)12 (3)

Where θx,y is the spectral angle, xi : Spectral signature vector of the pixel image, yi : Spectral signature vector of the area of a sample, n: Number of bands in the image.

The several combinations of identification criteria, classification algorithms, and number of samples allowed us to choose the best ones for this study. The results with the highest percentage of correct answers regarding the type and extent of pathologies were obtained with the Maximum Likelihood classifier using the polygon method with 24 and 48 samples. This study discarded tests with more samples due to the machine's capacity and the program's operating mode, which, during processing, continually overload the machine's memory.

2.4 Comparative and statistical analysis

After comparing the manual classification of images with data from the walking survey, manual classification was considered a reference for supervised classifications due to the high conformity of manual classification with pathologies in the field.

The Kappa index, a multivariate statistical technique, allowed for the assessment of the classifier's accuracy in identifying pathologies. The study compared the supervised classification with the samples used for training the algorithm and the supervised classification with the manual one. The Kappa indices obtained results from the Error Matrix (or confusion) provided in QGIS post-processing and the Manual versus Supervised Success Matrix, respectively. In addition to this, another parameter used for class analysis was the Matthews Correlation Coefficient (MCC).

2.4.1 Comparison of supervised classification successes with manual classification

With the aid of QGIS, the polygons obtained in the manual classification were superimposed on the polygons from the supervised classification, resulting from transforming of the raster image into polygons. Thus, the study obtained common area of both classifications for each element. To verify the percentage of correct pathology identification, the manual classification results were confronted with those of the supervised classification from 24 and 48 samples. As an example, Figure 3 shows the same section manually classified (a) and supervised, with 24 samples (b) and 48 samples (c).

Figure 3 (a) Manual, (b) Supervised with 24 samples and (c) Supervised with 48 samples. 

3.1 Flight and processing

After performing the flight and processing the images obtained in the RPA survey, a final orthomosaic was generated (Fig. 4). The GSD was 1.39 cm per pixel. To optimize the classification, we cut out of the image the excess edges that contained the surroundings beyond the limits of the track lane.

Figure 4 Final orthoimage. 

3.2 Manual image classification

Manual classification of pavement surface pathologies was relatively quick due to the ease of identification from orthoimaging. On the other hand, after a subsequent check with the survey by walking, it was evident that the shadows of trees adjacent to the road made it difficult to visualize some pathologies and their extent, mainly cracks. In addition, vegetation hid parts of the pavement.

RPA images did not identify five pathologies noted in the walking survey. The area of patches, potholes, and cracks observed in the collection by walking was 1,759.45 m², while in the manual survey, it was 1,638.42 m², resulting in the identification of 93.1% of pathologies. The percentage was calculated based on geometric measurements recorded in sketches, with scale, in the walking survey.

3.3 Supervised classification of images

The classification of samples by polygons showed few regions in agreement with the actual surface condition of the pavement, as shown in Fig. 5. In addition, it resulted in different classifications for most of the selected image elements.

Figure 5 (a) Pavement, (b) classification by polygons with 24 samples and (c) 48 samples. 

Pixels from different categories, such as good pavement and patches, presented similar shades in some images, resulting in similar spectral values and, consequently, in classification errors. The results found that some errors occurred due to shading and overlapping vegetation, making it impossible to classify these locations.

3.4 Comparative and statistical analysis

In this topic, we present confusion matrices and scatter plots generated from the confusion matrix resulting from the relationship between manual and supervised classifications. These results allowed calculating the Kappa index (K) and the Matthews Correlation Coefficient (MCC).

3.4.2 Confusion matrices and scatterplots

Tabs 3 and 4 represent classification error matrices. 'Px', 'Pav', and 'class' refer to Pixels, Pavement good, and Classifieds, respectively. Rows contain sorted data, while columns contain reference data. Internal values represent the pixels, and correctly identified pixels were highlighted.

Table 3 Classifier confusion matrix with 24 samples. 

	Pav.	Crack	Pothole	Patch	Vegetation	Soil	Px class
Pav.	48682	9749	148	726	430		59735
Cracking	3722	61794	123	667	1458		67764
Pothole	307	5885	489	222	1150	755	8808
Patch	9887	13114	24	12162	182		35369
Vegetation	34	97		39	62564	207	62941
Soil		19	3		7	61161	61190
Px reais	62632	90658	787	13816	65791	62123	295807

Table 4 Classifier confusion matrix with 48 samples. 

	Pav.	Crack	Pothole	Patch	Vegetation	Soil	Px class
Pav.	190051	12879	450	1130	551	11	205072
Cracking	18926	55103	785	1757	299		76870
Pothole	2367	4299	2354	1183	312	89	10604
Patch	9549	5383	229	12819	21		28001
Vegetation	402	88	15	2	35516	17	36040
Soil	309	12	44		26	6725	7116
Px reais	221604	77764	3877	16891	36725	6842	363703

The overlapping polygons defined in the manual and supervised classifications verified the correspondence of the identified pathologies. Figures 6 and 7 show the scatter plots of the percentage of pathologies identified by class for 24 samples (Figure 6) and 48 samples (Figure 7) derived from the confusion matrices for these cases.

Figure 6 Dispersion of assertiveness: supervised classification 24 samples x manual 

Figure 7 Dispersion of assertiveness: supervised classification 48 samples x manual 

Fig. 6 shows that the land and vegetation classes presented better results, with percentages of correct answers more significant than 60 and 70%, respectively. Good pavement, cracks, and patches presented values between 30 and 50%, understood as a bad result. The pans present the worst result, with a hit of approximately 25% and a higher number of areas erroneously classified as patches.

Fig. 7 shows that the percentages of correct answers were similar to those in Fig. 6a, but with a significant increase for the pans. The Maximum Likelihood algorithm applies maximum likelihood (MaxVer), which explains these results. According to ^{INPE (2006)}, the method weights the distance between the average values of the element's pixels through statistical parameters, assuming that all bands have a normal distribution, and calculates the likelihood that a pixel belongs to a specific element based on the samples. Thus, since the pot class presents different characteristics in the same occurrence, more samples contributed to a higher percentage of correct answers for this classifier.