Application of Principal Component Analysis to Identify the Inputs of the ANNs Models

Dimensionality reduction methods have been widely used to reduce the number of input parameters to develop ANN models (^{Yuce et al., 2014}). One of the classical dimensionality reduction methods is the principal component analysis (PCA), which transforms a set of observations of correlated variables into a set of principal components, which are linearly uncorrelated. Based on the identified principal components, the amount of total variance contributed by each component is assed to select a reduced number of principal components which cumulative variance is within predefined acceptable values. Once a reduced number of principal components is selected, the relative importance of each input parameter to a particular component is evaluated by using correlation coefficients that relate the reduced set of principal components and input parameters. The reduced number of input parameters is selected based on predefined thresholds of correlation coefficients.

To illustrate the use of PCA in the dimensionality reduction of the input neurons for the development of the ANN models, we use the data for inslab events for firm soil for places outside Mexico. To proceed with the calculation of the principal components, the correlation coefficient matrix between the input variables is given in Table 3.

The correlation coefficient matrix is then decomposed by using the singular value decomposition to calculate the amount of total variance contributed by each principal component. Table 4 summarizes the percentage of variance associated with each principal component and its corresponding eigenvalue.

It is noted that there are different criteria to select the reduced number of principal components. According to ^{Lovric (2011)}, the reduced number of principal components can be selected when they account for a cumulative variance within 70 to 90%. A simpler criterion, which is adopted in this study, is to select those principal components whose eigenvalues are greater than one. Based on the afore-mentioned criterion, from Table 4, the reduced number of principal components is equal to 4.

By using the first 4 principal components, the correlation coefficients that relate the reduced set of principal components and input parameters is presented in Table 5.

It is observed in Table 5 that there are considerable variation in the correlation coefficients. To identify the reduced input parameters, we use two thresholds of correlation coefficients (^{Moore et al., 2013}), the first one equals 0.7 and the second one equals 0.55. It is noted that strong relation between variables is associated with correlation coefficients greater than 0.7, while moderate relation is associated with correlation coefficients greater than 0.55. Table 6 presents the reduced number of input parameters when the correlation thresholds are adopted. Also in Table 6, the reduced number of input parameters for the rest of the database are summarized. It is observed from Table 6 that the number of input parameters with strong correlation for interplate events ranges from 2 to 3, while for inslab it varies from 2 to 6. For the input parameters with strong correlation, the greater number of input parameters are related to the SS-MC case. If moderate correlation is considered, the number of input parameters is within 6 to 7 and 5 to 7 for interplate and inslab events, respectively. Based on the PCA results, ANN models are developed by using the identified input parameters based on the type of relation (i.e., strong and moderate relation). Furthermore, for the sake of comparison, ANN models by using the complete set of input parameters, referred to as ‘all inputs’, was included. One more ANN model with predefined input parameters is considered, this model includes Mw, natural logarithmic of R_c, H and T as input neurons and is referred to as ‘original case’. This last ANN model is considered as much of the information available in applications of engineering include those parameters. It should be pointed out that all the ANN models employ the natural logarithmic of R_C instead of the R_C. The output layer of the ANN models consists of a single neuron that represents the natural logarithmic of the SGMD for a considered earthquake and soil type.

The activation functions as well as the setups used during the training process are presented in Table 7. Some authors have suggested rules to identify the number of hidden neurons and their relation with the hidden layers (Masters, 1993; Swingler, 1996; Berry and Linoff, 1997), yet no closed-form expression is available to indicate the total number of hidden layers and neurons, a trial and error scheme is adopted to determine the structure of the ANN model (^{Shahin et al., 2004}).

During the trial and error process, ANN models with one and two hidden layers with 3 and up to 50 hidden neurons in each hidden layer were considered. The former was carried out to avoid potential overfitting of the ANN model.

It is noted that different metrics as the mean square error (MSE) or the mean absolute error (MAE) are available to evaluate the performance of the ANN models, each metric has advantages and disadvantages depending of the problem at hand. According to ^{Twomey and Smith (1995)}, there is no consensus as to which measure should be reported, and thus, comparisons among techniques and results of different researchers are practically impossible. In this study we adopted the MSE to evaluate the performance of the ANN models.

To evaluate the impact of the number of hidden neurons and layers on the trained ANN models by using both: the samples used for training and those not employed for training, the average MSE based on 300 trials (^{Pozos-Estrada et al., 2014}), is presented in Figure 5. It is observed in Figure 5 that, in general, when the data used for training is employed, the average MSE decreases as the number of hidden neurons increases. It is also observed from Figure 5 that the average MSE when the samples used for testing are employed is greater than that observed when the samples used for training are considered. This can be explained by noting that the trained ANN models are tested with input parameters that are different than those used during the training process. This trend is observed for the ANN models with different number of input neurons (i.e., all inputs, strong relation, moderate relation and original case) with one and two hidden layers, irrespective of the type of earthquake or soil considered. It is also observed from Figure 5 that the lowest average MSE associated with the testing stage is obtained for ANN models with 3 to 50 hidden neurons, and that the best ANN models for interplate and inslab are associated with 1HL and 2HL, respectively. Based on these observations, the parameters of the optimum ANN models (those associated with the minimum average MSE) are summarized in Table 8. It is observed in Table 8 that the number of hidden layers for the cases with different number of input neurons (i.e., all inputs, strong relation, moderate relation and original case) for interplate events ranges from 3 to 30, while that for inslab ranges from 3 to 50. This observation indicates that the selection of the inputs neurons is of paramount importance. Based on the identified optimum models with different number of input neurons, a further selection was carried out to employ a single ANN model (i.e., a single ANN model among all inputs, strong relation, moderate relation and original case) to predict the SGMD for SS-MC, FS-MC and FS-M. The criterion used to identify a single ANN model was based on selecting the ANN model which exhibits the best prediction behavior under several case scenarios. The ANN models selected to predict the SGMD for SS-MC, FS-MC and FS-M are indicated in Table 8 in italics. The obtained weights and biases for the selected trained ANN models for interplate and inslab, summarized in Table 8, are presented in Appendix A.

Figure 5 Average MSE for the developed ANN models by using the samples employed for training (in blue) and testing (in gray). Interplate events with one hidden layer: (a) Mexico City soft soil, (b) Mexico City firm soil, (c) Outside Mexico City firm soil. Interplate events with two hidden layers: (d) Mexico City soft soil, (e) Mexico City firm soil, (f) Outside Mexico City firm soil. Inslab events with one hidden layer: (g) Mexico City soft soil, (h) Mexico City firm soil, (i) Outside Mexico City firm soil. Inslab events with two hidden layers: (j) Mexico City soft soil, (k) Mexico City firm soil, (i) Outside Mexico City firm soil. 

Comparison Between the Observed and the Predicted SGMD by Using Trained ANN

Figure 6 presents a comparison between the observed and the predicted SGMD by using the trained ANN models with the datasets used for training and testing to those obtained from the actual records. It is observed from Figure 6 that there is good agreement between the observed and predicted values in most of the cases, with a correlation coefficient, ρ, ranging from 0.50 to 0.92 when the dataset used for testing is considered, and from 0.74 to 0.96 when the dataset used for training is employed. It is also observed from Figure 6 that the highest ρ values are associated the ANN models for inslab.

Figure 6 Comparison of the predicted SGMD with the trained ANN models and the observed values. Interplate events: (a) Mexico City soft soil, (b) Mexico City firm soil, (c) Outside Mexico City hard soil. Inslab events: (d) Mexico City soft soil, (e) Mexico City firm soil, (f) Outside Mexico City firm soil. 

Among all the ANN models developed, those that predict the SGMD for sites outside Mexico City present the highest values for interplate and inslab events. Further, the ANN model developed for soft soil in Mexico City for interplate events is similar to the ANN model for inslab events. On the other hand, the ANN model developed for hard soil in Mexico City for inslab events showed better predicting results that the ANN model for interplate events.

Comparison of the Predicted SGMD Using Trained ANN and Empirical Equations

Empirical Models to Estimate SGMD for Interplate and Inslab Events

The functional form of the empirical equation adopted in this study for soft and hard soils is the one proposed in ^{Reinoso and Ordaz (2001)}, which is shown in Table 9, where D denotes the SGMD, in s, M_w is the moment of magnitude, R_C is the closest distance to the fault, in km, and T is the soil period, in s. In this study the model coefficients were estimated using a least-squares regression algorithm, implemented in ^{MATLAB (2019)}. It is noted that that more accurate regression methods are available in the literature (^{Joyner and Boore, 1993}; ^{Boore et al., 1997}; ^{Bommer et al., 2010}) and that the use of the traditional least-squares method is an oversimplification; however, for simplicity, and since the main focus of this work is on the development and discussion of the ANN models to predict the SGMD, we adopted the traditional least-squares method.

Table 9 Coefficients of the empirical equation. 

	D = c 1 exp(M w ) + (c 2 M w + c 3 )R C + (c 4 M w + c 5 )(T + c 6 ) + ε
		Interplate events
Case	c 1	c 2	c 3	c 4	c 5
SS-MC	0.0237	-0.0212	0.3063	6.345	-25.013
FS-MC	0.0332	0.0035	0.1528	-	-
FS-M	0.0160	-0.0090	0.2361	-	-
		Inslab events
Case	c 1	c 2	c 3	c 4	c 5
SS-MC	0.0684	-0.0852	0.6722	-2.6447	38.11
FS-MC	0.0501	-0.0931	0.764	-	-
FS-M	0.027	-0.0233	0.3278	-	-

Notes: c _i , _i= 1,2,3,4,5, are regression coefficients; c ₆ = 0.5; T is set equal to 0.5 s for firm soils (^{Reinoso E. Ordaz M (2001)}, ^{NTC-SISMO (2017)}); ε is the error term.

The estimated coefficients for each type of earthquake and soil are summarized in Table 9. Figure 7 presents a comparison of the predicted SGMD with the fitted empirical models and the observed values for interplate and inslab events for soft and firm soils. Similar conclusions to those draw from Figure 6 are applicable to Figure 7, except that the correlation coefficients between the observed and predicted values are in general slightly smaller than those obtained when the ANN models are used. This difference is related to the number of data used to fit the empirical models, which is greater than that employed to test the ANN models.

Figure 7 Comparison of the predicted SGMD with the fitted empirical models and the observed values. Interplate events: (a) Mexico City soft soil, (b) Mexico City firm soil, (c) Outside Mexico City firm soil. Inslab events: (d) Mexico City soft soil, (e) Mexico City firm soil, (f) Outside Mexico City firm soil. 

Comparison Between Predicted SGMD Using Trained ANN and Empirical Equations

Figure 8 presents a comparison of the variation of the SGMD predicted with the trained ANN models and that obtained with the empirical equations as a function of M_w and modal values of R_c, T, H, M₀, φ, δ and λ for soft soil and modal values of Rc, H, M0, φ, δ and λ and T = 0.5 (s) for firm soil. It is noted that the use of H, M₀, φ, δ and λ and are input parameters of the trained ANN models; however, they were not considered as input parameter in the empirical expressions. In Figure 8, the empirical model developed in ^{Reinoso and Ordaz (2001)}, is also included for comparison purposes. It is observed from Figure 8, that in general, the predicted values by using the ANN models follow those predicted by the developed empirical equations. It is also observed that, for interplate events, the SGMD predicted with the trained ANN models oscillates within those predicted with the empirical equations. It is further observed that, for inslab events, the SGMD predicted with the trained ANN models follows the trend of the SGMD predicted with the empirical equation developed in this study. For case 8c that corresponds to the SGMD for inslab events for firm soil for places outside Mexico, the SGMD predicted by using the ANN model presents a bump for M_W within 5.5 and 6.5. The rest of the cases showed a smoother behavior. In practically all the cases presented in Figure 8, the model proposed in ^{Reinoso and Ordaz (2001)}, predicted smaller SGMD values with respect to those predicted by the trained ANN models and the empirical equations developed in this study. The differences between the SGMD values predicted by the ANN models and those with the empirical equation given in ^{Reinoso and Ordaz (2001)}, was also observed in the study by ^{Alcántara et al. (2014)}.

Figure 8 Comparison of SGMD predicted by the trained ANN models and by the empirical models with variation in Mw. Interplate events: (a) Mexico City soft soil, (b) Mexico City firm soil, (c) Outside Mexico City firm soil. Inslab events: (d) Mexico City soft soil, (e) Mexico City firm soil, (f) Outside Mexico City firm soil. 

Figure 9 presents a comparison of the variation of the SGMD predicted with the trained ANN models and that obtained with the empirical equations as a function of R_C and selected values of Mw and modal values of T, H, M ₀, , and . Similar conclusions to those drawn for Figure 8 are applicable to Figure 9, except that a better behavior of the predictions made with the ANN models are observed for both inteplate and inslab events. It is noted that the ANN models used to predict the SGMD for Figure 8, were also employed for Figure 9; however, it was observed that better predictions could be obtained if ANN models with different number of neurons were used. The ANN models employed in Figure 9 are also reported in Table 8. The previous observation indicates that the trained ANN models for the considered records are not very robust because the trained models with almost identical mean square errors do not always lead to the same predicted SGMD.

Figure 9 Comparison of SGMD predicted by the trained ANN models and by the empirical models with variation in R_C. Interplate events: (a) Mexico City soft soil, (b) Mexico City firm soil, (c) Outside Mexico City firm soil. Inslab events: (d) Mexico City soft soil, (e) Mexico City firm soil, (f) Outside Mexico City firm soil. 

To further compare the trained ANN models and the empirical equations, a probabilistic characterization of the error, defined as the difference of the logarithmic of the predicted values of SGMD by using the trained ANN models or the empirical equations for the results presented in Figure 8, and the logarithmic of the observed values is carried out. Figure 10 presents a comparison of the calculated errors in Normal probability paper. It is observed from Figure 10 that the error could be modeled as a normal variate. This was also verified with the Kolmogorov-Smirnov goodness-of-fit test (^{Benjamin and Cornell, 1970}), which indicates that the normality hypothesis could not be rejected at a significance level of at least 1% for both type of seismic events. The mean and standard deviation of the errors presented in Figure 10 are summarized in Table 10. It is observed from Table 10 that the trained ANN models and the empirical equations are only slightly biased and that the statistics of the error for the developed ANN models are similar to those of the empirical equations. Similar results were observed for the ANN models and the empirical equations presented in Figure 9 and for that reason they are not shown.

Figure 10 Normal probability paper of the error.