nueva página del texto (beta)
Inglés (pdf)
Artículo en XML
Referencias del artículo
Traducción automática
Enviar artículo por email
Citado por SciELO 
Similares en
SciELO 
Permalink
1 Introduction
Breast cancer originates from the inner lining of milk ducts or the lobules responsible for supplying milk to the ducts. It manifests as a tumor within the breast, which can either be benign (non-cancerous) or malignant (cancerous).
Malignant tumors grow and develop into cancer. Breast cancer is a leading cause of mortality worldwide. Every year the breast cancer is detected in 1,3 million women on the world, the mortality rate increases rapidly with more than 1,6 million cases in 2010 which corresponds to 425000 deaths [6]. The correct diagnosis of breast cancer is very important to help the doctors.
Detecting cancerous cells at an early stage, before they spread, can significantly increase the survival rate for patients by over 97%fn Nevertheless, classifier systems are widely used to solve the problem of cancer classification and to help the experts to make a good diagnosis. The major advantages of classifier systems are the minimization of possible errors that might be made and the ability to give a detailed examination.
The Support Vector Machine (SVM) is recognized as one of the most widely used classifier systems which has become quite an active research field in machine learning. Recently, Support Vector Machines have been shown to give good results and a good generalization performance in the medical diagnosis field precisely in cancer classification [19, 8, 11, 3, 17].
Achieving strong performance with the SVM method heavily relies on the selection of appropriate kernel functions. These functions enable the algorithm to identify the maximum-margin hyperplane within a transformed feature space.
Additionally, the effectiveness of the SVM hinges on the careful tuning of kernel function parameters and the cost parameter
For instance, Quinlan achieved a classification accuracy rate of 94,74% through
Nauck and Kruse obtained 95,06% classification accuracy rate by using the neuron-fuzzy techniques [13]. Albrecht et al. reached 98,80% classification accuracy rate with logarithmic simulated annealing with the perceptron algorithm [2].
Übeyli, by using SVM reached 99,54% accuracy [21]. Polat and Günes used the LS-SVM (Lest Square SVM) and 98,53% was obtained [15]. Guijarro-Berdias et al. achieved 96,00% classification accuracy rate by applying linear-lest squares [7].
Akay, by using SVM with feauter selection, reached 99,51% classification accuracy rate [1]. Marcano-Cedeño et al., by applying artificial metaplasticity neural network reached 99.26% classification accuracy rate [12]. In this paper, we compare and analyze several kernel functions proposed in the literature by using different values of kernel functions parameters and cost parameters.
This study has been applied to the Wisconsin Breast Cancer Dataset (WBCD) which is a widely studied data set from the field of breast cancer diagnosis. We evaluate the results by calculating the performance measures: classification accuracy rate, sensitivity, specificity, positive and negative predictive values.
In this work we have done two studies: the first one is the realization of a comparison protocol between the different kernels and the second is to reach a high classification accuracy rate in the context of breast cancer diagnosis.
Also, we show that the Cauchy kernel and the Rational Quadratic kernel are identical and give the same results.
The rest of the paper is organized as follows: First in Section 2, we give an overview of SVM. In Section 3, we recall some kernel functions defined in the literature. In Section 4, we analyze the results of the different kernel function. Finally, we conclude with some perspectives.
2 Overview of Support Vector Machine
Vapnik [4] introduced the Support Vector Machine (SVM) as a learning algorithm aimed at minimizing structural risk. SVM is a method used for data analysis and is employed in both classification and regression tasks.
Given a set of input data, SVM predicts which of two possible classes each input belongs to. To achieve classification, SVM constructs a hyperplane in a high-dimensional space to effectively separate the data into classes.
This hyperplane is positioned to maximize the distance to the nearest training point of any class, ensuring optimal separation. While multiple valid hyperplanes exist, SVM uniquely identifies the optimal hyperplane. The data points that are closest to this maximum margin hyperplane are referred to as Support Vectors.
Identifying this hyperplane involves reformulating the classification problem into a quadratic optimization task, which can be resolved using various algorithms such as Sequential Minimal Optimization, Trust Region, Interior Point, Active-Set, and others. One key benefit of SVM is its effectiveness in high-dimensional spaces, even when the number of dimensions exceeds the number of samples.
2.1 Mathematical Formulation
Given a training data set of
The hyperplane that optimally separates the data is the one that minimizes :
This gives the final standard formulation of an SVM as a minimization problem:
This represents a quadratic programming optimization challenge. Quadratic optimization problems are a widely recognized category of mathematical optimization problems, with numerous algorithms available for their resolution. The dual problem is obtained by introducing Lagrange musltipliers:
Solving equation (3) with constraints equation determines the lagrange multipliers, and the optimal separating hyperplane is given by:
where
The hard classifier is then:
In this study, we choose the Sequential Minimal Optimization algorithm to solve the quadratic problem.
2.2 Sequential Minimal Optimization
The Sequential Minimal Optimization (SMO), introduced by [10, 14] is another widely used algorithm for training Support Vector Machines (SVMs). The basic idea of smo is to decompose the initial problem into sub problems (reducing the working sets to two points).
the optimal solution can be computed analytically for this two points in the working set [5]. Given the current solution
where
where
Next we clip
The bounds
– If
–
Finally, having solved for
The algorithm proceeds as follows:
Find a Lagrange multiplier
Pick a second multiplier
Repeat steps
Upon satisfaction of the Karush-Kuhn-Tucker (KKT) conditions by all Lagrange multipliers within a specified user-defined tolerance, the problem is considered solved.
While this algorithm ensures convergence, heuristics are employed to select the pair of multipliers to expedite convergence. To achieve optimal performance, certain parameters in SVM must be meticulously chosen. These parameters include:
– The regularization parameter
– The parameters of the kernel functions.
– The choice of the kernel affect the performance.
3 Kernel Functions
However, in 1992, V. Vapnik et al. proposed a method to generate nonlinear classifiers by employing the kernel trick with maximum-margin hyperplanes.
Kernel functions enable a nonlinear transformation of data into a linear separation of examples in a new space known as the ”feature space,” which is high-dimensional.
This characteristic enhances the likelihood of discovering a separating hyperplane. Nevertheless, in this new space, the goal is to find the following hyperplane:
We arrive at the following optimization problem:
By introducing the notion of kernel function we have:
The expression of the hyperplane will be defined as follows:
Under these conditions we didn’t need to know the transformation
We can directly construct a kernel function by respecting certain condition defined by the theorem of Mercer which states that: a kernel
Unfortunately, this theoretical condition is difficult to verify, it does not provide guidance for the construction of kernels or on the transformation
Table 1 Some kernel functions defined in the literature (M. is multiquadric and q. is quadratic)
| Kernel name | Formulation |
| Linear | |
| Polynomial | |
| Gaussian |
|
| Sigmoid | |
| Cauchy | |
| Inverse M. | |
| Quadratic | |
| Multiquadric | |
| Power | |
| Rational Q. | |
| Wave | |
| Spherical |
4 Experimentation
In this research, we analyze and evaluate the performance of different kernel functions in the SVM algorithm by using different values of kernel functions parameters and SVM model parameters. We use the SMO algorithm to solve the quadratic problem of maximizing the margin.
This analysis is conducted using the publicly accessible breast cancer database known as WBCD (Wisconsin Breast Cancer Database), which originates from the work undertaken at the University of Wisconsin Hospital.
This set of data was taken from Fine Needle Aspirates (FNA) of humain breast tissue classified as benign and maligne. The WBCD database contains 699 clinical cases, there is 458 (65,50%) benign cases and 241 (34,50%) malignant cases.
The data of WBCD contains 16 instances with missing attirubte values which led us to limir our experimentation to 683 clinical cases. Nevertheless, the class has a distribution of 444 (65%) besnigs cases and 239 (35%) malignant cases. Each instance in the database has a nine attributes; each attributes has an integer value between 1 and 10.
In the following table, we detailed the attributes of WBCD: The SVM method consists of two phases: training and testing. To randomly divide the database into two parts, we employ the holdout method, a form of cross-validation.
This method randomly partitions the initial data into two sets: the training set and the testing set. Less than one-third of the initial data is allocated for testing purposes. With the holdout method, we obtained 455 samples(65,10%) for the training phase and 244 samples (34,90%) for the testing phase.
In the first step, we analyze and compare the results obtained by the different kernel functions in terms of accuracy classification rate. The values of kernel functions parameters were chosen by experimentation.
For the cost parameter
In the second step, we take the four kernel functions which have given a good results and we analyze these kernels in the function of: sensitivity, specificity, positive and negative predictive values. The parameters mesearse are calculated using the following equations:
where:
| Classification accuracy: | |
| Sensitivity: | |
| Specificity: | |
| Positive Predictive Value: | |
| Negative Predictive Value: |
In table 5, we present the good results (the high value of: classification accuracy rate and its kernel parameters) obtained by each kernel function after several trials and evaluation of different values of the kernel functions parameters with SVM model parameters. We clearly observe, that the Gaussian, Cauchy, inverse multiquadratic, rational quadratic, linear and polynomial kernels have given a good results with an advantage for Gaussian kernel function (99,13%).
Table 3 WBCD description of attributes
| Attribute numbers | Attribute description | Values of attribute |
| 1 | Clump thickness | 1-10 |
| 2 | Uniformity of cell size | 1-10 |
| 3 | Uniformity of cell shape | 1-10 |
| 4 | Marginal adhesion | 1-10 |
| 5 | Single epithelial cell size | 1-10 |
| 6 | Bare nuclei | 1-10 |
| 7 | Bland chromatin | 1-10 |
| 8 | Normal nucleoli | 1-10 |
| 9 | Mitoses | 1-10 |
Table 4 The results obtained by different kernel functions with the bestvalue of kernel parameter
| Kernel name | Results | Kernel Parameter |
| Linear | 97.40 | |
| Polynomial | 96.54 | |
| Gaussian | 99.13 | |
| Sigmoid | 96.53 |
|
| Cauchy | 98.27 | |
| Inverse Multiquadric | 98.27 | |
| Quadratic | 93.59 | |
| Multiquadric | 64.93 | |
| Power | 34.63 | |
| Rational Quadratic | 98.27 | |
| Wave | 97.83 | |
| Spherical | 64.93 |
Table 5 The results obtained by different kernel functions with the bestvalue of SVM model parameter
| Kernel name |
cost parameter |
| Linear | |
| Polynomial | |
| Gaussian | |
| Sigmoid | |
| Cauchy | |
| Inverse Multiquadric | |
| Quadratic | |
| Multiquadric | |
| Power | 2 |
| Rational Quadratic | |
| Wave | |
| Spherical |
We show also, that the Cauchy, inverse multiquadratic and rational quadratic kernel have nearly the same classification accuracy rate. The low classification accuracy rate is registered for the multiquadratic and spherical kernel function with 64,93%.
During the assessments, we show that the results obtained by the rational quadratic kernel and the Cauchy kernel are identical despite the different tests with different training and testing set.
Therefore, we will show that the rational quadratic kernel and the Cauchy kernel are identical and converge to the same value:
The Rational Quadratic kernel is defined as:
Finally, we can say that using the rational quadratic kernel function gives the same results as the Cauchy kernel function. So, the transformation of data by these kernel are the same and give the identical new space. In table 7, we show: sensitivity, specificity, positive and negative predictive values obtained by the four kernels which have given a high diagnostic accuracy.
Table 6 Sensitivity and specificity calculated for the four best kernel functions
| Kernel name | Sensitivity | Specificity |
| Gaussian | ||
| Inverse M. | ||
| Cauchy | ||
| Wave |
Table 7 Positive predictive value and negative predictive value calculated for the four best kernel functions
| Kernel name | Pos. Pre. Val. | Neg. Pre. Val. |
| Gaussian | ||
| Inverse M. | ||
| Cauchy | ||
| Wave |
Table 8, gives the classification accuracies of SVM with Gaussian kernel and previous methods applied to the same database.
Table 8 Classification accuracy rate (CAR) obtained with SVM by using the Gaussian kernel and other classifiers
| Author and years | CAR |
| Quinlan (1996) | |
| Hamiton et al. (1996) | |
| Nauck and Kruse (1999) | |
| Albrecht et al. (2002) | |
| Polat and Günes (2007) | |
| Guijarro-Berdias et al. (2007) | |
| Akay (2009) | |
| A. Marcano-Cedeño (2011) | |
| This Study |
5 Conclusion
In this study, we have analyze and compared the performance of several kernel on the support vector machine in the context of breast cancer diagnosis. We conducted our experimentation on the WBCD database. We have shown that the Gaussian kernel has given a good results in term of classification accuracy rate (99,13%). The Gaussian kernel have a great performance with: 98,67% in sensitivity and 100% in specificity. Also, we have shown that the both Cauchy kernel and the Rational Quadratic kernel are the same and give the same results.
