SVM Classifiers : The Objects Identification on the Base of Their Hyperspectral Features

The problem of the objects identification on the base of their hyperspectral features has been considered. It is offered to use the SVM classifiers on the base of the modified PSO algorithm, adapted to specifics of the problem of the objects identification on the base of their hyperspectral features. The results of the objects identification on the base of their hyperspectral features with using of the SVM classifiers have been presented.


Page layout
The problem of analysis of the hyperspectral information formed on the base of the hyperspectral images of the Earth's surface is one of the actual problems solved by the remote sensing systems.The russian "Resource-P" spacecrafts No. 1 -3 with the hyperspectral equipment on the board give out the hyperspectral image (HSI) in the form of pictures in 130 narrow adjoining ranges of visible area of spectrum.During the HSI processing much attention is paid to the questions of the objects identification on the base of their hyperspectral features (HSF).
The object's HSF in a graphic form can be presented as the relationship mapping between the wavelength and the values of the spectral brightness coefficient (SBC), or the spectral reflection coefficient (SRC) of the analyzed object.However, at the solution of the problem of the objects identification on the base of their HSF the use of the SRC dependence on wavelength is more preferable because the brightness feature doesn't depend on the shooting conditions in such degree as the SBC.Besides, unlike the SBC, for obtaining the SRC values the standard reflecting surface in sight of the analyzed object isn't necessary.
In this paper we suggest to use the SVM classifiers on the base of the modified PSO algorithm, adapted to specifics of the problem of the objects identification on the base of their hyperspectral features.Herewith, we test the different approaches to application of the SVM algorithm to development of the SVM classifier: the basic SVM classifier, the two-level classifier, the SVM ensemble.All of them apply the modified PSO algorithm to find the type of the kernel function, the values of the parameters of the kernel function and the value of the regularization parameter of the SVM classifier.Also, the SVM ensemble uses the principle of maximum decorrelation to choose the individual SVM classifier, which will be included into the SVM ensemble.
The SVM algorithm (Support Vector Machines, SVM) is successfully used for development of the SVM classifiers for a wide range of the classification problems [1].The SVM classifier uses the special kernel function to construct a hyperplane separating the classes of data.The SVM classifier is used for training and testing.The satisfactory quality of training and testing allows using the resulting SVM classifier in the classification of new objects.
Choosing the optimal parameters values for the SMV classifier is a significant problem at the moment.It is necessary to find the kernel function type, values of the kernel function parameters and the value of the regularization parameter [1,2].It is impossible to provide implementing of high-accuracy data classification with the use of the SVM classifier without adequate solution to this problem.In the simplest case solution to this problem can be found by a search of the kernel function types, the values of the kernel function parameters and the value of the regularization parameter that demands significant computational expenses.The gradient methods are not suitable for search of the optimum of this objective function, but search algorithms of stochastic optimization, such as the genetic algorithm, the artificial bee colony algorithm, the particle swarm algorithm, etc., have been used.Each of the optimal decision is carried out at once in all space of possible decisions.
The PSO algorithm, which is based on an idea of possibility to solve the optimization problems using modelling of animals' groups' behaviour is the simplest algorithm of evolutionary programming because for its implementation it is necessary to be able to determine only value of the optimized function.
The traditional approach to application of the PSO algorithm consists of the repeated applications of the PSO algorithm for the fixed type of the kernel function to choose the optimal values of the kernel function parameters and the value of the regularization parameter with the subsequent choice of the best type of the kernel function and the values of the kernel function parameters and the value of the regularization parameter corresponding to this kernel function type.
Here, we suggest to use the modified PSO algorithm to find the type of the kernel function, the values of the parameters of the kernel function and the value of the regularization parameter of the SVM classifier simultaneously., values of the kernel parameters and value of the regularization parameter C , which allows finding a compromise between maximizing of the gap separating the classes and minimizing of the total error [1][2][3][4].A herewith typically one of the following functions is used as the kernel function ) , ( W N z z i : linear function; polynomial function; radial basis function; sigmoid function [5,6].

Theoretical part
To build "the best" SVM classifier it is necessary to implement the numerous repeated training (for the training data set with S elements) and testing (for the test data set S s elements, s S ) on the different randomly generated training and test sets with following determination of the best SVM classifier in terms of the highest possible classification quality provision.The SVM classifier with satisfactory training and testing results can be used to classify new objects [6].
As a result of the training, the classification function is determined in the following form [1][2][3]

N O
The classification decision, associating the object z to the class −1 or +1, is adopted in accordance with the rule [1][2][3]: The training of the SVM classifier results in determining the support vectors [1][2][3][4].Using the modified PSO algorithm provides the better classification accuracy by choosing the kernel function type, the values of the kernel function parameters and the value of the regularization parameter.Also, the modified PSO algorithm allows reducing the time expenditures for development of the SVM classifier [5].The quality of the SVM classifier can be measured by different classification quality indicators.
In the traditional PSO algorithm the n -dimensional search space ( n is the number of parameters which are subject to optimization) is inhabited by a swarm of m agents-particles.The position of the i -th particle is determined by vector , which defines a set of values of optimization parameters.These parameters can be presented in an explicit form or even absent in the analytical record of the objective function ) (x u of the optimization algorithm Each i -th particle ( m i , 1 ) has its own vector of speed ) coordinates' values in every single moment of time corresponding to some iteration of the PSO algorithm.
The coordinates of the i -th particle ( m i , 1 ) in the n -dimensional search space uniquely determine the value of the objective function which is a certain solution of the optimization problem.For each position of the n -dimensional search space where the i -th particle ( m i , 1 ) was placed, the calculation of value of the objective function is performed.Each i -th particle remembers the best value of the objective function found personally as well as the coordinates of the position in the n -dimensional space corresponding to the value of the objective function.Moreover each i - th particle ( m i , 1 ) "knows" the best position among all positions that had been "explored" by particles.At each iteration particles correct their velocity to, on the one hand, move closer to the best position which was found by the particle independently and, on the other hand, to get closer to the position which is the best globally at the current moment.After a number of iterations particles must come close to the best position (globally the best for all iterations).However, it is possible that some particles will stay somewhere in the relatively good local optimum.
There are different approaches to implementation of velocity vector i v correction for the i -th particle ).In the classical version of the PSO algorithm correction of each j -th coordinate of velocity vector ) is made in accordance with formula [6]: where j i v is the j -th coordinate of velocity vector of the i -th particle; j i x is the j -th coordinate of vector i x , defining the position of the i -th particle; j i х ˆ is the j -th coordinate of the best position vector found by the i -th particle during its existence; j х ~ is the j -th coordinate of the globally best position within the particles swarm in which the objective function has the optimal value; rˆ and r ~ are random numbers in interval (0, 1); Mˆ and M ~ are personal and global coefficients for particle acceleration.
With personal and global acceleration coefficients in (3) random numbers rˆ and r ~must be scaled; the global acceleration coefficient M ~ operates by the impact of the global best position on the speeds of all particles and the personal acceleration coefficient Mˆ operates by the impact of the personal best position on the velocity of some particle.
Currently different versions of the traditional PSO algorithm are known.In one of the most known canonical version it is supposed to undertake the normalization of the acceleration coefficients Mˆ and M ~ to make the convergence of the algorithm not so much dependent on the choice of their values [6].
The correction of each j -th coordinate of the velocity vector ( ) is performed in accordance with formula [6]: where F is a compression ratio; K is some scaling coefficient, which takes values from the interval (0, 1).
Let the correction of velocity vector of the i -th particle ( m i , 1 ) is executed in accordance with one of the formulas (3) or (4).The correction of the j -th coordinate of the i -th particle ( m i , 1 ) can be executed in accordance with the formula: Then for each i -th particle ( m i , 1 ) the new value of the objective function can be calculated and the following check must be performed: whether a new position with coordinates vector i x became the best among all positions in which the i -th particle has previously been placed.If new position of the i -th particle is recognized to be the best at the current moment the information about it must be stored in a vector i x ˆ.
The value of the objective function for this position must be remembered.Then among all new positions of the swarm particles the check of the globally best position must be carried out.If some new position is recognized as the best globally at the current moment, the information about it must be stored in vector x ~.
Value of the objective function for this position must be remembered.In the case of the SVM classifier development with the use of the PSO algorithm the swarm particles can be defined by vectors declaring their position in the search space and corded by the kernel function parameters and the regularization parameter: ) , , ( , where i is a number of particle ( m i , 1 ); to the kernel function parameter 1 k , if the swamp particle corresponds to the sigmoid type of the kernel function, otherwise this parameter is assumed to be zero); i C is the regularization parameter [5, 6].As a result, for each type T of the kernel function, participating in the search, the particle with the optimal combination of the parameters values ) , , ~ ( providing high quality of classification will be defined [6].The best type and the best values of the required parameters are found using the results of the comparative analysis of the best particles received at realization of the PSO algorithm with the fixed kernel function type. Along with the traditional approach to the application of the PSO algorithm in the development of the SVM classifier there is a new approach that implements a simultaneous search for the best kernel function type T ~, parameters' values 1 x and 2 x of the kernel function and value of the regularization parameter C ~ [6].At such approach each i -th particle in a swamp ( m i , 1 ) defined by a vector which describes particle's position in the search space: ) , , , ( , where i T is the number of the kernel function type (for example, 1, 2, 3 -for polynomial, radial basis and sigmoid functions accordingly); parameters , C are defined as in the previous case.It is possible to "regenerate" particle through changing its coordinate i T on number of that kernel function type, for which particles show the highest quality of classification.In the case of particles' "regeneration" the parameters' values change so that they corresponded to new type of the kernel function.Particles which didn't undergo "regeneration", carry out the movement in own space of search of some dimension.The number of particles taking part in "regeneration" must be determined before start of algorithm.This number must be equal to 15% -25% of the initial number of particles.
The offered modified PSO algorithm can be presented by the following consequence of steps [5,6].
Step 1.To determine parameters of the PSO algorithm: number m of particles in a swamp, velocity coefficient K , personal and global velocity coefficients Mˆ and M ~, maximum iterations number max

N
of the PSO algorithm.To determine types T of kernel functions, which take part in the search and ranges boundaries of ).To determine the particles' "regeneration" percentage p .
Step 2. To define equal number of particles for each kernel type function T , included in search, to initialize coordinate i T for each i -th particle ( m i , 1 ), other coordinates of the i -th particle ( m i , 1 ) must be generated randomly from the corresponding ranges: ).To establish initial position of the i -th particle ( m i , 1 ) as its best known position , to determine the best particle with coordinates' vector ) , , ~ , ( C x x T from all the m particles, and to determine the best particle for each kernel function type T , including in a search, with coordinates' vector . Number of executed iterations must be considered as 1.
Step 3. To execute while the number of iterations is less than the fixed number max N : • "regeneration" of particles: to choose p % of particles which represent the lowest quality of classification from particles with coordinate • correction of velocity vector ) , , ( of the i -th particle ( m i , 1 ) using formulas: where rˆ and r ~ are random numbers in interval (0, 1), F is a compression ratio calculated using the formula (5); formula (8) is the modification of formula (4): the coordinates' values of the globally best particle; • accuracy calculation of the SVM classifier with parameters' values ) , , , ( ) with aim to find the optimal combination ) , , , ~ ( , which will provide high quality of classification; • increase the number of iterations on 1.
The particle with the optimal combination of the parameters' values which provides the highest quality of classification on chosen the function types will be defined after execution of the offered algorithm.

Experimental studies
The SVM classifiers were applied for classification of the objects of the Earth's surface.Herewith, we considered the identification problem of the water objects and the identification problem of the anthropogenous objects.
The datasets have been created on the base of 127 hyperspectral features of 220 standard objects.Information on these objects has been obtained according to the hyperspectral shooting from the "Resource-P" spacecraft No. 1.This information forms the spectral library of the HSFs.The HSF of the analyzable object represents a vector of the coefficients dependence of the spectral reflection on the wavelength of the registered radiation.
At the development of the SVM classifier we used the several train and test sets, formed randomly on the base of the dataset with the HSFs of the standard objects.Herewith, we applied the several kernel function types with the values of parameters set by default.The size of the test set was equal to 20% of the size of the initial dataset with the HSFs of the standard objects.
For evaluation of the classification quality of the developed SVM classifier we used the well-known classification quality indicators, such as the F -measure indicator, the sensitivity indicator ( Se ), the specificity indicator ( Sp ), the accuracy indicator, the number of errors of I and II type, the AUC indicator, calculated on the test set, etc.The results of the SVM classifier development are given in Table 1.
The SVM classifiers developed with the values of the parameters set by default for the water objects have the high ability to generalization: the values of all quality indicators are high (Table 1), and the SVM classifiers developed with the values of the parameters set by default for the anthropogenous objects are inferior in the classification quality to the SVM classifiers for the water objects (Table 1).Therefore, there is a need for finding of the values of the parameters of the SVM classifiers allowing to identify the anthropogenous objects with the more high classification quality.
The search of the optimum values of the SVM classifier parameters for the anthropogenous objects was performed using the traditional and modified PSO algorithms (with the identical ranges of change of the parameters values and with the identical parameters values of the PSO algorithm).We used the swarm of 300 particles and carried out 15 iterations of the PSO algorithms.Hence, we analyzed 4500 SVM classifiers and have chosen from them the best.As a result, both PSO algorithms determined the radial kernel function as the optimum kernel function type, the similar values of the parameter of kernel function and the value of the regularization parameter, and also the similar values of the accuracy indicator on the train and test sets for the SVM classifier developed taking into account the found parameter values.However, the modified PSO algorithm was more effective on the search time: the search time of the required decision spent by it appeared less (by 2 -3 times) than the search time using by the traditional PSO algorithm.Table 2 shows the results of the SVM classifier development with the values of the kernel function parameters which are found the modified PSO algorithm for the anthropogenous objects.

Conclusions
The use of the SVM classifier to the objects identification on the base of their HSF allows providing the high classification accuracy.The results of experimental studies confirm the expediency of further development of the offered approach to the solution of the objects identification on the base of their HSF.
The use of the offered approach will allow to solve the objects identification's problem of the Earth's surface on allocated from the processed space images of the "Resource-P" spacecraft HSF with the subsequent accumulation of the standard HSFs in the database that, in turn, will provide creation of actual domestic spectral library of standards HSFs which can be applied when monitoring a condition of agricultural grounds, forests, water resources, an ecological condition of soils, etc. [7].Herewith, it is necessary to create the representative dataset of the HSFs and use it to learn the SVM-classifiers.Then, we can identify the new HSF as the HSF of the known class.
function parameters of the i -th particle, (parameter 1 i x is equal to the kernel function parameters d , V or 2 k ; parameter 2 i x is equal " particles to let them correspond to a new kernel function type T ~; M

Table 1 .
The results of the SVM classifier development with the values of the kernel function parameters set by default.

Table 2 .
The results of the SVM classifier development with the values of the kernel function parameters which are found the modified PSO algorithm for the anthropogenous objects.