[R] Recursive Feature Elimination with SVM

[David Winsemius]

On 1/1/19 4:40 AM, Priyanka Purkayastha wrote:
> I have a dataset (data) with 700 rows and 7000 columns. I am trying to do
> recursive feature selection with the SVM model. A quick google search
> helped me get a code for a recursive search with SVM. However, I am unable
> to understand the first part of the code, How do I introduce my dataset in
> the code?


Generally the "labels" is given to such a machine learning device as the 
y argument, while the "features" are passed as a matrix to the x argument.


-- 

David.

>
> If the dataset is a matrix, named data. Please give me an example for
> recursive feature selection with SVM. Bellow is the code I got for
> recursive feature search.
>
>      svmrfeFeatureRanking = function(x,y){
>
>      #Checking for the variables
>      stopifnot(!is.null(x) == TRUE, !is.null(y) == TRUE)
>
>      n = ncol(x)
>      survivingFeaturesIndexes = seq_len(n)
>      featureRankedList = vector(length=n)
>      rankedFeatureIndex = n
>
>      while(length(survivingFeaturesIndexes)>0){
>      #train the support vector machine
>      svmModel = svm(x[, survivingFeaturesIndexes], y, cost = 10,
> cachesize=500,
>                  scale=FALSE, type="C-classification", kernel="linear" )
>
>      #compute the weight vector
>      w = t(svmModel$coefs)%*%svmModel$SV
>
>      #compute ranking criteria
>      rankingCriteria = w * w
>
>      #rank the features
>      ranking = sort(rankingCriteria, index.return = TRUE)$ix
>
>      #update feature ranked list
>      featureRankedList[rankedFeatureIndex] =
> survivingFeaturesIndexes[ranking[1]]
>      rankedFeatureIndex = rankedFeatureIndex - 1
>
>      #eliminate the feature with smallest ranking criterion
>      (survivingFeaturesIndexes = survivingFeaturesIndexes[-ranking[1]])}
>      return (featureRankedList)}
>
>
>
> I tried taking an idea from the above code and incorporate the idea in my
> code as shown below
>
>      library(e1071)
>      library(caret)
>
>      data<- read.csv("matrix.csv", header = TRUE)
>
>      x <- data
>      y <- as.factor(data$Class)
>
>      svmrfeFeatureRanking = function(x,y){
>
>        #Checking for the variables
>        stopifnot(!is.null(x) == TRUE, !is.null(y) == TRUE)
>
>        n = ncol(x)
>        survivingFeaturesIndexes = seq_len(n)
>        featureRankedList = vector(length=n)
>        rankedFeatureIndex = n
>
>        while(length(survivingFeaturesIndexes)>0){
>          #train the support vector machine
>          svmModel = svm(x[, survivingFeaturesIndexes], y, cross=10,cost =
> 10, type="C-classification", kernel="linear" )
>
>          #compute the weight vector
>          w = t(svmModel$coefs)%*%svmModel$SV
>
>          #compute ranking criteria
>          rankingCriteria = w * w
>
>          #rank the features
>          ranking = sort(rankingCriteria, index.return = TRUE)$ix
>
>          #update feature ranked list
>          featureRankedList[rankedFeatureIndex] =
> survivingFeaturesIndexes[ranking[1]]
>          rankedFeatureIndex = rankedFeatureIndex - 1
>
>          #eliminate the feature with smallest ranking criterion
>          (survivingFeaturesIndexes = survivingFeaturesIndexes[-ranking[1]])}
>
>        return (featureRankedList)}
>
> But couldn't do anything at the stage "update feature ranked list"
> Please guide
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.