[R] Can ROC be used as a metric for optimal model selection for randomForest?

[Max Kuhn]

XiaoLiu,

I can't see the options in bootControl you used here. Your error is
consistent with leaving classProbs and summaryFunction unspecified.
Please double check that you set them with classProbs = TRUE and
summaryFunction = twoClassSummary before you ran.

Max

On Thu, May 12, 2011 at 7:04 PM, Jing Liu <quiet_jing0920 at hotmail.com> wrote:
>
> Dear all,
>
> I am using the "caret" Package for predictors selection with a randomForest model. The following is the train function:
>
> rfFit<- train(x=trainRatios, y=trainClass, method="rf", importance = TRUE, do.trace = 100, keep.inbag = TRUE,
>    tuneGrid = grid, trControl=bootControl, scale = TRUE, metric = "ROC")
>
> I wanted to use ROC as the metric for variable selection. I know that this works with the logit model by making sure that classProbs = TRUE and summaryFunction = twoClassSummary in the trainControl function. However if I do the same with randomForest, I get a warning saying that
>
> "In train.default(x = trainPred, y = trainDep, method = "rf",  :
>  The metric "ROC" was not in the result set. Accuracy will be used instead."
>
> I wonder if ROC metric can be used for randomForest? Have I missed something? Very very grateful if anyone can help!
>
> Best regards,
> XiaoLiu
>
>
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 

Max