[R] Custom caret metric based on prob-predictions/rankings

Yang Zhang yanghatespam at gmail.com
Fri Feb 10 01:30:48 CET 2012

I'm dealing with classification problems, and I'm trying to specify a
custom scoring metric (recall at p, ROC, etc.) that depends on not just
the class output but the probability estimates, so that caret::train
can choose the optimal tuning parameters based on this metric.

However, when I supply a trainControl summaryFunction, the data given
to it contains only class predictions, so the only metrics possible
are things like accuracy, kappa, etc.

Is there any way to do this that I'm looking?  If not, could I put
this in as a feature request?  Thanks!

Yang Zhang

More information about the R-help mailing list