[R] Custom caret metric based on prob-predictions/rankings

[Yang Zhang]

Actually, is there any way to get at additional information beyond the
classProbs?  In particular, is there any way to find out the
associated weights, or otherwise the row indices into the original
model matrix corresponding to the tested instances?

On Thu, Feb 9, 2012 at 4:37 PM, Yang Zhang <yanghatespam at gmail.com> wrote:
> Oops, found trainControl's classProbs right after I sent!
>
> On Thu, Feb 9, 2012 at 4:30 PM, Yang Zhang <yanghatespam at gmail.com> wrote:
>> I'm dealing with classification problems, and I'm trying to specify a
>> custom scoring metric (recall at p, ROC, etc.) that depends on not just
>> the class output but the probability estimates, so that caret::train
>> can choose the optimal tuning parameters based on this metric.
>>
>> However, when I supply a trainControl summaryFunction, the data given
>> to it contains only class predictions, so the only metrics possible
>> are things like accuracy, kappa, etc.
>>
>> Is there any way to do this that I'm looking?  If not, could I put
>> this in as a feature request?  Thanks!
>>
>> --
>> Yang Zhang
>> http://yz.mit.edu/
>
>
>
> --
> Yang Zhang
> http://yz.mit.edu/



-- 
Yang Zhang
http://yz.mit.edu/