[R] standard error for lda()

[David Winsemius]

On Feb 9, 2012, at 6:30 PM, array chip wrote:

> David, thanks for your response, hope this stirs more...
>
> Ok, a simple code:
>
> y<-as.factor(rnorm(100)>0.5)
> x1<-rnorm(100)
> x2<-rnorm(100)
> obj<-glm(y~x1+x2,family=binomial)
> predict(obj,type='response',se.fit=T)
>
> predict(obj,...) will give predicted probabilities in the "fit"  
> element; and the associated estimated standard errors in the  
> "se.fit" element (if I understand correctly). The predicted  
> probability from logistic regression is ultimately a function of y  
> and thus a standard error of it should be able to be computed. So  
> one of my questions is whether we can use normal approximation to  
> construct 95% CI for the predicted probabilities using standard  
> errors, because I am not sure if probabilities would follow normal  
> distribution?

Wouldn't it be a binomial distribution if you're dealing with  
classification.

>
> Now, if we try lda():
>
> library(MASS)
> obj2<-lda(y~x1+x2)
> predict(obj2)
>
> where predict(obj2) produces posterior probabilities, the predicted  
> class, etc. My question is whether it's possible to produce standard  
> errors for these posterior probabilities?

The heuristic I use in situations like this: If the authors didn't  
think this was a desirable feature, they probably had sensible reasons  
for _not_ including it (or they decided that another method, such as  
logistic regression, was better). I cannot think of a good metric for  
probability along the line perpendicular to the "line of maximal  
discrimination" for which I confess I cannot remember the accepted  
name. If I were asked to come up with an estimate I would probably  
revert to a bootstrap strategy.


>
> Thanks again.
>
> John
>
>
> From: David Winsemius <dwinsemius at comcast.net>
> To: array chip <arrayprofile at yahoo.com>
> Cc: "r-help at r-project.org" <r-help at r-project.org>
> Sent: Thursday, February 9, 2012 2:59 PM
> Subject: Re: [R] standard error for lda()
>
>
> On Feb 9, 2012, at 4:45 PM, array chip wrote:
>
> > Hi, didn't hear any response yet. want to give it another try..  
> appreciate any suggestions.
> >
>
> My problem after reading this the first time was that I didn't agree  
> with the premise that logistic regression would provide a standard  
> error for a probability. It provides a standard error around an  
> estimated coefficient value. And then you provided no further  
> details or code to create a simulation, and there didn't seem much  
> point in trying to teach you statistical terminology that you were  
> throwning around in a manner that seems rather cavalier , ....  
> admittedly this being a very particular reaction from this non- 
> expert audience of one.
>
>
> > John
> >
> >
> > ________________________________
> >
> > To: "r-help at r-project.org" <r-help at r-project.org>
> > Sent: Wednesday, February 8, 2012 12:11 PM
> > Subject: [R] standard error for lda()
> >
> > Hi, I am wondering if it is possible to get an estimate of  
> standard error of the predicted posterior probability from LDA using  
> lda() from MASS? Logistic regression using glm() would generate a  
> standard error for predicted probability with se.fit=T argument in  
> predict(), so would it make sense to get standard error for  
> posterior probability from lda() and how?
> >
> > Another question about standard error estimate from glm(): is it  
> ok to calculate 95% CI for the predicted probability using the  
> standard error based on normal apprximation, i.e.  
> predicted_probability +/- 1.96 * standard_error?
> >
> > Thanks
> >
> > John
> >    [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >     [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> David Winsemius, MD
> West Hartford, CT
>
>
>

David Winsemius, MD
West Hartford, CT