[R] predict.naiveBayes() bug in e1071 package

[David Meyer]

Confirmed & fixed upstream.

Thanks,
David

On 2012-02-07 18:43, Ali Tofigh wrote:
> Hi,
>
> I'm currently using the R package e1071 to train naive bayes
> classifiers and came across a bug: When the posterior probabilities of
> all classes are small, the result from the predict.naiveBayes function
> become NaNs. This is an issue with the treatment of the
> log-transformed probabilities inside the predict.naiveBayes function.
> Here is an example to demonstrate the problem (you might need to
> increase 'nvar' depending on your machine):
>
> -------------------- 8<  --------------------
> N<- 100
> nvar<- 60
> varnames<- paste("v", 1:nvar, sep="")
>
> dat<- sapply(1:nvar, function(dummy) {c(rnorm(N/2, 0, 1), rnorm(N/2, 10, 1))})
> colnames(dat)<- varnames
>
> out<- rep(c("a","b"), each=N/2)
> names(dat)<- varnames
>
> nb<- naiveBayes(x=dat, y=out)
>
> new.dat<- t(rnorm(nvar, 5, 0.1))
> colnames(new.dat)<- varnames
>
> predict(nb, new.dat, type="raw")
> -------------------- 8<  --------------------
>
> the results of the last line is usually NaNs. As for the solution:
>
> To protect agains very small numbers, the e1071:::predict.naiveBayes
> function takes the probabilities into log-space and adds instead of
> multiplying probabilities. However, when calculating the posterior
> probabilities of each class (when type = "raw"), the log of the
> probabilities are exponentiated, which defeats the purpose of the
> logspace transformation. I suggest the following change to the code:
>
> Towards the end of the predict.naiveBayes function, you currently do:
>
> L<- exp(L)
> L / sum(L)   # this is what is returned
>
> you can instead use
>
> sapply(L, function(lp) {1 / sum(exp(L - lp))})
>
> the above comes from the following equality:
>
> x / (x + y + z) = 1 / (1 + exp(log(y) - log(x)) + exp(log(z) - log(x)))
>
> Best wishes,
> /Ali Tofigh
>
>
>

-- 
Priv.-Doz. Dr. David Meyer
Department of Information Systems and Operations

WU
Wirtschaftsuniversität Wien
Vienna University of Economics and Business
Augasse 2-6, 1090 Vienna, Austria
Tel: +43-1-313-36-4393
Fax: +43-1-313-36-90-4393
HP:  http://ec.wu.ac.at/~meyer