[R] normality tests [Broadcast]
Thomas Lumley
tlumley at u.washington.edu
Mon May 28 20:58:37 CEST 2007
On Mon, 28 May 2007, Martin Maechler wrote:
>>>>>> "LuckeJF" == Lucke, Joseph F <Joseph.F.Lucke at uth.tmc.edu>
>>>>>> on Fri, 25 May 2007 12:29:49 -0500 writes:
>
> LuckeJF> Most standard tests, such as t-tests and ANOVA,
> LuckeJF> are fairly resistant to non-normalilty for
> LuckeJF> significance testing. It's the sample means that
> LuckeJF> have to be normal, not the data. The CLT kicks in
> LuckeJF> fairly quickly.
>
> Even though such statements appear in too many (text)books,
> that's just plain wrong practically:
>
> Even though *level* of the t-test is resistant to non-normality,
> the power is not at all!! And that makes the t-test NON-robust!
While it is true that this makes the t-test non-robust, it doesn't mean
that the statement is just plain wrong practically.
The issue really is more complicated than a lot of people claim (not you
specifically, Martin, but upthread and previous threads).
Starting with the demonstrable mathematical facts:
- lots of rank tests are robust in the sense of Huber
- rank tests are optimal for specific location-shift testing problems.
- lots of rank tests have excellent power for location shift alternatives
over a wide range of underlying distributions.
- rank tests fail to be transitive when stochastic ordering is not
assumed (they are not consistent with any ordering on all distributions)
- rank tests do not lead to confidence intervals unless a location shift
or similar one-dimensional family is assumed
- No rank test is uniformly more powerful than any parametric test or
vice versa (if we rule out pathological cases)
- there is no rank test that is consistent precisely against a difference
in means
- the t-test (and essentially all tests) can be made distribution-free in
large samples (for small values of 'large', usually)
- being distribution-free does not guarantee robustness of power (for the
t-test or for any other test)
Now, if we assume stochastic ordering is the Wilcoxon rank-sum test more
or less powerful than the t-test? Everyone knows that this depends on the
null hypothesis distribution. Fewer people seem to know that it also
depends on the alternative, especially in large samples.
Suppose the alternative of interest is not that the values are uniformly
larger by 1 unit, but that 5% of them are about 20 units larger. The
Wilcoxon test -- precisely because it gives less weight to outliers --
will have lower power. For example (ObR)
one.sim<-function(n, pct, delta){
x<-rnorm(n)
y<-rnorm(n)+delta*rbinom(n,1,pct)
list(x=x,y=y)
}
mean(replicate(100, {d<-one.sim(100,.05,20); t.test(d$x,d$y)$p.value})<0.05)
mean(replicate(100, {d<-one.sim(100,.05,20); wilcox.test(d$x,d$y)$p.value})<0.05)
mean(replicate(100, {d<-one.sim(100,.5,1); t.test(d$x,d$y)$p.value})<0.05)
mean(replicate(100, {d<-one.sim(100,.5,1); wilcox.test(d$x,d$y)$p.value})<0.05)
Since both relatively uniform shifts and large shifts of small fractions
are genuinely important alternatives in real problems it is true in
practice as well as in theory that neither the Wilcoxon nor the t-test is
uniformly superior.
This is without even considering violations of stochastic ordering --
which are not just esoteric pathologies, since it is quite plausible for a
treatment to benefit some people and harm others. For example, I've seen
one paper in which a Wilcoxon test on medical cost data was statistically
significant in the *opposite direction* to the difference in means.
This has been a long rant, but I keep encountering statisticians who think
anyone who ever recommends a t-test just needs to have the number 0.955
quoted to them.
<snip>
>
> LuckeJF> Testing for normality prior to choosing a test
> LuckeJF> statistic is generally not a good idea.
>
> Definitely. Or even: It's a very bad idea ...
>
I think that's something we can all agree on.
-thomas
More information about the R-help
mailing list