[R] explaining curious result of aov

[Peter Dalgaard]

"Bill Shipley" <bill.shipley at usherbrooke.ca> writes:

> Hello.  I have come across a curious result that I cannot explain.
> Hopefully, someone can explain this.  I am doing a 1-way ANOVA with 6
> groups (example: summary(aov(y~A)) with A having 6 levels).  I get an F
> of 0.899 with 5 and 15 df (p=0.51).  I then do the same analysis but
> using data only corresponding to groups 5 and 6.  This is, of course,
> equivalent to a t-test.  I now get an F of 142.3 with 1 and 3 degrees of
> freedom and a null probability of 0.001.  I know that multiple
> comparisons changes the model-wise error rate, but even if I did all 15
> comparisons of the 6 groups, the Bonferroni correction to a 5% alpha is
> 0.003, yet the Bonferroni correction gives conservative rejection
> levels.
> 
> How can such a result occur?  Any clues would be helpful.

It's a question of assumptions. 

Notice first that you have some very small groups there. Comparing two
groups with 3df means that there are five observations in all,
presumably two in one group and three in the other (although it could
be 4-1).

The joint F test assumes that all the groups have a similar
(theoretical) SD, whereas the two group comparison only assumes that
those two groups are similar.

Suppose one of the other groups had a huge SD; then a joint comparison
would clearly lose power if the actual differences were between some
of the groups with a smaller SD. 

On the other hand, the test on 3df is extremely dependent on
distributional assumptions, and if data are non-normally distributed,
there may be an increased probability of getting a very small variance
(quantization can do that, e.g.) and thus a falsely significant
result.

I.e. I'd take a closer look at the SD's for the 6 groups and perhaps
make a dotplot.

-- 
   O__  ---- Peter Dalgaard             Blegdamsvej 3  
  c/ /'_ --- Dept. of Biostatistics     2200 Cph. N   
 (*) \(*) -- University of Copenhagen   Denmark      Ph: (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)             FAX: (+45) 35327907