[R] How to solve this problem ?

[Greg Snow]

No, a higher p-value does not mean the variances are more likely to be 0, this is a common misconception.  If the variances are equal (the null hypothesis is true), then for an accurate test the p-value will follow a uniform(0,1) distribution, so p-values of 0.01, 0.049, 0.051, 0.5, 0.99, and 1 are all equally likely to be seen.

Under the frequentist philosophy, the null is either true or not; there is no talking about the likelihood of it being true.

Under the Bayesian philosophy you can talk about probabilities or likelihoods of the variances being equal, but you need additional information, not just a frequentist p-value.

Imagine a simple case that our null hypothesis is that a coin is a 2-headed coin, we flip the coin once and it comes up heads.  The frequentist p-value in this case is 1, what we have seen is perfectly consistent with the null hypothesis; but would you say based on that bit of data that it is guarenteed to be a 2-headed coin?

You may get more information by looking at a confidence interval of the ratio of the variances or some other measure than from a specific test.  Also ask yourself what the costs are for type I and type II errors on the variance test, you may decide that the best approach is to just assume one way or the other, rather than doing the test (and if you test the variances, then do another test and which test is based on the results of the variance test, then you should really take that into account in your final conclusions).

Hope this helps,


--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.snow at imail.org
(801) 408-8111



> -----Original Message-----
> From: r-help-bounces at r-project.org
> [mailto:r-help-bounces at r-project.org] On Behalf Of Daren Tan
> Sent: Wednesday, August 20, 2008 8:09 PM
> To: r-help at stat.math.ethz.ch
> Subject: Re: [R] How to solve this problem ?
>
>
> Small progress, I am relying on levene test to check for
> equality of variances. Is my understanding correct, the
> larger the p-value, the more likely the variances are the same ?
>
> > trt
>  [1] 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4
> Levels: 1 2 3 4
> > levene.test(rep(rnorm(5), 4), trt, option="median")
>
>         Modified Robust Brown-Forsythe Levene-type test based
> on the absolute
>         deviations from the median
>
> data:  rep(rnorm(5), 4)
> Test Statistic = 0, p-value = 1
>
>
>
> > From: daren76 at hotmail.com
> > To: r-help at stat.math.ethz.ch
> > Subject: How to solve this problem ?
> > Date: Wed, 20 Aug 2008 13:33:23 +0000
> >
> >
> > I have disabled html text editing mode, thanks to Prof.
> Ripley for the kind reminder.
> >
> > Given three geological devices that takes 5 readings at 4
> environmental conditions (A to D). What will be the proper
> approach to select the most reliable device ?
> >
> > m1 <- c(73,52,49,53,83,43,58,94,53,62,75,66,41,72,70,75,57,59,85,84)
> > m2 <- c(31,38,30,35,36,26,27,38,22,31,24,35,36,31,38,33,32,28,33,30)
> > m3 <- c(65,57,36,40,36,30,40,34,37,40,33,33,37,29,37,37,30,33,40,35)
> >
> > names(m1) <- rep(LETTERS[1:4], each=5)
> > names(m2) <- rep(LETTERS[1:4], each=5)
> > names(m3) <- rep(LETTERS[1:4], each=5)
> >
> > Before writing this email, I have tried to compare the sd
> for each device at each condition, but ran into obstacle on
> how to formulate the null hypothesis. Alternative solution
> tried was ANOVA, I am unsure whether it can help, as it
> compares the differences in means of each group.
> >
> > Thanks
> >
> > _________________________________________________________________
> > Easily edit your photos like a pro with Photo Gallery.
> > http://get.live.com/photogallery/overview
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>