[R] t-test behavior given that the null hypothesis is true

[(Ted Harding)]

On 09-Jan-2013 08:50:46 Pavlos Pavlidis wrote:
> Dear all,
> I observer a strange behavior of the pvalues of the t-test under
> the null hypothesis. Specifically, I obtain 2 samples of 3
> individuals each from a normal distribution of mean 0 and variance 1.
> Then, I calculate the pvalue using the t-test (var.equal=TRUE,
> samples are independent). When I make a histogram of pvalues
> I see that consistently the bin of the smallest pvalues has a
> lower frequency. Is this a known behavior of the t-test or it's
> a kind of bug/random number generation problem?
> 
> kind regards,
> idaios

Using the following code, I did not observe the behavious you describe.
The histograms are consistent with a uniform distribution of the
P-values, and the lowest bin for the P-values (when the code is
run repeatedly) is not consistently lower (or higher, or anything
else) than the other bins.

## My code:
N <- 10000
Ps <- numeric(N)
for(i in (1:N)){
  X1 <- rnorm(3,0,1) ; X2 <- rnorm(3,0,1)
  Ps[i] <- t.test(X1,X2,var.equal=TRUE)$p.value
}
hist(Ps)
################################################

If you would post the code you used, the reason why you are observing
this may become more evident!

Hoping this helps,
Ted.

-------------------------------------------------
E-Mail: (Ted Harding) <Ted.Harding at wlandres.net>
Date: 09-Jan-2013  Time: 10:29:21
This message was sent by XFMail