[R] Significance test, Cronbach's Alpha

[Chris Evans]

Jonathan Baron sent the following  at 05/06/2006 12:05:

... some snipped ...

> It is rare to see anyone report a test for alpha because it is
> usually used descriptively.  If it isn't .7 or higher, people get 
> upset, yet even .5 would be wildly significant in most cases.
> 
> Jon

Feldt did a lot of good work on ANOVA models of alpha, well summarised
in his paper with Salih.  Here are some functions.  Please don't
ridicule my R style, I'm a psychotherapist first, researcher second, and
R enthusiast third.  Amused advice on how to write better code warmly
received.

I'm sure that jackknifing or bootstrapping would give much more
distributionally robust CIs and p values but, as Jon's point above makes
so simply, the real problem is that people don't think through what
they're looking for from an alpha.  I find there are situations in which
I'm genuinely interested in the null: is there some evidence of
covariance here?; and other situations where I want a high alpha because
I'm postulating that we have a useful measure, in the latter case, all
these totemistic values that are "acceptable", "excellent" etc. are
often misleading and a CI around the observed alpha and some exploration
of the factor structure, EFA or CFA, or IRT model explorations, will be
far more important than exactly what alpha you got.

Oops </FLAME>  not quite sure where I should have put the starter on that!

I'll go back to enjoying the fact that I think this is the first time
I've posted something that might be useful to someone!

Very best all:



Chris

feldt1.return <- function(obs.a, n, k, ci = 0.95, null.a = 0)
{
	if(obs.a > null.a)
		f <- (1 - obs.a)/(1 - null.a)
	else f <- (1 - null.a)/(1 - obs.a)	
              # allows for testing against a higher null
	n.den <- (n - 1) * (k - 1)
	n.num <- n - 1
	null.p <- pf(f, n.num, n.den)	
              # set the upper and lower p values for the desired C.I.
	p1 <- (1 - ci)/2
	p2 <- ci + p1	# corresponding F values
	f1 <- qf(p1, n.num, n.den)
	f2 <- qf(p2, n.num, n.den)	# confidence interval
	lwr <- 1 - (1 - obs.a) * f2
	upr <- 1 - (1 - obs.a) * f1
	cat(round(lwr,2), "to",round(upr,2),"\n")
	interval <- list(lwr,upr)
        return(interval)
}

feldt1.lwr <- function(obs.a, n, k, ci = 0.95, null.a = 0)
{
	if(obs.a > null.a)
		f <- (1 - obs.a)/(1 - null.a)
	else f <- (1 - null.a)/(1 - obs.a)	
             # allows for testing against a higher null
	n.den <- (n - 1) * (k - 1)
	n.num <- n - 1
	null.p <- pf(f, n.num, n.den)	
             # set the upper and lower p values for the desired C.I.
	p1 <- (1 - ci)/2
	p2 <- ci + p1	# corresponding F values
	f1 <- qf(p1, n.num, n.den)
	f2 <- qf(p2, n.num, n.den)	# confidence interval
	lwr <- 1 - (1 - obs.a) * f2
        return(lwr)
}

feldt1.upr <- function(obs.a, n, k, ci = 0.95, null.a = 0)
{
	if(obs.a > null.a)
		f <- (1 - obs.a)/(1 - null.a)
	else f <- (1 - null.a)/(1 - obs.a)	
              # allows for testing against a higher null
	n.den <- (n - 1) * (k - 1)
	n.num <- n - 1
	null.p <- pf(f, n.num, n.den)	
              # set the upper and lower p values for the desired C.I.
	p1 <- (1 - ci)/2
	p2 <- ci + p1	# corresponding F values
	f1 <- qf(p1, n.num, n.den)
	f2 <- qf(p2, n.num, n.den)	# confidence interval
	upr <- 1 - (1 - obs.a) * f1
        return(upr)
}



-- 
Chris Evans <chris at psyctc.org>
Hon. Professor of Psychotherapy, Nottingham University;
Consultant Psychiatrist in Psychotherapy, Rampton Hospital;
Research Programmes Director, Nottinghamshire NHS Trust;
Hon. SL Institute of Psychiatry, Hon. Con., Tavistock & Portman Trust
**If I am writing from one of those roles, it will be clear. Otherwise**

**my views are my own and not representative of those institutions    **