[R] cohen kappa for two-way table

[Jim Lemon]

Dylan Beaudette wrote:
> Greetings,
> 
> I am a bit confused by the results returned by the functions:
> cohen.kappa {concord}
> classAgreement {e1071}
> 
> when using a two-way table.
> 
> 
> for example, if I have an matrix A, and a similar matrix B (same
> dimensions), then:
> 
> matrix A and B can be found:
> http://casoilresource.lawr.ucdavis.edu/drupal/files/a_40.txt
> http://casoilresource.lawr.ucdavis.edu/drupal/files/b_40.txt
> 
> A <- matrix(unlist( read.table('a_40.txt'), use.names=FALSE), ncol=14)
> B <- matrix(unlist( read.table('b_40.txt'), use.names=FALSE), ncol=14)
> 
If I interpret this correctly, you are considering the numbers in A and 
B to be nominal variables representing levels of some data attribute.
What cohen.kappa wants is:

1(type=counts) a matrix of counts where each cell represents the number 
of methods (rows are different methods) that classified the data objects 
into levels (columns are different levels).
2(type=scores) a matrix of scores where each cell represents the level 
assigned by a method (rows again) to a data object (columns are data 
objects here).

If your numbers are counts, you can get a sensible kappa for _each_ matrix.
If your numbers are scores, you can get a sensible kappa by specifying 
type=scores for _each_ matrix
As far as I can see, you are computing a sparsely populated table of the 
two matrices and passing that, and I don't see how that can give a 
meaningful result. I suspect that the numbers might be counts in the 
first place, but they are being used as scores.

> Secondly, when using the classAgreements() function I get different numbers:
> 
I'll let someone who knows more about the classAgreement function deal 
with this.

Jim