[R] Function to compute multi-response, multi-rater kappa?

[David Winsemius]

On Feb 1, 2012, at 3:13 PM, Luk Arbuckle wrote:

> I'm very sorry for double posting!  My r-help setting Receive your own
> posts to the list? is set to Yes, and Mail delivery is Enabled.  Yet  
> I did
> not get a copy of my post (this message is a reply from my sent  
> mail).  I
> only learned of the double posting when I found it copied in an r-help
> archive.  Again, my apologies.

I actually had a chance to prevent that second posting. It looked  
familiar when viewed in the moderation queue and I took a quick look  
at what was in my inbox but since you used a different subject line my  
search failed.

Speaking of searching ... you are asked in the Posting Guide (that no  
one reads) to post the specifics of your own efforts. My first search  
with "multi-rater kappa" failed. My second search is here:

http://search.r-project.org/cgi-bin/namazu.cgi?query=multiple+raters+kappa&max=100&result=normal&sort=score&idxname=functions&idxname=Rhelp08&idxname=Rhelp10&idxname=Rhelp02

Having gotten more than one apparently on-target result with  
relatively minor effort, I see no point in my expending even more time.

-- 
David.




>
> Luk Arbuckle
>
> On Wed, Feb 1, 2012 at 13:47, Luk Arbuckle wrote:
>
>> I'm looking for a function in R that extends kappa to multiple  
>> raters when
>> there is more than one response per subject.  For example, say a  
>> group of
>> doctors have to assign diseases to patients.  Each patient will be  
>> assigned
>> one to many diseases, and the number of doctors assigning diseases  
>> to any
>> one patient will be two to many.
>>
>> Here's an extremely simple example of the type of data I might have  
>> (two
>> patients, three doctors, five diagnoses, and doctor 'x' assigns  
>> multiple
>> diagnoses per patient):
>>
>> pat<-c('a','a','a','b','b','b','b')
>> doc<-c('x','x','y','x','x','y','z')
>> dx<-c('1','2','2','3','4','4','5')
>> df<-data.frame(pat=pat,doc=doc,dx=dx)
>> df
>>
>> I found a paper that can address this [1], although I can't find it
>> referenced on cran.  What I have found on cran doesn't address the
>> multi-response case.  I don't want to reinvent the wheel if this has
>> already been implemented in R, and I'm still new to the language so  
>> I'm not
>> a fast coder.  If anyone can help it would be greatly appreciated.
>>
>> Cheers!
>>
>> Luk Arbuckle
>>
>> [1] Kraemer, Helena. “Extension of the Kappa Coefficient.”  
>> Biometrics 36,
>> no. 2 (1980). http://dx.doi.org/10.2307/2529972.
>>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius, MD
West Hartford, CT