[R] Problem applying McNemar's - Different values in SPSS and R

[Marc Schwartz]

On Dec 29, 2010, at 6:48 AM, Manoj Aravind wrote:

> Thank you Marc :)
> It Certainly helped me to get the exact value of P. 
> How to understand when to apply mcnemar.exact or just mcnemar.test?
> I'm a beginner to biostatistics.
> 
> Manoj Aravind


Generally speaking, exact tests are used for "small-ish" sample sizes. Frequently when n <100 and in many cases, much lower (eg. <50 or <30). The methods tend to become computationally impractical on "larger" data sets.

Since you are coming from SPSS, you might find this document helpful in providing a general framework:

  http://support.spss.com/productsext/spss/documentation/spssforwindows/otherdocs/SPSS%20Exact%20Tests%207.0.pdf

The document is written by Mehta and Patel of Cytel/StatXact, who are historical advocates of the techniques.

That being said and as I noted in my reply to Johannes, I am not typically involved in situations where exact tests make sense, thus am probably not the best resource. I would steer you towards using various reference texts on analyzing categorical data (eg. Agresti) for more information. 

One exception to the above comment, is the use of Fisher's Exact Test (FET), which is typically advocated by folks as an alternative to a chi-square test when **expected** cell counts are <5. However, much has been written in recent times relative to just how conservative the FET is. One resource is:

  http://www.iancampbell.co.uk/twobytwo/twobytwo.htm

Another reference is:

How conservative is Fisher's exact test?
A quantitative evaluation of the two-sample comparative binomial trial
Gerald G. Crans, Jonathan J. Shuster 
Stat Med. 2008 Aug 15;27(18):3598-611.
http://onlinelibrary.wiley.com/doi/10.1002/sim.3221/abstract


So you might want to consider those resources as arguments against using the FET under situations that are likely more commonly observed in day to day practice.

HTH,

Marc