[R] Clustered data with Design package--bootcov() vs. robcov()

[Frank E Harrell Jr]

jjh21 wrote:
> Another question related to bootcov():
> 
> A reviewer is concerned with the fact that bootstrapping the standard errors
> does not give the same answers each time. What is a good way to address this
> concern? Could I bootstrap, say, 100 times and report the mean standard
> error of those 100 estimates? I am already doing 1,000 replications in the
> bootstrap, but of course the answer is still slightly different each time.

First, you can argue that everything we estimate has a margin of error 
and that the variation across different runs of the bootstrap is within 
the statistical precision of what can be estimated.  Second, run the 
bootstrap with 10,000 replications and be done with it.

Frank

> 
> 
> 
> Frank E Harrell Jr wrote:
>>
>> robcov does not use bootstrapping.  It uses the cluster sandwich 
>> (Huber-White) variance-covariance estimator for which there are 
>> references in the help file (see especially Lin).
>>
>> Both robcov and bootcov work best when there is a large number of small 
>> clusters.  If the clusters are somewhat large and greatly vary in size, 
>> expect to be in trouble and consider a full modeling approach 
>> (generalized least squares, mixed models, etc.).
>>
>> One advantage of robcov is that you get the same result every time, 
>> unlike bootstrapping.  But even in the case of cluster sizes of one, the 
>> sandwich estimator can be inefficient (see the Gould paper) or can 
>> result in the "right" estimates of the "wrong" quantity (see a paper by 
>> Friedman in American Statistician).
>>
>> Frank
>>
>>> Thank you.
>>
>> -- 
>> Frank E Harrell Jr   Professor and Chair           School of Medicine
>>                       Department of Biostatistics   Vanderbilt University
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
> 


-- 
Frank E Harrell Jr   Professor and Chair           School of Medicine
                      Department of Biostatistics   Vanderbilt University