[R] metafor package: effect sizes are not fully independent

[Mike Cheung]

Dear Gang,

Here are just some general thoughts. Wolfgang Viechtbauer will be a
better position to answer questions related to metafor.

For multivariate effect sizes, we first have to estimate the
asymptotic sampling covariance matrix among the effect sizes. Formulas
for some common effect sizes are provided by Gleser and Olkin (2009).

If a fixed-effects model is required, it is quite easy to write your
own GLS function to conduct the multivariate meta-analysis (see e.g.,
Becker, 1992). If a random-effects model is required, it is more
challenging in R. SAS Proc MIXED can do the work (e.g., van
Houwelingen, Arends, & Stijnen, 2002).

Sometimes, it is possible to transform the multivariate effect sizes
into independent effect sizes (Kalaian & Raudenbush, 1996; Raudenbush,
Becker, & Kalaian, 1988). Then univariate meta-analysis, e.g.,
metafor(), can be performed on the transformed effect sizes. This
approach works if it makes sense to pool the multivariate effect sizes
as in your case (2)- the effect sizes are the same but in different
conditions (happy, sad, and neutral). However, this approach does not
work if the multivariate effect sizes are measuring different
concepts, e.g., verbal achievement and mathematical achievement.

Hope this helps.

Becker, B. J. (1992). Using results from replicated studies to
estimate linear models. Journal of Educational Statistics, 17,
341-362.
Gleser, L. J., & Olkin, I. (2009). Stochastically dependent effect
sizes. In H. Cooper, L. V. Hedges, and J. C. Valentine (Eds.), The
handbook of research synthesis and meta-analysis, 2nd edition (pp.
357-376). New York: Russell Sage Foundation.
Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed
linear model for meta-analysis. Psychological Methods, 1, 227-235.
Raudenbush, S. W., Becker, B. J., & Kalaian, H. (1988). Modeling
multivariate effect sizes. Psychological Bulletin, 103, 111-120.
van Houwelingen, H.C., Arends, L.R., & Stijnen, T. (2002). Advanced
methods in meta-analysis: multivariate approach and meta-regression.
Statistics in Medicine, 21, 589-624.

Regards,
Mike
--
---------------------------------------------------------------------
 Mike W.L. Cheung               Phone: (65) 6516-3702
 Department of Psychology       Fax:   (65) 6773-1843
 National University of Singapore
 http://courses.nus.edu.sg/course/psycwlm/internet/
---------------------------------------------------------------------

On Sat, Feb 6, 2010 at 6:07 AM, Gang Chen <gangchen6 at gmail.com> wrote:
> In a classical meta analysis model y_i = X_i * beta_i + e_i, data
> {y_i} are assumed to be independent effect sizes. However, I'm
> encountering the following two scenarios:
>
> (1) Each source has multiple effect sizes, thus {y_i} are not fully
> independent with each other.
> (2) Each source has multiple effect sizes, each of the effect size
> from a source can be categorized as one of a factor levels (e.g.,
> happy, sad, and neutral). Maybe better denote the data as y_ij, effect
> size at the j-th level from the i-th source. I can code the levels
> with dummy variables into the X_i matrix, but apparently the data from
> the same source are correlated with each other. In this case, I would
> like to run a few tests one of which is, for example, whether there is
> any difference across all the levels of the factor.
>
> Can metafor handle these two cases?
>
> Thanks,
> Gang
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
---------------------------------------------------------------------
 Mike W.L. Cheung               Phone: (65) 6516-3702
 Department of Psychology       Fax:   (65) 6773-1843
 National University of Singapore
 http://courses.nus.edu.sg/course/psycwlm/internet/