[R] Metafor: How to integrate effectsizes?

Tue May 6 19:14:32 CEST 2014

At 14:23 06/05/2014, Viechtbauer Wolfgang (STAT) wrote:
>Without the sample size of a study (i.e., either 
>the group sizes or the total sample size), you 
>cannot convert the p-value to a t-value or a 
>t-value to a d-value. And for studies where you 
>have the d-value but no sample size, you cannot 
>compute the corresponding sampling variance. So, 
>without additional information, you cannot 
>include these studies. Maybe studies where a 
>d-value is directly reported also report a CI 
>for the d-value? Then the sampling variance can 
>be back-calculated (since a 95% CI for d is 
>typically computed with d +- 1.96 sqrt(vi), where vi is the sampling variance).

Verena,
What Wolfgang says is true of course but if you 
have _both_ the t value and the p value you can 
backcalculate the number of degrees of freedom 
and then if you are willing to assume equal arms you have the sample sizes.

finddf <- function(t, pval) {
    helper <- function(df) {res <- pval - pt(t, df, lower.tail = FALSE); res}
    res <- uniroot(helper, interval = c(5, 10000))
    res
}

If you call finddf with the value of t and the 
_one-sided_ p-value (divide by 2 if two-sided) it 
should give you a return value which, if you look 
at the element of the list called root is its 
estimate of the degrees of freedom. If you get 
errors from uniroot the interval supplied in the call may need to be widened.

I would suggest that when you have your final 
dataset it would be a really good idea to do some 
model checks using plot.influence to see whether 
the studies for which you have imputed values are 
fundamentally different for some reason. This 
will also check your calculations as a bonus.

>Best,
>Wolfgang
>
> > -----Original Message-----
> > From: Verena Weinbir [mailto:vweinbir at gmail.com]
> > Sent: Tuesday, May 06, 2014 15:09
> > To: Michael Dewey
> > Cc: Viechtbauer Wolfgang (STAT); r-help at r-project.org
> > Subject: Re: [R] Metafor: How to integrate effectsizes?
> >
> > Thank you very much for your illustration, Wolfgang! It helped me a
> > lot.Â  And also thank you for the package-hint, Michael!
> >
> > Now, I have re-checked the respective studies, and there still are a
> > couple of studies left, only stating cohens d, and the respective t-value
> > and p-value - sample and group sizes are not addressed (its data from an
> > older meta-analysis). Is there a way to embed these studies in my sample?
> > Wolfgangs illustration addresses only cases in which group sizes are
> > stated, if I understand you correctly...
> >
> > Many thanks in advance,
> >
> > Verena
> >
> > On Sat, Apr 26, 2014 at 1:38 PM, Michael Dewey <info at aghmed.fsnet.co.uk>
> > wrote:
> > At 20:34 25/04/2014, Viechtbauer Wolfgang (STAT) wrote:
> > If you know the d-value and the corresponding group sizes for a study,
> > then it's possible to add that study to the rest of the dataset. Also, if
> > you only know the test statistic from an independent samples t-test (or
> > only the p-value corresponding to that test), it's possible to back-
> > compute what the standardized mean difference is.
> >
> > I added an illustration of this to the metafor package website:
> >
> > http://www.metafor-project.org/doku.php/tips:assembling_data_smd
> >
> > Verena might also like to look at the compute.es package available from
> > CRAN to see whether any of the conversions programmed there do the job.
> >
> >
> > Best,
> > Wolfgang
> >
> > --
> > Wolfgang Viechtbauer, Ph.D., Statistician
> > Department of Psychiatry and Psychology
> > School for Mental Health and Neuroscience
> > Faculty of Health, Medicine, and Life Sciences
> > Maastricht University, P.O. Box 616 (VIJV1)
> > 6200 MD Maastricht, The Netherlands
> > +31 (43) 388-4170Â | http://www.wvbauer.com
> >
> > > -----Original Message-----
> > > From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-
> > project.org]
> > > On Behalf Of Michael Dewey
> > > Sent: Friday, April 25, 2014 16:23
> > > To: Verena Weinbir
> > > Cc: r-help at r-project.org
> > > Subject: Re: [R] Metafor: How to integrate effectsizes?
> > >
> > > At 12:33 25/04/2014, you wrote:
> > > >Thank you very much for your reply and the book recommendation,
> > Michael.
> > > >
> > > >Yes, I mean Cohen's d - sorry for the typo :-)
> > > >
> > > >Just to make this sure for me: There is no
> > > >possibility to integrate stated Cohens' ds in an
> > > >R-Metaanalysis (or a MA at all), if there is no
> > > >further information traceable regarding SE or the like?
> > >
> > > If there is really no other information like
> > > sample sizes, significance level, value of some
> > > significance test then you would have to impute a
> > > value from somewhere. That would seem a last resort.
> > >
> > > I have cc'ed this back to the list, please keep
> > > it on the list so others may benefit and contribute.
> > >
> > >
> > > >best regards,
> > > >
> > > >Verena
> > > >
> > > >
> > > >On Fri, Apr 25, 2014 at 1:21 PM, Michael Dewey
> > > ><<mailto:info at aghmed.fsnet.co.uk>info at aghmed.fsnet.co.uk> wrote:
> > > >At 13:15 24/04/2014, Verena Weinbir wrote:
> > > >Hello!
> > > >
> > > >I am using the metafor package for my master's thesis as an R-newbie.
> > > While
> > > >calculating effectsizes from my dataset (mean values and
> > > >standarddeviations) using "escalc" shouldn't be a problem (I hope ;-
> > )),
> > > I
> > > >wonder how I could at this point integrate additional studies, which
> > > only
> > > >state conhens d (no information about mean value and sds available),
> > to
> > > >calculate an overall analysis. Ã‚ I would be very grateful for your
> > > support!
> > > >
> > > >
> > > >You mean Cohen's d I think.
> > > >
> > > >You will need some more information to enable
> > > >you to calculate its standard error. Have a look at Rosenthal's
> > chapter
> > > in
> > > >@book{cooper94,
> > > >Ã‚ Â Ã‚ author = {Cooper, H and Hedges, L V},
> > > >Ã‚ Â Ã‚ title = {A handbook of research synthesis},
> > > >Ã‚ Â Ã‚ year = {1994},
> > > >Ã‚ Â Ã‚ publisher = {Russell Sage},
> > > >Ã‚ Â Ã‚ address = {New York},
> > > >Ã‚ Â Ã‚ keywords = {meta-analysis}
> > > >}
> > > >(There is an updated edition)
> > > >This gives you more information about converting
> > > >effect sizes and extracting them from unpromising beginnings.
> > > >
> > > >It often requires some ingenuity to get the
> > > >information you need so have a go and then get
> > > >back here with more details if you run into problems
> > > >
> > > >
> > > >Best regards,
> > > >
> > > >Verena

Michael Dewey
info at aghmed.fsnet.co.uk
http://www.aghmed.fsnet.co.uk/home.html