[R] Metafor multilevel metaregression: total variance increases when moderator added?

[Michael Dewey]

Dear Laura

If you are unable for some reason to share the data why not incorporate 
the output into an e-mail (and please turn of HTML as it mangles 
everything). Putting the plots from profiling somewhere we can read them 
would be a useful addition.

This looks at first glance one of those situations where sadly one has 
insufficient data for the models one would like to fit. We feel your pain.

On 28/02/2017 12:54, Viechtbauer Wolfgang (SP) wrote:
> Very difficult to diagnose what is going on without actually seeing the data. But as I said on CV: Depending on the data, the variance components may not be estimated precisely, so negative values for those kinds of pseudo-R^2 statistics are quite possible. In fact, if a particular moderator is actually unrelated to the outcomes, then in roughly 50% of the cases, the pseudo-R^2 statistic will be negative.
>
> See also:
>
> Lopez-Lopez, J. A., Marin-Martinez, F., Sanchez-Meca, J., Van den Noortgate, W., & Viechtbauer, W. (2014). Estimation of the predictive power of the model in mixed-effects meta-regression: A simulation study. British Journal of Mathematical and Statistical Psychology, 67(1), 30-48.
>
> We only examined the standard mixed-effects meta-regression model with a single moderator, but found that the pseudo-R^2 statistic can be all over the place unless k is quite large.
>
> Now you seem to have a larger number of estimates (170), but these are nested in 'only' 26 studies. So, I suspect that the estimate-level variance component is estimated fairly precisely, but not the study-level variance component. You may want to examine the profile plots (with the profile() function) and/or get (profile-likelihood) CIs of the variance components (using the confint() function). Probably the CI for the study-level variance component is quite wide.
>
> Best,
> Wolfgang
>

-- 
Michael
http://www.dewey.myzen.co.uk/home.html