Question about meta analysis: means

[ltcdatagonza]

Hi!
First, thanks for your hard work!
I wanted to ask you a question:
When doing a means meta-analysis, you have the option to use original units or SMD.
How does ESCI calculate the SMD? (MD/SD?) and how about the SD? Does it calculate a pooled SD for all the means?
And if you input a reference value to compare... it's only used for the figure or is it used to for calculation?

thanks,
Regards,

[rcalinjageman]

Glad if you're finding esci useful. Are you using it through jamovi or through R?

Difference in Means (two groups)
In jamovi, there is the Meta-Analysis -> Difference in Means command. Under the hood, this command in jamovi is calling meta_mdiff_two function, which is documented here: https://rcalinjageman.github.io/esci/reference/meta_mdiff_two.html. This function obtains SMDs and their sampling variance from CI_smd_ind_contrast, which is documented here: https://rcalinjageman.github.io/esci/reference/CI_smd_ind_contrast.html. I probably need to create a table showing how commands in jamovi map onto the underlying esci functions in R, so that it is easy to find corresponding documentation.

Now to the question: SMDs for two groups are surprisingly difficult. The idea is easy: difference / standard deviation... but what standard deviation to use?

• If the "assume equal variance" option is selected, esci uses the pooled sd (sd_pooled)
• If the "assume equal variance" option is not selected, esci uses the average sd (sd_avg)

Single Group
In jamovi, there is also Meta-Analysis -> Means, which works synthesizes 1-group designs and which can return SMDs. That one uses meta_mean, which is documented here: https://rcalinjageman.github.io/esci/reference/meta_mean.html, which in turn uses CI_smd_one to obtain SMDs and their sampling variance, documented here: https://rcalinjageman.github.io/esci/reference/CI_smd_one.html.

SMDs for a single group are pretty easy: d = M / sd, and there is only 1 sd, so that's what is used.

The single group functions allow the specification of a reference value. The reference value is used to calculate cohen's d. That is, when a reference value is specificed, d1 = (M - reference_value) / sd.

Hope this helps? Sometimes, trying to explain stats is like dancing a painting. My goal with esci is to hopefully make it easy to know what's happening under the hood, both in terms of calculations and references as to where those calculations came from. For esci in r, the documentation is kind of reasonable (though it could use some additional work, for sure). For esci in jamovi, I need to make a table to connect to the r documentation.

Where the documentation is incomplete or unclear, a good second sources is the statpsych package (https://dgbonett.github.io/statpsych/). esci uses statpsych for effect sizes, SEs, and CIs in nearly all cases (2-group SMDs being a notable exception). statpsych has really good documentation and references, and is a good supplement where my work falls short.

[ltcdatagonza]

Hi! Thanks for your thorough explanation!
It clears some of my misconceptions. I will read that documentation now, to get further info.
You clarified what i needed, that meta_any uses SMD for a single group.

I'm working with values from many populations for a meta-analysis. I have 27 manuscripts after applying exclusion criteria,
I'm comparing between populations (a big list of 127 populations), and also i'm working with averaged values and pooled SDs for each manuscript to compare between manuscripts.

Each manuscript uses the same methodology to measure the outcome (Lethal Dose 50, a continuous value with both upper and lower confidence limit at 95%) so the values are comparable, but i have no "reference" value, or control group to compare to (we know a reference value where populations start to have problems but that's it). Each population has a lethal dose 50, and for each manuscript an averaged value of the LD50 and a pooled SD.

Because some populations have smaller sample sizes, others have bigger limits, etc, the SD between populations varies greatly. And between manuscripts, more so.

When comparing means between manuscripts, for example, using raw data i have a lot of values with small SD but 2 manuscripts have bigger SDs, and the forest plot gets very "compressed" because of that. Using SMD so the scales go from 0 to 1 makes things more clear to view and analyze (even more if i input the value in "compare to reference value" that we know is the real reference for all the data), and the biological background of the effect can be analyzed too, but i am not very sure if it is correct (statistically speaking) to do the analysis this way.

Thanks for your time and help. I will now read all the documentation.