mediation() "Proportion mediated" estimate should be bound to 0-100

[amynang]

It's not the most essential aspect of the output but as it is the estimate for proportion mediated can have nonsensical intervals:

library(brms)
library(bayestestR)
library(piecewiseSEM)

data(keeley)

rich_bf <- 
  bf(rich ~ age + firesev) +
  bf(firesev ~ age) +
  set_rescor(FALSE)

rich_brms = brm(rich_bf,
                family = gaussian(),
                backend = "cmdstanr",
                chains = 4,
                iter = 4000,
                warmup = 2000,
                cores = 4,
                refresh = 0,
                data = keeley
)

mediation(rich_brms)

# Causal Mediation Analysis for Stan Model

Treatment: age
Mediator : firesev
Response : rich

Effect                 | Estimate |          95% ETI
----------------------------------------------------
Direct Effect (ADE)    |   -0.204 | [-0.471,  0.059]
Indirect Effect (ACME) |   -0.153 | [-0.305, -0.034]
Mediator Effect        |   -2.636 | [-4.610, -0.604]
Total Effect           |   -0.363 | [-0.610, -0.113]

Proportion mediated: 42.19% [-23.03%, 107.40%]

• Package Version [0.14.0]

[mattansb]

The proportion is computed as $\frac{\text{Indirect Effect}}{\text{Total Effect}}$.
In cases with suppression (the sign of the direct and indirect effects are opposite), this proportion can be negative or larger than 100%.

Here are two examples:

1. %mediated is about 200%

set.seed(111)
x <- gl(2, 1000)
m <- as.numeric(x == 2) + rnorm(1000)
y <- - 0.5 * as.numeric(x == 2) + m + rnorm(1000)

df <- data.frame(x, m, y)

mm <- lm(m ~ x, df)
my <- lm(y ~ x + m, df)


mediation::mediate(mm, my, 
                   treat = "x", mediator = "m") |> 
  parameters::model_parameters()
#> Parameter              | Estimate |         95% CI |      p
#> -----------------------------------------------------------
#> Indirect Effect (ACME) |     1.05 | [ 0.94,  1.15] | < .001
#> Direct Effect (ADE)    |    -0.55 | [-0.65, -0.45] | < .001
#> Total Effect           |     0.50 | [ 0.37,  0.62] | < .001
#> Prop. Mediated         |     2.09 | [ 1.75,  2.65] | < .001

2. %mediated is about -110%

set.seed(111)
x <- gl(2, 1000)
m <- as.numeric(x == 2) + rnorm(1000)
y <- - as.numeric(x == 2) + 0.5 * m + rnorm(1000)

df <- data.frame(x, m, y)

mm <- lm(m ~ x, df)
my <- lm(y ~ x + m, df)


mediation::mediate(mm, my, 
                   treat = "x", mediator = "m") |> 
  parameters::model_parameters()
#> Parameter              | Estimate |         95% CI |      p
#> -----------------------------------------------------------
#> Indirect Effect (ACME) |     0.55 | [ 0.48,  0.61] | < .001
#> Direct Effect (ADE)    |    -1.05 | [-1.15, -0.95] | < .001
#> Total Effect           |    -0.50 | [-0.60, -0.40] | < .001
#> Prop. Mediated         |    -1.10 | [-1.45, -0.84] | < .001

[bwiernik]

Maybe we should show some text that proportion mediated may not be meaningful in the case of suppression when the direct and indirect effect signs are opposite?

[mattansb]

Maybe - but also note that this can happen in the CIs when the level of uncertainty is high for any of the effects (in the example above the point est. for all effects has the same sign, but the CIs of the prop mediated are negative/above 100%).

If we do add a note here, we should also add it to the output of parameters::model_parameters(<mediate>).

[amynang]

I guess it depends on what you want to quantify. Perhaps

$\frac{|Indirect Effect|}{|Direct Effect| + |Indirect Effect|}$

would also make sense, as the proportion of the overall influence that is mediated.

[bwiernik]

But that proportion isn't particularly meaningful. The question "How much of the effect of X can be explained by M?" really only makes sense if X and M have effects in the same direction.

I'm not a fan of proportion mediated as a metric for mediation analysis—for this and other reasons, it is often much more misleading than informative. When I taught regression analysis, I specially advised my students not to use such a metric and instead to focus on the absolute magnitudes of direct and total effects.

[amynang]

If the direct and indirect effect cancel out, isn't it sensible to say that 50% of the influence of A on C is direct, 50% indirect but due to their opposite direction the net effect is 0.

library(brms)
library(bayestestR)
set.seed(42)
d = data.frame(A = rnorm(500),
                 B = NA,
                 C = NA)
d$B = rnorm(500, 10 - 2*d$A, 1)
d$C = rnorm(500,  5 + 2*d$A + 1*d$B, 1)

m = brm(bf(B ~ A) + bf(C ~ A + B) + set_rescor(F),
        backend = "cmdstanr",
        data = d)
mediation(m)

# Causal Mediation Analysis for Stan Model

  Treatment: A
  Mediator : B
  Response : C

Effect                 | Estimate |          95% ETI
----------------------------------------------------
Direct Effect (ADE)    |    2.042 | [ 1.849,  2.228]
Indirect Effect (ACME) |   -2.009 | [-2.203, -1.823]
Mediator Effect        |    1.001 | [ 0.917,  1.083]
Total Effect           |    0.031 | [-0.096,  0.159]

Proportion mediated: -6566.35% [-51445.70%, 38313.00%]

Direct and indirect effects have opposite directions. The proportion mediated is not meaningful.

There is already a note. I may have missed it before.

[mattansb]

If the direct and indirect effect cancel out, isn't it sensible to say that 50% of the influence of A on C is direct, 50% indirect

No, because the proportion is with respect to the ATE, not out of "total potential influence" (which would be interesting to see defined causally).