Why You Should Always Include a Random Slope for the Lower-Level Variable Involved in a Cross-Level Interaction
Research output: Contribution to journal › Journal article › Research › peer-review
Standard
Why You Should Always Include a Random Slope for the Lower-Level Variable Involved in a Cross-Level Interaction. / Heisig, Jan Paul; Schaeffer, Merlin.
In: European Sociological Review, Vol. 35, No. 2, 2019, p. 258-279.Research output: Contribution to journal › Journal article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - Why You Should Always Include a Random Slope for the Lower-Level Variable Involved in a Cross-Level Interaction
AU - Heisig, Jan Paul
AU - Schaeffer, Merlin
PY - 2019
Y1 - 2019
N2 - Mixed-effects multilevel models are often used to investigate cross-level interactions, a specific type of context effect that may be understood as an upper-level variable moderating the association between a lower-level predictor and the outcome. We argue that multilevel models involving cross-level interactions should always include random slopes on the lower-level components of those interactions. Failure to do so will usually result in severely anti-conservative statistical inference. We illustrate the problem with extensive Monte Carlo simulations and examine its practical relevance by studying 30 prototypical cross-level interactions with European Social Survey data for 28 countries. In these empirical applications, introducing a random slope term reduces the absolute t-ratio of the cross-level interaction term by 31 per cent or more in three quarters of cases, with an average reduction of 42 per cent. Many practitioners seem to be unaware of these issues. Roughly half of the cross-level interaction estimates published in the European Sociological Review between 2011 and 2016 are based on models that omit the crucial random slope term. Detailed analysis of the associated test statistics suggests that many of the estimates would not reach conventional thresholds for statistical significance in correctly specified models that include the random slope. This raises the question how much robust evidence of cross-level interactions sociology has actually produced over the past decades.
AB - Mixed-effects multilevel models are often used to investigate cross-level interactions, a specific type of context effect that may be understood as an upper-level variable moderating the association between a lower-level predictor and the outcome. We argue that multilevel models involving cross-level interactions should always include random slopes on the lower-level components of those interactions. Failure to do so will usually result in severely anti-conservative statistical inference. We illustrate the problem with extensive Monte Carlo simulations and examine its practical relevance by studying 30 prototypical cross-level interactions with European Social Survey data for 28 countries. In these empirical applications, introducing a random slope term reduces the absolute t-ratio of the cross-level interaction term by 31 per cent or more in three quarters of cases, with an average reduction of 42 per cent. Many practitioners seem to be unaware of these issues. Roughly half of the cross-level interaction estimates published in the European Sociological Review between 2011 and 2016 are based on models that omit the crucial random slope term. Detailed analysis of the associated test statistics suggests that many of the estimates would not reach conventional thresholds for statistical significance in correctly specified models that include the random slope. This raises the question how much robust evidence of cross-level interactions sociology has actually produced over the past decades.
UR - https://osf.io/preprints/socarxiv/bwqtd/
U2 - 10.1093/esr/jcy053
DO - 10.1093/esr/jcy053
M3 - Journal article
VL - 35
SP - 258
EP - 279
JO - European Sociological Review
JF - European Sociological Review
SN - 0266-7215
IS - 2
ER -
ID: 212852717