A method to explore the best mixed-effects model in a data-driven manner with multiprocessing: applications in public health research

• Hyemin Han ^[1]
  1. [1] University of Alabama
• Localización: EJIHPE: European Journal of Investigation in Health, Psychology and Education, ISSN 2174-8144, ISSN-e 2254-9625, Vol. 14, Nº. 5, 2024, págs. 1338-1350
• Idioma: inglés
• DOI: 10.3390/ejihpe14050088
• Enlaces
  • Texto completo (pdf)
  • Texto completo
• Resumen

  • In the present study, I developed and tested an R module to explore the best models within the context of multilevel modeling in research in public health. The module that I developed, explore.models, compares all possible candidate models generated from a set of candidate predictors with information criteria, Akaike information criterion (AIC), and Bayesian information criterion (BIC), with multiprocessing. For testing, I ran explore.models with datasets analyzed in three previous studies in public health, which assumed candidate models with different degrees of model complexity.

    These three studies examined the predictors of psychological well-being, compliance with preventive measures, and vaccine intent during the COVID-19 pandemic. After conducting model exploration with explore.models, I cross-validated the nomination results with calculated model Bayes Factors to examine whether the model exploration was performed accurately. The results suggest that explore.models using AIC and BIC can nominate best candidate models and such nomination outcomes are supported by the calculated model Bayes Factors. In particular, all the identified models are superior to the full models in terms of model Bayes Factors. Also, by employing AIC and BIC with multiprocessing, explore.models requires a shorter processing time than model Bayes Factor calculations. These results indicate that explore.models is a reliable, valid, and feasible tool to conduct data-driven model exploration with datasets collected from multiple groups in research on health psychology and education.

• Referencias bibliográficas
  • Brown, V.A. An introduction to linear mixed-effects modeling in R. Adv. Methods Pract. Psychol. Sci. 2021, 4, 2515245920960351.
  • Schmidt-Catran, A.W.; Fairbrother, M. The random effects in multilevel models: Getting them wrong and getting them right. Eur. Sociol. Rev....
  • Schielzeth, H.; Forstmeier, W. Conclusions beyond support: Overconfident estimates in mixed models. Behav. Ecol. 2009, 20, 416–420.
  • Blackburn, A.M.; Han, H.; Gelpí, R.A.; Stöckli, S.; Jefti´c, A.; Ch’ng, B.; Koszałkowska, K.; Lacko, D.; Milfont, T.L.; Lee, Y.; et al. Mediation...
  • Han, H. Trust in the scientific research community predicts intent to comply with COVID-19 prevention measures: An analysis of a large-scale...
  • Han, H. Testing the validity of the modified vaccine attitude question battery across 22 languages with a large-scale international survey...
  • Choi, S.I.; Kim, S.; Jin, Y.; Valentini, C.; Badham, M.; Colleoni, E.; Romenti, S. Effects of individuals’ cultural orientations and trust...
  • Nasvytiene, D.; Lazdauskas, T. Temperament and academic achievement in children: A meta-analysis. ˙ Eur. J. Investig. Health Psychol. Educ....
  • Ta, K.N.T.; Bai, C.H.; Chuang, K.J. Prevention of Work Absence Due to Back Pain: A Network Meta-Analysis. Eur. J. Investig. Health Psychol....
  • Lochbaum, M.; Sisneros, C. A Systematic Review with a Meta-Analysis of the Motivational Climate and Hedonic Well-Being Constructs: The Importance...
  • Bagherzadeh-Khiabani, F.; Ramezankhani, A.; Azizi, F.; Hadaegh, F.; Steyerberg, E.W.; Khalili, D. A tutorial on variable selection for clinical...
  • Wagenmakers, E.J.; Love, J.; Marsman, M.; Jamil, T.; Ly, A.; Verhagen, J.; Selker, R.; Gronau, Q.F.; Dropmann, D.; Boutin, B.; et al. Bayesian...
  • Han, H.; Dawson, K.J. Applying elastic-net regression to identify the best models predicting changes in civic purpose during the emerging...
  • Han, H.; Dawson, K.J.; Walker, D.I.; Nguyen, N.; Choi, Y.J. Exploring the association between character strengths and moral functioning. Ethics...
  • Babyak, M.A. What you see may not be what you get: A brief, nontechnical introduction to overfitting in regression-type models. Psychosom....
  • Genell, A.; Nemes, S.; Steineck, G.; Dickman, P.W. Model selection in medical research: A simulation study comparing Bayesian model averaging...
  • Smith, G. Step away from stepwise. J. Big Data 2018, 5, 32.
  • Raftery, A.E. Bayesian model selection in social research. Sociol. Methodol. 1995, 25, 111–163.
  • Astin, A.W.; Denson, N. Multi-campus studies of college impact: Which statistical method is appropriate? Res. High. Educ. 2009, 50, 354–367....
  • Wetzels, R.; van Ravenzwaaij, D.; Wagenmakers, E.J. Bayesian analysis. In The Encyclopaedia of Clinical Psychology; Wiley: Hoboken, NJ, USA,...
  • Rouder, J.N.; Morey, R.D. Default Bayes factors for model selection in regression. Multivar. Behav. Res. 2012, 47, 877–903.
  • Rouder, J.N.; Haaf, J.M.; Vandekerckhove, J. Bayesian inference for psychology, part IV: Parameter estimation and Bayes factors. Psychon....
  • Han, H. BayesFactorFMRI: Implementing Bayesian Second-Level fMRI Analysis with Multiple Comparison Correction and Bayesian Meta-Analysis of...
  • Morey, R.D.; Rouder, J.N.; Jamil, T.; Urbanek, K.; Ly, A. Package ‘BayesFactor’, 2022.
  • Kuha, J. AIC and BIC: Comparisons of assumptions and performance. Sociol. Methods Res. 2004, 33, 188–229.
  • Chakrabarti, A.; Ghosh, J.K. AIC, BIC and recent advances in model selection. Philos. Stat. 2011, 7, 583–605.
  • Aho, K.; Derryberry, D.; Peterson, T. Model selection for ecologists: The worldviews of AIC and BIC. Ecology 2014, 95, 631–636.
  • Weakliem, D.L. A critique of the Bayesian information criterion for model selection. Sociol. Methods Res. 1999, 27, 359–397.
  • Raftery, A.E. Comment on “A Critique of the Bayesian Information Criterion for Model Selection”. Sociol. Methods Res. 1999, 27, 411–427.
  • Raftery, A.E.; Madigan, D.; Hoeting, J.A. Bayesian model averaging for linear regression models. J. Am. Stat. Assoc. 1997, 92, 179–191.
  • Zou, H.; Hastie, T. Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. Stat. Methodol. 2005, 67, 301–320.
  • McGrath, R.E.; Han, H.; Brown, M.; Meindl, P. What does character education mean to character education experts? A prototype analysis of expert...
  • Makieła, K.; Mazur, B. Bayesian model averaging and prior sensitivity in stochastic frontier analysis. Econometrics 2020, 8, 13.
  • Buja, A.; Brown, L. Discussion: “A significance test for the lasso”. Ann. Stat. 2014, 42, 509–517.
  • Ntontis, E.; Blackburn, A.M.; Han, H.; Stöckli, S.; Milfont, T.L.; Tuominen, J.; Griffin, S.M.; Ikizer, G.; Jeftic, A.; Chrona, S.; et al....
  • Lieberoth, A.; Lin, S.Y.; Stöckli, S.; Han, H.; Kowal, M.; Gelpi, R.; Chrona, S.; Tran, T.P.; Jefti´c, A.; Rasmussen, J.; et al. Stress and worry...
  • Blackburn, A.M.; Han, H.; Jefti´c, A.; Stöckli, S.; Gelpí, R.; Acosta-Ortiz, A.M.; Travaglino, G.A.; Alvarado, R.; Lacko, D.; Milfont, T.L.;...
  • Neville, F.G.; Templeton, A.; Smith, J.R.; Louis, W.R. Social norms, social identities and the COVID-19 pandemic: Theory and recommendations....
  • Merkley, E. Anti-intellectualism, populism, and motivated resistance to expert consensus. Public Opin. Q. 2020, 84, 24–48.
  • Weston, S.; Calaway, R. Getting Started with doParallel and Foreach. 2015. Available online: https://cran.r-project.org/web/ packages/doParallel/vignettes/gettingstartedParallel.pdf...
  • Kuznetsova, A.; Brockhoff, P.B.; Christensen, R.H.B. lmerTest Package: Tests in Linear Mixed Effects Models. J. Stat. Softw. 2017, 82, 1–26....
  • Pelˇci´c, G.; Karaˇci´c, S.; Mikirtichan, G.L.; Kubar, O.I.; Leavitt, F.J.; Tai, M.C.t.; Morishita, N.; Vuleti´c, S.; Tomaševi´c, L. Religious exception...
  • Bürkner, P.C. brms: An R Package for Bayesian Multilevel Models Using Stan. J. Stat. Softw. 2017, 80, 1–28.
  • Bürkner, P.C. Advanced Bayesian Multilevel Modeling with the R Package brms. R J. 2018, 10, 395–411.
  • Kass, R.E.; Raftery, A.E. Bayes factors. J. Am. Stat. Assoc. 1995, 90, 773–795.
  • Vrieze, S.I. Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and...
  • Yamada, Y.; Cepuli´c, D.B.; Coll-Martín, T.; Debove, S.; Gautreau, G.; Han, H.; Rasmussen, J.; Tran, T.P.; Travaglino, G.A.; ´ COVIDiSTRESS...
  • Blackburn, A.M.; Vestergren, S. COVIDiSTRESS diverse dataset on psychological and behavioural outcomes one year into the COVID-19 pandemic....
  • Pietsch, W. Aspects of theory-ladenness in data-intensive science. Philos. Sci. 2015, 82, 905–916.
  • Blagden, D. Induction and deduction in international relations: Squaring the circle between theory and evidence. Int. Stud. Rev. 2016, 18,...