Pre-enrollment tests to predict first-year academic achievement in open-admission higher STEM education

• Jolan Hanssens ^[1] ; Carolien Van Soom ^[1] ; Greet Langie ^[1]
  1. [1] KU Leuven

     KU Leuven

     Arrondissement Leuven, Bélgica

     
• Localización: European Journal of Psychology of Education, ISSN-e 1878-5174, ISSN 0256-2928, Vol. 40, Nº 1, 2025
• Idioma: inglés
• DOI: 10.1007/s10212-024-00918-2
• Enlaces
  • Texto completo
• Resumen

  • The societal demand for graduates with expertise in Science, Technology, Engineering, and Mathematics (STEM) stands in contrast with prevalent issues of diminished interest among high school students in STEM subjects and low completion rates in STEM academic programs. In the Flemish educational context, heterogeneity in STEM preparedness of incoming students, caused by open admission to higher education (HE) and lack of centralized exams at the end of secondary education (SE), contributes to low completion rates. Pre-enrollment, low-stakes positioning tests aim to inform students of their preparedness. This is an in-depth study of the predictive validity for first-year academic achievement in STEM of these tests. We used data from three academic years, four universities, and six study programs (n = 1973). Using nested linear and logistic regression analyses, we found incremental predictive validity of positioning tests over SE outcomes, and learning and study strategies, especially of the mathematics problems on the tests. Furthermore, we found no evidence for differences in predictive validity between relevant sub-populations (related HE programs, or different SE programs). Finally, we introduced utility functions to determine optimal cut-offs for positioning test scores.

• Referencias bibliográficas
  • Ackerman, P. L., Kanfer, R., & Beier, M. E. (2013). Trait complex, cognitive ability, and domain knowledge predictors of baccalaureate...
  • Allen, J., Robbins, S. B., & Sawyer, R. (2009). Can measuring psychosocial factors promote college success? Applied Measurement in Education,...
  • Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508–600). American...
  • Bejar, I. I. (1983). Subject matter experts' assessment of item statistics. Applied Psychological Measurement, 7, 303–310.
  • Carroll, D., Ng, E., & Birch, D. (2009). Retention and progression of postgraduate business students: An Australian perspective. Open...
  • Cole, J. S., & Osterlind, S. J. (2008). Investigating differences between low- and high-stakes test performance on a general education...
  • Credé, M., & Kuncel, N. R. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives...
  • De Winter, J. C. F., & Dodou, D. (2011). Predicting academic performances in engineering using high school exam scores. International...
  • Delnoij, L. E. C., Dirkx, K. J. H., Janssen, J. P. W., & Martens, R. L. (2020). Predicting and resolving non-completion in higher (online)...
  • Delnoij, L. E. C., Janssen, J. P. W., Dirkx, K. J. H., Gijselaers, H. J. M., De Groot, R. H. M., Neroni, J., De Bie, M., & Martens, R....
  • Finn, B. (2015). Measuring motivation in low-stakes assessments. ETS research report series ISSN 2330-8516, 1-17. https://doi.org/10.1002/ets2.12067
  • Fitchett, S., King, K., & Champion, J. (2011). Outcomes of mathematics placement: An analysis of advising and enrollment data. PRIMUS-Problems,...
  • Flemish Government. (2021). Stem-Monitor. https://publicaties.vlaanderen.be/view-file/45890. Accessed 3 Dec 2024.
  • Fonteyne, L., De Fruyt, F., Dewulf, N., Duyck, W., Erauw, K., Goeminne, K., Lammertyn, J., Marchant, T., Moerkerke, B., Oosterlinck, T., &...
  • Fonteyne, L., Duyck, W., & De Fruyt, F. (2017). Program-specific prediction of academic achievement on the basis of cognitive and non-cognitive...
  • Fonteyne, L., Obbels, W., Dorissen, T., Tambuyzer, B., De Laet, T. De, Adriaens, M., & Melis, I. (2019). Validiteitsrapport ijkingstoetsen...
  • Fonteyne, L., Marconato, A., Tambuyzer, B., De Laet, T. De, Adriaens, M., & Melis, I. (2021). Validiteitsrapport ijkingstoetsen 2020.
  • Fonteyne, L., Acke, S., Marconato, A., Tambuyzer, B., De Laet, T., & Melis, I. (2022). Validiteitsrapport ijkingstoetsen 2021.
  • Fonteyne, L., Acke, S., Hanssens, J., Marconato, A., Schütz, J., Tambuyzer, B., De Laet, T., & Melis, I. (2023). Validiteitsrapport ijkingstoetsen...
  • Gonyea, R. M. (2005). Self-reported data in institutional research: Review and recommendations. New Directions for Institutional Research,...
  • Hanssens, J., Langie, G., & Van Soom, C. (2023a). Students’ perceptions of low stakes positioning tests at the start of higher STEM education:...
  • Hanssens, J., Van Soom, C., & Langie, G. (2023b). The stability of pre-enrolment prediction of academic achievement: Criterion-referencing...
  • Hanssens, J., Van Soom, C., & Langie, G. (2021). The predictive validity of the engineering technology positioning test. Presented at...
  • Husbands, A., Rodgerson, M. J., Dowell, J., & Patterson, F. (2015). Evaluating the validity of an integrity-based situational judgement...
  • Impara, J. C., & Plake, B. S. (2005). Teachers” ability to estimate item difficulty: A test of the assumptions in the Angoff standard...
  • Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73.
  • OECD. (2017). Education at a Glance 2017: OECD Indicators. OECD Publishing.
  • OECD. (2022). Education at a Glance 2022: OECD Indicators. OECD Publishing.
  • Pinxten, M., Van Soom, C., Peeters, C., De Laet, T., & Langie, G. (2019). At-risk at the gate: Prediction of study success of first-year...
  • Revelle, W. (2023). Psych: Procedures for psychological, psychometric, and personality research. Northwestern University, Evanston, Illinois....
  • Richardson, K. (2002). What IQ tests test. Theory & Psychology, 12(3), 283–314.
  • Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of University students’ academic performance: A systematic review...
  • Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college...
  • Schelfout, S., Wille, B., Fonteyne, L., Roels, E., Derous, E., De Fruyt, F., & Duyck, W. (2022). How accurately do program-specific basic...
  • Schippers, M. C., Scheepers, A. W. A., & Peterson, J. B. (2015). A scalable goal-setting intervention closes both the gender and ethnic...
  • Schmitt, N., Keeney, J., Oswald, F. L., Pleskac, T. J., Billington, A. Q., Sinha, R., & Zorzie, M. (2009). Prediction of 4-year college...
  • Stigler, G. J. (1950). The development of utility theory. Journal of Political Theory, 58(4), 307–327.
  • Thiele, C., & Hirschfeld, G. (2021). cutpointr: Improved estimation and validation of optimal cutpoints in R. Journal of Statistical Software,...
  • Thiele, C. (2021). cutpointr: Determine and evaluate optimal cutpoints in binary classification tasks. R package version 1.1.1. https://CRAN.R-project.org/package=cutpointr....
  • Van Soom, C., & Donche, V. (2014). Profiling first-year students in STEM programs based on autonomous motivation and academic self-concept...
  • Vanderoost, Jef, Callens, R., Vandewalle, J., & De Laet, T. (2014). Engineering positioning test in Flanders : A powerful predictor for...
  • Vanderoost, J., Van Soom, C., Langie, G., Van Den Bossche, J., Callens, R., Vandewalle, J., & De Laet, T. (2015). Engineering and science...
  • Van den Broeck, L., De Laet, T., Lacante, M., Pinxten, M., Van Soom, C., & Langie, G. (2019). Predicting the academic achievement of students...
  • Weinstein, C. E., & Palmer, D. R. (2002). Learning and study strategies inventory (2nd ed.). H&H Publishing Company.
  • Weinstein, C. E., Palmer, D. R., & Acee, T. W. (2016). User’s manual learning and study strategies inventory (3rd ed.). H&H Publishing...
  • Willems, J., Coertjens, L., Tambuyzer, B., & Donche, V. (2019). Identifying science students at risk in the first year of higher education:...