Does understanding what a test measures make a difference? On the relevance of the ability to identify criteria for situational judgment test performance
Publikation: Bidrag til tidsskrift › Tidsskriftartikel › Forskning › fagfællebedømt
Standard
Does understanding what a test measures make a difference? On the relevance of the ability to identify criteria for situational judgment test performance. / Reznik, Nomi; Krumm, Stefan; Freudenstein, Jan Philipp; Heimann, Anna L.; Ingold, Pia; Schäpers, Philipp; Kleinmann, Martin.
I: International Journal of Selection and Assessment, 2024.Publikation: Bidrag til tidsskrift › Tidsskriftartikel › Forskning › fagfællebedømt
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - Does understanding what a test measures make a difference? On the relevance of the ability to identify criteria for situational judgment test performance
AU - Reznik, Nomi
AU - Krumm, Stefan
AU - Freudenstein, Jan Philipp
AU - Heimann, Anna L.
AU - Ingold, Pia
AU - Schäpers, Philipp
AU - Kleinmann, Martin
N1 - Publisher Copyright: © 2023 The Authors. International Journal of Selection and Assessment published by John Wiley & Sons Ltd.
PY - 2024
Y1 - 2024
N2 - Situational judgment tests (SJTs) are low-fidelity simulations that are often used in personnel selection. Previous research has provided evidence that the ability to identify criteria (ATIC)—individuals' capability to detect underlying constructs in nontransparent personnel selection procedures—is relevant in simulations in personnel selection, such as assessment centers and situational interviews. Building on recent theorizing about response processes in SJTs as well as on previous empirical results, we posit that ATIC predicts SJT performance. We tested this hypothesis across two preregistered studies. In Study 1, a between-subjects planned-missingness design (N = 391 panelists) was employed and 55 selected items from five different SJTs were administered. Mixed-effects-modeling revealed a small effect for ATIC in predicting SJT responses. Results were replicated in Study 2 (N = 491 panelists), in which a complete teamwork SJT was administered with a high- or a low-stakes instruction and showed either no or a small correlation with ATIC, respectively. We compare these findings with other studies, discuss implications for our understanding of response processes in SJTs, and derive avenues for future research.
AB - Situational judgment tests (SJTs) are low-fidelity simulations that are often used in personnel selection. Previous research has provided evidence that the ability to identify criteria (ATIC)—individuals' capability to detect underlying constructs in nontransparent personnel selection procedures—is relevant in simulations in personnel selection, such as assessment centers and situational interviews. Building on recent theorizing about response processes in SJTs as well as on previous empirical results, we posit that ATIC predicts SJT performance. We tested this hypothesis across two preregistered studies. In Study 1, a between-subjects planned-missingness design (N = 391 panelists) was employed and 55 selected items from five different SJTs were administered. Mixed-effects-modeling revealed a small effect for ATIC in predicting SJT responses. Results were replicated in Study 2 (N = 491 panelists), in which a complete teamwork SJT was administered with a high- or a low-stakes instruction and showed either no or a small correlation with ATIC, respectively. We compare these findings with other studies, discuss implications for our understanding of response processes in SJTs, and derive avenues for future research.
KW - ability to identify criteria
KW - planned missingness
KW - situational judgment test
UR - http://www.scopus.com/inward/record.url?scp=85176947110&partnerID=8YFLogxK
U2 - 10.1111/ijsa.12458
DO - 10.1111/ijsa.12458
M3 - Journal article
AN - SCOPUS:85176947110
JO - International Journal of Selection and Assessment
JF - International Journal of Selection and Assessment
SN - 0965-075X
ER -
ID: 379037636