BMJ Open. 2023 Oct 31;13(10):e074440. doi: 10.1136/bmjopen-2023-074440.
ABSTRACT
OBJECTIVES: We aimed to explore the psychometric properties of the first known online asynchronous multiple mini-interview (MMI) designed for fairness with subgroup analyses by key characteristics, usability and acceptability.
DESIGN: Cross-discipline multimethod evaluation.
SETTING: One UK University.
PARTICIPANTS: Applicants to nursing, midwifery and paramedic science undergraduate programmes during 2021-2022.
PRIMARY, SECONDARY OUTCOME MEASURES: Psychometric properties (internal consistency, construct validity, dimensionality) were assessed using Cronbach’s alpha (α), parallel analysis (PA), Schmid-Leiman transformation and ordinal confirmatory factor analysis (CFA). Usability and acceptability were evaluated using descriptive statistics and conventional content analysis.
METHODS: The system was configured in a seven question 4 min MMI. Applicants’ videorecorded their answers which were later assessed by interviewers and scores summed. Applicants and interviewers completed online evaluation questionnaires.
RESULTS: Performance data from 712 applicants determined good-excellent reliability for the asynchronous MMI (mean α 0.72) with similar results across subgroups (gender, age, disability/support needs, UK/non-UK). PA and factor analysis results suggested there were seven factors relating to the MMI questions with an underlying general factor that explained the variance in observed candidate responses. A CFA testing a seven-factor hierarchical model showed an excellent fit to the data (Confirmatory Fit Index=0.99), Tucker Lewis Index=0.99, root mean square error (RMSE) =0.034). Applicants (n=210) viewed the flexibility, relaxed environment and cost savings advantageous. Interviewers (n=65) reported the system to be intuitive, flexible with >70% time saved compared with face-to-face interviews. Reduced personal communication was cited as the principal disadvantage.
CONCLUSIONS: We found that the asynchronous MMI was reliable, time-efficient, fair and acceptable and building fairness in was lost-cost. These novel, insights are applicable across health professions selection internationally informing the future configuration of online interviews to ensure workforces represent the societies they serve.
PMID:37907294 | DOI:10.1136/bmjopen-2023-074440
