Access Type

Open Access Dissertation

Date of Award

January 2020

Degree Type

Dissertation

Degree Name

Ph.D.

Department

Education Evaluation and Research

First Advisor

Shlomo S. Sawilowsky

Abstract

To meet the needs of complex and/or underserved patient populations, health care professionals must possess diverse backgrounds, qualities, and skill sets. Holistic review has been used to diversify student admissions through examination of non-cognitive attributes of health care applicants. The objective of this study was to develop a novel methodology, the computer-based assessment of non-cognitive attributes of health professionals (CANA­ HP), to effectively screen non-cognitive attributes of applicants. Three research questions were delineated; 1.) To determine the CANA-HP instrument reliability (internal consistency & interrater), 2.) To determine if the CANA-HP measured attributes of non­cognitive variables, as demonstrated by low construct validity scores when correlating the CANA-HP to traditional assessments reported to measure cognition, and 3.) To determine if differential item functioning on the CANA-HP revealed differences between groups based a variety of variables.

The study used a sample of convenience of students interviewed as part of the admission process into the occupational therapy program at Wayne State University (N=37). Participants who consented to the study, completed a demographic survey followed by the 12 question CANA-HP. Data were analyzed using SPSS v. 25.0 (IBM, 2018) or Iteman v. 4.3 (ASC, 2013). Descriptive statistics of the sample population and 12 CANA-HP stations were computed. Cronbach's coefficient alpha was conducted on all of the stations for reliability, while interclass correlation estimates were run for interrater reliability. Pearson's correlation coefficients were calculated between CANA-HP scores and GRE / GPA scores at the time of program admission. Item difficulty, item discrimination, and bias were analyzed using mean average (P), Rbpis, and Fisher’s exact tests respectively.

The six open-ended scenarios had minimally adequate internal reliability (α = 0.71), adequate item discrimination (Rbpis = 0.15 – 0.56), and adequate difficulty (P = 3.51 – 3.70). The traditional multiple choice questions need further refinement as these six scenarios had low reliability and discrimination. Initial results support the hypothesis of no correlation between the CANA-HP and standardized cognitive assessments (GRE and GPA scores). The one exception was non-science GPA which was significantly correlated to the total open-ended scores (p = .002) and total overall score (p = .008) and should be further examined. The CANA-HP is not biased toward the variables of sex, ethnicity, Pell grant status, family college history, or income level. Homogenous raters may improve interrater reliability which ranged from 0.67 – 0.91.

These results should be viewed with caution due to the small sample size conducted at only one university. Predictive validity of this methodology is needed. The CANA-HP remains a work in progress.

Download

Share

COinS