Access Type

Open Access Dissertation

Date of Award

1-1-2010

Degree Type

Dissertation

Degree Name

Ph.D.

Department

Instructional Technology

First Advisor

James L. Moseley

Abstract

A growing number of residency programs are instituting curricula to include the component of evidence-based medicine (EBM) principles and process. However, these curricula may not be able to achieve the optimal learning outcomes, perhaps because various contextual factors are often overlooked when EBM training is being designed, developed, and implemented. The purpose of the study was to develop and validate an instrument, the EBM Environment Scale, to analyze the environment for EBM learning and practice as perceived by medical residents.

The development of the EBM Environment Scale underwent the process of content domain identification, item generation, review by content experts and a focus group of chief residents. All items on the scale measured responses on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Low scores on the scale represented unfavorable perceptions and high scores represented favorable perceptions of the EBM environment for residents. An EBM Environment Survey that contained the EBM Environment Scale and several demographic questions was administered to residents recruited from 6 programs at 6 training sites (four programs in internal medicine, one in family medicine, and one in pediatrics). The psychometric properties of the scale were tested with Cronbach's coefficient alpha and split-half reliability. Validity was assessed by comparing predetermined subscales with the scale's internal structure assessed via factor analysis. The scale was further validated with the Mann-Whitney U and Kruskal-Wallis tests to evaluate any differences on perception scores among groups of participants identified by gender, country of the medical school attended, level of residency training, affiliated residency program, level of prior EBM training in medical school, and level of prior EBM training during residency.

One hundred twenty four out of 262 eligible residents completed the survey (a response rate of 47%). The overall mean score from the sample was 3.89 with a SD of .56. The initial reliability analysis of the 48 item scale had a high reliability coefficient (Cronbach  = .94). Factor analysis and further item analysis resulted in a shorter 36-item scale with a satisfactory reliability coefficient (Cronbach  =.86). The reliability coefficients for the subscales range from .62 to .98. Factor analysis verified the pre-identified structure of 7 factors, which accounted for 63.57% of the variance. These factors reflected different attributes or aspects that contributed to the EBM environment, including situational cues, learner role, utility and accountability, learning culture, resource availability, learning support, and social support. Perception scores differed significantly (p〈.05) by residency program affiliation with mean scores ranging from 3.51 to 4.13 and by prior EBM training level in medical school with means scores ranging from 3.62 to 4.14 and during residency with mean scores ranging from 3.69 to 4.25.

In initial psychometric testing, the EBM Environment Scale exhibited evidence of adequate internal consistency and construct validity. If further testing confirms its properties, it has potential to be used to understand the influence of the learning environment on the effectiveness of EBM training for residents and to evaluate the quality of the training along with other objective measures to monitor any change in learning outcomes resulted from an EBM training intervention. Additionally, it may be used as a diagnostic tool to detect changes in the EBM learning environment in response to any performance support system interventions. The results of the study suggest strong implications for instructional designers, performance improvement professionals, medical educators, and health information professionals. Recommendations for future research are provided.

Download

Share

COinS