Access Type
Open Access Dissertation
Date of Award
1-1-2013
Degree Type
Dissertation
Degree Name
Ph.D.
Department
Instructional Technology
First Advisor
James L. Moseley
Abstract
Selecting appropriate performance improvement interventions is a critical component of a comprehensive model of performance improvement. Intervention selection is an interconnected process involving analysis of an organization's environment, definition of the performance problem, and identification of a performance gap and identification of causal factors. When the performance gap relates to a lack of knowledge or information on the job, instructional approaches such as training have been traditionally used; however, non-instructional interventions such as electronic performance support systems (EPSS) have gained increasing attention as alternatives to training interventions.
Electronic Performance Support Systems (EPSS) use computing technology to support task performance on demand, any time, any place, at the point of need with a minimum need for in-person intervention such as instructors, peer mentors or supervisors. Providing on-demand access requires that the EPSS be integrated into the performer's work environment. However, performance support tools can be integrated into a work environment in a variety of ways. In particular, research suggests that if an EPSS interface design is too complex, it diminished performance, and may not be accepted by users.
The purpose of this research study was to investigate whether progressively increasing the level of integration - external, extrinsic or intrinsic - would make a difference in a user's performance outcomes. According to the EPSS design literature, performance accuracy should increase, and completion time should decrease, along with the level of integration, due to fewer pauses in workflow and more immediate access to support. However, research data to support this assumption has not provided definitive guidance.
The current study sought to determine whether these assumptions can be used as reliable design principles for those seeking to employ an EPSS as part of a performance improvement solution. To this end, an experiment was conducted that compared the performance outcomes of users of three levels of EPSS integration using a library catalog to perform library research tasks. Study participants were randomly assigned to one of four treatment groups representing progressive levels of integration - external, extrinsic or intrinsic, along with a control group. Upon completing a series of library tasks, participants indicated the frequency with which they used the EPSS took to perform their tasks. They also received scores for task accuracy and task completion. Finally, the EPSS system measured participants' time on task with each EPSS type.
Study results indicated that increasing the level of EPSS integration can produce improvement in performance accuracy, but had no effect on task completion, frequency of use or time on task. In fact, the data suggest that time on task may actually increase with the level of integration. A further notable observation from the findings was that overall usage of the EPSS tool was relatively low; on average, 67% of study participants did not use the EPSS tool at all.
Implications for instructional designers include the recommendation that for situations in which performance accuracy is the primary goal, designers can feel confident in recommending an Intrinsic EPSS. However, in situations where speed of performance is a key issue, a less integrated EPSS should be considered. From the perspective of Library and Information Science, the study results support the use of EPSS as a promising tool for enhancing the ability of library users to locate and connect with library resources, and open opportunities to further explore the performance improvement approach in library settings.
From the performance improvement perspective, low usage of a performance support intervention raises questions concerning user acceptance. While critical, selecting an appropriate performance intervention must be done as part of a comprehensive performance improvement approach featuring through performance and cause analysis, and effective planning for implementation and change management. Integral to this approach must be the collection and use of reliable evidence which can be shared with stakeholders to achieve buy-in and extend understanding of the performance improvement process throughout the organization. While more data is needed to extend the reliability of the performance improvement knowledge base, results from the current study have highlighted avenues along which further evidence can be developed.
Recommended Citation
Phillips, Sharon A., "Electronic Performance Support Systems: Comparison Of Types Of Integration Levels On Performance Outcomes" (2013). Wayne State University Dissertations. 739.
https://digitalcommons.wayne.edu/oa_dissertations/739
Data used in producing this dissertation