Off-campus WSU users: To download campus access dissertations, please use the following link to log into our proxy server with your WSU access ID and password, then click the "Off-campus Download" button below.
Non-WSU users: Please talk to your librarian about requesting this thesis through interlibrary loan.
Access Type
WSU Access
Date of Award
January 2025
Degree Type
Thesis
Degree Name
M.S.
Department
Psychology
First Advisor
Cort W. Rudolph
Second Advisor
Andrew B. Speer
Abstract
This research investigated advances in natural language processing (NLP) techniques for measuring work attitudes and perceptions from organizational text. Traditional Bag-of-Words (BOW) approaches, while widely used, fail to capture semantic nuances and contextual meanings critical for accurate theme identification. This study compared BOW with three alternative approaches: Fully Fine-Tuned Standard Transformer-Based Theme Score Algorithm (FAST), Partially Fine-Tuned Standard Transformer-Based Theme Score Algorithm (PAST), and Generative Automated Theme Scoring (GATS) with zero-shot and few-shot configurations. Using a dataset of approximately 5,000 employee narratives across 21 work-related constructs, models were evaluated against subject matter expert (SME) ratings. Results demonstrated that FAST significantly outperformed BOW across multiple metrics, while PAST revealed diminishing returns as labeled data increased. Construct complexity moderated this relationship, with less complex constructs requiring fewer examples to reach optimal performance. Zero-shot GATS demonstrated strong performance without task-specific training, while few-shot GATS with 3-12 examples yielded minimal additional improvements. These findings advance organizational science by providing empirically validated approaches for efficiently analyzing qualitative employee data at scale. Practically, organizations can strategically allocate resources when implementing these methods, selecting appropriate approaches based on available data, computational resources, and construct complexity. This research contributes to more accurate, efficient, and accessible measurement of work attitudes and perceptions from narrative data.
Recommended Citation
Perrotta, James, "From Bow To Llms: Advancing Work Attitudes And Perceptions Measurement From Organizational Text" (2025). Wayne State University Theses. 1009.
https://digitalcommons.wayne.edu/oa_theses/1009