Access Type

Open Access Dissertation

Date of Award

January 2014

Degree Type

Dissertation

Degree Name

Ph.D.

Department

Education Evaluation and Research

First Advisor

Barry Markman

Abstract

When conducting a statistical test the initial risk that must be considered is a Type I error, also known as a false positive. The Type I error rate is set by nominal alpha, assuming all underlying conditions of the statistic are met. Experiment-wise Type I error inflation occurs when multiple tests are conducted overall for a single experiment. There is a growing trend in the social and behavioral sciences utilizing nested designs. A Monte Carlo study was conducted using a two layer design. Five theoretical distributions and four real datasets taken from Micceri (1989) were used, each with five different samples sizes and conducted with nominal alpha set to 0.05 and 0.01. These were conducted both unconditionally and conditionally. All permutations were executed for 1,000,000 repetitions. It was found that when conducted unconditionally, the experiment-wise Type I error rate increases from alpha = 0.05 to 0.10 and 0.01 increases to 0.02. Conditionally, it is extremely unlikely to ever find results for the factor, as it requires a statistically significant nest as a precursor, which leads to extremely reduced power. Hence, caution should be used when interpreting nested designs.

Download

Share

COinS