Education Commons^™

The Problem Of Too Many Statistical Tests: Subgroup Analyses In A Study Comparing The Effectiveness Of Online And Live Lectures, David M. Lane

Numeracy

The more statistical analyses performed in the analysis of research data, the more likely it is that one or more of the conclusions will be in error. Multiple statistical analyses can occur when the sample contains several subgroups and the researchers perform separate analyses for each subgroup. For example, separate analyses may be done for different ethnic groups, different levels of education, and/or for both genders. Media reports of research frequently omit information on the number of subgroup analyses performed thus leaving the reader with insufficient information to assess the validity of the conclusions. This article discusses the problems with …