Post Hoc Procedures

When a researcher uses ANOVA to test the null hypothesis that three means are the same, the resulting statistically significant F ratio tells the researcher only that two or more of the means are different. Usually the researcher needs to employ further statistical tests that will indicate those means that are different from each other. These tests are called post hoc comparisons.

There are five common multiple comparison tests: Fisher’s LSD, Duncan’s new multiple range test, the Newman-Keuls multiple range test, Tukey’s HSD, and the Scheffé test. Each test is used in the same way, but they differ in the ease with which a significant difference is obtained; for some tests, that is, the means need to be farther apart than for other tests for the difference to be statistically significant. Tests that require a greater difference between the means are said to be conservative, while those that permit less difference are said to be liberal. The listing of the tests above is sequential, with Fisher’s test considered most liberal and Scheffé’s test most conservative. The two most common tests are Tukey and Scheffé, but different conclusions can be reached in a study depending on the multiple comparison technique employed.