In a study, analysts often need to compare treatment means to determine which treatments are better or worse and by how much. There are a variety of ways to compare treatment means. Some of the most common are all pairwise comparisons of treatment means, comparisons of all treatment means against a control, and comparisons of all treatment means to the overall average.
The new Multiple Comparisons option in Fit Model Least Squares in JMP 11 makes these type of multiple comparisons easy and accessible. This blog post will demonstrate using Multiple Comparisons to compare all treatment means with the overall average, also known as Analysis of Means or ANOM for short.
We will look at an example from The Analysis of Means (Nelson et al., 2005) where researchers conducted a trial to determine how two types of cancer therapy (chemo and radiation) and three types of cancer drugs affect hemoglobin levels in males. Since no interactions were found to be significant, we only include the main effects of the two factors in our model. Figure 1 shows how we set up our model in in the Fit Model launch dialog in JMP.
We would like to know if either cancer therapy treatment mean is significantly different from the overall average or if any of the three cancer drug treatment means is significantly different from the overall average at a 95% confidence level. We can answer this question by using the “Multiple Comparisons” option under the “Estimates” submenu of the Red Triangle menu. Figure 2 shows the Multiple Comparisons launch dialog with the default estimate type, “Least Squares Means Estimates,” and the effect, “Therapy,” selected, and the Comparison, “Comparisons with Overall Average,” checked.
The resulting Multiple Comparisons report is shown in Figure 3. The least squares means estimates for Therapy are displayed first. (Note: Because the Fit Model Least Squares platform can handle any type of linear model, we estimate least squares means instead of simple means because there can be other effects in the model.) Next, the Therapy least square means estimates are compared to the overall average. The differences between the least squares means estimates and the overall average are shown in a table and displayed graphically in the Comparisons with Overall Average Decision Chart (an ANOM chart).
An ANOM chart looks similar to a control chart but has a different purpose and interpretation. In an ANOM chart, the central line represents the overall average. The treatment means are plotted as deviations from the overall average and are compared to the upper and lower decision limits, which are the top and bottom lines. The treatment means that fall beyond the decision limits are statistically significantly different from the overall average. ANOM charts are in some ways nicer than ANOVA effect tests because you can easily see which estimates are statistically different from the overall average. It is also easy to determine practical significance of the differences, such as which treatment is significantly lower or higher than the overall average if statistical significance has been found.
In the Comparisons with Overall Average Chart in Figure 3, we see that both the chemo and radiation least squares means are significantly different from the overall average. The hemoglobin level with radiation is significantly lower than the overall average, and the hemoglobin level with chemo is significantly higher than the overall average.
Now we will look at the Multiple Comparison results for the Drug effect. Once again, we choose the “Multiple Comparisons” option under the “Estimates” submenu and choose the same options as Figure 2 except that we choose “Drug” as the effect instead of “Therapy.” The resulting Multiple Comparisons report is shown in Figure 4. Examining this Comparisons with Overall Average Decision Chart, we see that least squares mean for Drug 3 falls outside of the decision limits and is significantly different from the overall average. The least squares means for Drug 1 and Drug 2 fall within the decision limits, and we do not conclude that they are significantly different from the overall average. The hemoglobin level using Drug 3 is significantly higher than the overall average hemoglobin level.
This example demonstrated using the new Multiple Comparison option in Fit Model Least Squares to compare treatment means to the overall average, but Multiple Comparisons has many more options and can also handle more complicated models. Look for my upcoming blog posts where we will examine different types of models and multiple comparisons.
Nelson, P.R., Wludyka, P. ., and Copeland, K.A.F., The Analysis of Means: A Graphical Method for Comparing Means, Rates, and Proportions. ASA-SIAM Series on Statistics and Applied Probability, SIAM, Philadelphia, ASA, Alexandria, 2005.