For testing the equality of several treatment effects in a one-way fixed effects model, or for testing the significance of the treatment variance component in a one-way random effects model, the usual F test is appropriate when error variances are assumed to be equal. When this assumption is violated, the F test may not be appropriate. Many alternative tests have been suggested in the literature. When applied to actual data, the different tests can yield drastically different p-values and opposing conclusions. This brings up the issue of which test should be chosen for practical use. To address this, the different tests are compared in terms of their Type I error probability and power, estimated by Monte Carlo simulation. It turns out that there are scenarios where many of the tests have Type I error probabilities far greater than the nominal level. Based on the numerical results, recommendations are made on the choice of the test for practical use. The results are applied to several examples.