Simple examples of type-II error, beta, and power

Suppose we want to do another chocolate taste comparison experiment, using Sam's Choice and Hershey's Cocoa Reserve as the alternative, which is the Consumer Reports top rated gourmet chocolate in a recent study. We let success='prefers Hersheys' in this case. We will use a random sample of n=100 students from Grinnell College.

We have H_0: p = .5 versus H_1: p > .5. We choose alpha=.05.

Question: Find the critical value, x*, if the sample size is n=100.

Answer: Note that:

qbinom(.95,100,.5)
yields an answer of 58, but we notice the following:

1-pbinom(57,100,.5)
[1] 0.06660531


1-pbinom(58,10,.5)
[1] 0.04431304

These results would suggest that we use x*=59, since that is the smallest value for which the true alpha is less than or equal to .05. In class, we used the CLT (normal approximation) to estimate x*.

Thus, alpha=P(reject H_0 | H_0 is true) = .0443. Let's now consider beta=P(type II error)=P(not rejecting H_0 | H_1 is true), which we realize is a function of the particular value of p that makes H_1 true. Let us use R for the computations:


p <- seq(.5,1,length=11)
beta <- pbinom(58,100,p)  # getting 58 or fewer successes leads to
                  # not rejecting H_0.

round(cbind(p,beta),3)

         p   beta
 [1,] 0.50 0.9557
 [2,] 0.55 0.7585
 [3,] 0.60 0.3775
 [4,] 0.65 0.0877
 [5,] 0.70 0.0072
 [6,] 0.75 0.0001
 [7,] 0.80 0.0000
 [8,] 0.85 0.0000
 [9,] 0.90 0.0000
[10,] 0.95 0.0000
[11,] 1.00 0.0000

We see that for p near .50 the chance of making a type II error is high, but it tails off as p moves away from .5 and it essentially 0 when p is at or above .80.

The complement of a type II error is to make the correct decision of rejecting H_0 when H_1 is true. The probability of this event is called power and we will compute it and graph the value for the power function.


p <- seq(.5,1,length=101)
power <- 1-pbinom(58,100,p) # rejecting H_0 means X >= 59
plot(p,power, type='l', tck=2)

Question: Suppose you are interested in the power when the true proportion who favor Hershey's is .60. Use the graph to estimate the power.