Simple examples of power with binomial testing

Suppose we want to do another chocolate taste comparison experiment, using Sam's Choice and Hershey's Cocoa Reserve as the alternative, which is the Consumer Reports top rated gourmet chocolate in a recent study. We let success='prefers Hersheys' in this case.

We have H_0: p = .5 versus H_1: p > .5. We choose alpha=.01.

Question: Find the critical value, x*, if the sample size is n=10; is n=19.

Answer: For n = 10:

qbinom(.99,10,.5)
yields an answer of 9, but notice the following:

1-pbinom(8,10,.5)
[1] 0.01074219

1-pbinom(9,10,.5)
[1] 0.0009765625

These results would suggest that we use x*=10, since that is the smallest value for which the true alpha is less than or equal to .01; but we will break convention here since .01074219 is so close to .01 and use x*=9.

The n=19 case is more straightforward:

qbinom(.99,19,.5)
[1] 14

1-pbinom(13,19,.5)
[1] 0.03178406

1-pbinom(14,19,.5)
[1] 0.009605408

Which says that x* for this test would be 15.

Question: Now, suppose that the true proportion of students who prefer Hersheys is .8. What would be the probability the test rejects H_0 in favor of H_1; such a result would be a correct decision, since H_1 is true if p=.8. Compute this probability for both the n=10 test and the n=19 test.

Answer: Here are the answers through R:

1-pbinom(8,10,.8)
[1] 0.3758096

1-pbinom(14,19,.8)
[1] 0.6732881

The language we use is this: "With n=10, the .01-level test has a power of .376 for a true p of .8. With n=19, the .01-level test has a power of .673 for a true p of .8."

Pictures and R code to go along with n=10 and n=19 Type II error discussion.

x <- 0:20
y10.0 <- dbinom(x,10,.5)
y10.1 <- dbinom(x,10,.8)

y19.0 <- dbinom(x,19,.5)
y19.1 <- dbinom(x,19,.8)

par(mfrow=c(2,1)) # for two graphs in window

plot(x,y10.0,type='h')
points(x+.1,y10.1,type='h',lty=3) # fudged the plot right a tad

plot(x,y19.0,type='h')
points(x+.1,y19.1,type='h', lty=3)  # fudged the plot right a tad

par(mfrow=c(1,1))

# end of code for these plots

Power curves

Here is a simple example of having R draw power curves. Here we have n Bernoulli trials, p=success probability, we are using an alpha of approximately .01. When n=10, the critical value is 9. When n=19, the critical value is 15. The true alphas for these 2 cases are .01074 and .0096, which are close enough to consider them .01-level tests. We want to compare two tests with the same alpha; hence the rare appearance of the number 19 in a classroom example.

# R code for drawing power curves with n=10 and n=19
p <- seq(.5, 1, .01)  # set up the p axis.
pi <- 1-pbinom(8,10,p)  # Compute power values (pi) at each p.
plot(p,pi,type='l',tck=1)

pi2 <- 1 - pbinom(14,19,p)  # The power function with n=19.
lines(p,pi2,lty=2)  # lines adds a figure to an existing plot.

# end of code for these plots