Type I Errors, Type II Errors, and Power
Define Type I error (reject a true H₀, probability = α), Type II error (fail to reject a false H₀, probability = β), and power (1 − β). Understand how changing α, n, and the true parameter value affect power.
A lesson of AP Statistics, in Inference for Categorical Data: Proportions, about 10 minutes.
Type I and Type II Errors in Hypothesis Testing
By the end of this lesson, you will be able to define Type I and Type II errors, explain the relationship between significance level, power, and error rates, and evaluate the consequences of each error type in context.
The Decision Framework
In hypothesis testing, we make a binary decision: reject H₀ or fail to reject H₀. Since we're working with sample data, our decision could be wrong.
| H₀ is TRUE | H₀ is FALSE | |
|---|---|---|
| Reject H₀ | Type I Error (False Positive) | Correct Decision |
| Fail to Reject H₀ | Correct Decision | Type II Error (False Negative) |
Type I Error
Definition: Rejecting H₀ when H₀ is actually true.
Also called: False positive, false alarm
Probability: P(Type I Error) = α (the significance level)
When you set α = 0.05, you're accepting a 5% chance of committing a Type I error.
Example: A new medication has no actual effect, but based on your sample, you conclude it does work. You've rejected a true null hypothesis.
Consequences of Type I errors:
- In medicine: approving an ineffective treatment, wasting resources, potentially exposing patients to side effects
- In quality control: shutting down a machine that's actually working fine
- In criminal justice analogy: convicting an innocent person
Type II Error
Definition: Failing to reject H₀ when H₀ is actually false.
Also called: False negative, miss
Probability: P(Type II Error) = β (beta)
Example: A new medication does work, but your sample doesn't provide sufficient evidence to conclude it does. You fail to detect a real effect.
Consequences of Type II errors:
- In medicine: failing to approve an effective treatment
- In quality control: missing a defective machine