Power Analysis
Uses
- retrospective: to judge what the present analysis
died of
- prospective: to provide a basis for decisions on experimental
design of a planned study by suggesting for example the minimum
sample size necessary for driving down the error variance sufficiently
so a given efect size may be picked up statistically at a given
a (i.e., least significant number)
How this is done
When drawing conclusions from the results of a statistical
analysis we can commit one of two errors:
- Alpha error: false positive, reject Ho
when you should not reject it
- Beta error: false negative, fail to reject Ho
when you actual should reject it
Power analysis provides a quantitative method to estimate
the probability of obtaining a significant results in a given
situation (i.e., rejecting the null hypothesis when the alternative
hypothesis is actually true). Power analysis thus allows you to
judge how likely it is that you will detect a treatment effect
that really exists. In summary, power measures the probability
of replicating your statistical decision given a repeated use
of the same experimental design and sampling procedure. Power
is affected by:
- Level of Significance (a):
changing a is not a useful choice
for fending off low power. By changing a
you are not making better decisions, you are just substituting
one error (a) for the other (b).
- Raw Effect Size (d): Predicting
true effect size is not trivial. You may use a combination of
approaches, including empirical data from the related literature,
pilot data, or a general consideration of the smallest effects
that you would regard as biologically meaningful in your scenario.
Within an ANOVA setting you can obtain an estimate for d as the square root of your model SS divided
by total sample size (i.e., SQRT [SSM / N] )
- Standard Deviation of the Residuals (s):
The power of your analysis increases as the error variance decreases.
Towards that goal you can reduce treatment variability or control
for confounding factors. Obtain an estimate for s
in an ANOVA setting as the square root of the error variance
(MS error).
- Sample Size (N): Standard deviation of the random
error decreases with increasing sample size.
Links:
last modified: 3/26/14