fundamental: alpha and beta

Fundamental stuff to know...

In research, statistical tests don’t prove an alternate hypothesis is true.  Instead, we use statistical analysis to provide evidence that supports rejecting or failing to reject the null hypothesis.       

P-value is a measure of the strength of evidence against the null hypothesis.  Alpha (α) is the cutoff for the p-value.  Alpha = .05 in my research, which is standard for most dissertations.    

 

• when p-value < α (in my case, α=.05), reject the null hypothesis
• when p-value > or =  α, we do not have evidence to reject the null hypothesis

When you draw conclusions about a population based on a sample, you have opportunity for error.  You hope your sample represents the population, but our world is not perfect...damnit.

• ·         Type I error = α = probability of rejecting a null hypothesis when you shouldn’t.  We reject a null hypothesis that is true.  You see effect when there really isn’t any effect.  It’s a false positive...we reject a true null hypothesis.    
• ·         Type II error =  β = the probability of not rejecting a null hypothesis, i.e. we accept a null hypothesis when we should not accept it (it is false)...we accept a false null hypothesis.  Effect is there, but we don’t find it.      

You can reduce the likelihood of committing a Type I error by making alpha smaller.  You can reduce the likelihood of committing a Type II error by increasing the sample size. 

The power of the hypothesis test = 1 – probability of committing a Type II error = probability of rejecting a null hypothesis when it should be rejected.  The power represents the likelihood of detecting effect that is real.