A beginner’s guide to statistical hypothesis tests

A simple introduction to hypothesis tests

Image by author

Statistics is a data scientist’s best friend. Hypothesis tests are a family of very useful tools for any kind of analysis. They can really help us assess the statistical significance of some phenomena. However, we have to master such tools properly in order to benefit from their advantages.

What is a hypothesis test?

A hypothesis test is a statistical test that assesses a particular hypothesis (called the “null” hypothesis) and calculates confidence about it. For example, if we calculate the height of Mount Everest (which we know to be 8848 meters) and we get 8845 meters, we would like to assess whether the difference between our measure and the known result is statistically significant. That’s the purpose of hypothesis tests. They make us calculate a confidence level and apply it to a particular hypothesis.

Here are the ingredients of a typical hypothesis test:

• The null hypothesis (e.g. our measure of Mount Everest height is statistically equivalent to the known one)
• A statistic value, that is a number calculated by the test and depends on the test type
• A p-value, that is the probability of having as extreme statistics as ours if the null hypothesis is true
• A confidence threshold, which is the minimum value of the p-value we allow not to reject the null hypothesis

The value of the statistics changes according to the type of the test and the p-value calculation changes as well. Usually, the p-value is calculated according to a particular probability distribution, related to the test.

Practically speaking, first of all, state your null hypothesis clearly. It’s a very important and often neglected step. Then, according to your test, calculate the statistics and the p-value. If it’s lower than your allowed threshold, reject the null hypothesis. If it’s greater than or equal to the threshold, you don’t reject the null hypothesis.

Left-tailed, right-tailed and two tails tests