Significance Testing

In this lesson, we'll learn about hypothesis testing and statistical significance. A hypothesis is a pattern or rule about a process that can be tested, and hypothesis testing helps us determine if a change we made had a meaningful impact or not.

For example, hypothesis testing can help you determine:

  • If a new banner ad on a website caused a meaningful drop in user engagement.
  • If raising the price of a product caused a meaningful drop in sales, or
  • If a new weight loss pill helped people lose more weight.

Just because user engagement or sales decreased after adding a new banner doesn’t necessarily mean the banner caused the decline -- it could have been random chance. Every process has an inherent amount of randomness, and hypothesis testing helps us better understand the role of chance so we can improve our conclusions.

As with all our courses, you can apply what you’ve learned in our in-browser exercises, and automatic answer checking will help you know whether you’ve mastered each concept.


  • Learn how hypothesis testing works.
  • Learn how statistical significance and hypothesis testing are related.

Mission Outline

1. Hypothesis testing
2. Research design
3. Statistical significance
4. Test statistic
5. Permutation test
6. Sampling distribution
7. Dictionary representation of a distribution
8. P value
9. Caveats
10. Next steps
11. Takeaways


Course Info:


The median completion time for this course is 6.49 hours. ​View Details​​​

This course requires a Basic subscription. It includes six missions, and one guided project. This course is 17th course in the Data Analyst in Python path and Data Scientist in Python path


Take a Look Inside