In the second mission of this course, we learned about holdout validation. Holdout validation allows us to test a machine learning model's accuracy on new data it hasn't seen before. Holdout validation is a good approach if you're starting out or testing the waters with different types of models, but there are more sophisticated forms of cross-validation that we can use.

In this mission, we'll learn about k-fold cross validation, implement and check the output of 5-fold cross validation, and discuss how we can best choose the number of folds to use.

Objectives

  • Learn about k-fold cross validation
  • Determine how many folds to choose for particular applications

Lesson Outline

  1. Weaknesses of holdout validation
  2. k-fold cross validation
  3. Using k-fold cross validation
  4. Checking the output of 5-fold cross validation
  5. To use less or more folds?
  6. Exploring different k values
  7. Next steps
  8. Takeaways