In the second mission of this course, we learned about holdout validation. Holdout validation allows us to test a machine learning model's accuracy on new data it hasn't seen before. Holdout validation is a good approach if you're starting out or testing the waters with different types of models, but there are more sophisticated forms of cross-validation that we can use.
In this mission, we'll learn about k-fold cross validation, implement and check the output of 5-fold cross validation, and discuss how we can best choose the number of folds to use.
Objectives
Lesson Outline
- Weaknesses of holdout validation
- k-fold cross validation
- Using k-fold cross validation
- Checking the output of 5-fold cross validation
- To use less or more folds?
- Exploring different k values
- Next steps
- Takeaways