Evaluating Model Performance
In the last mission, we started learning the machine learning workflow. We did a bit of exploration of Airbnb's Washington D.C listing data set and laid out the problem we wanted to solve with the data: if we had a new listing, how could we use the data to predict an acceptable rental price for it?
We ended the last mission predicting a price for a single listing with three rooms. However, we currently have no way of knowing if this prediction was good or not.
In this mission, we'll follow up on this question and learn how to evaluate the performance of our k-nearest neighbors algorithm. We'll define what we mean by performance, and then look at how to calculate metrics to judge whether the model is "good" or not.
We'll start learning an incredibly handy R library called
caret, which is used for creating machine learning models and automating the process of evaluating their performance as well. Instead of having to code everything by hand, we'll learn how to use the
caret library to perform the various steps of the machine learning workflow.
- Judging performance
- Introducing the caret library
- Setting up for training
- Training the algorithm
- Create predictions on the test data
- Evaluating predictions
- Summarizing errors into a single metric
- Next steps