It's an exciting time for data science. The field is new, but growing quickly. There's huge demand for data scientists — average compensation in SF is well north of 100 thousand dollars a year. Where there's money, there are also people trying to earn it. The data science skills gap means that many people are learning or trying to learn data science.
The first step to learning data science is usually asking "how do I learn data science?". The response to this question tends to be a long list of courses to take and books to read, starting with linear algebra or statistics. I went through this myself a few years ago when I was learning. I had no programming background, but knew that I wanted to work with data.
I can't fully explain how immensely unmotivating it is to be given a huge list of resources without any context. It's akin to a teacher handing you a stack of textbooks and saying "read all of these". I struggled with this approach when I was in school. If I had started learning data science this way, I never would have kept going.
Some people learn best with a list of books, but I learn best by building and trying things. I learn when I'm motivated, and when I know why I'm learning something. Best of all, when you learn this way, you come out with immediately useful skills. From my conversations with new learners over the years, I know many share these views.
That's why I don't think your first goal should be to learn linear algebra or statistics. If you want to learn data science, your first goal should be to learn to love data. Interested in finding out how? Read on to see how to actually learn data science.
An example of the visualizations you can make with data science (via The Economist)
1. Learn to love data
Nobody ever talks about motivation in learning. Data science is a broad and fuzzy field, which makes it hard to learn. Really hard. Without motivation, you'll end up stopping halfway through and believing you can't do it, when the fault isn't with you — it's with the teaching.
You need something that will motivate you to keep learning, even when it's midnight, formulas are starting to look blurry, and you're wondering if this will be the night that neural networks finally make sense.
You need something that will make you find the linkages between statistics, linear algebra, and neural networks. Something that will prevent you from struggling with the "what do I learn next?" question.
My entry point to data science was predicting the stock market, although I didn't know it at the time. Some of the first programs I coded to predict the stock market involved almost no statistics. But I knew they weren't performing well, so I worked day and night to make them better.
I was obsessed with improving the performance of my programs. I was obsessed with the stock market. I was learning to love data. And because I was learning to love data, I was motivated to learn anything I needed to make my programs better.
Not everyone is obsessed with predicting the stock market, I know. But it's important to find that thing that make you want to learn.
It can be figuring out new and interesting things about your city, mapping all the devices on the internet, finding the real positions NBA players play, mapping refugees by year, or anything else. The great thing about data science is that there are infinite interesting things to work on -- it's all about asking questions and finding a way to get answers.
Take control of your learning by tailoring it to what you want to do, not the other way around.
A map of all the devices on the internet
2. Learn data science by doing
Learning about neural networks, image recognition, and other cutting-edge techniques is important. But most data science doesn't involve any of it:
- 90% of your work will be data cleaning.
- Knowing a few algorithms really well is better than knowing a little about many algorithms.
- If you know linear regression, k-means clustering, and logistic regression well, can explain and interpret their results, and can actually complete a project from start to finish with them, you'll be much more employable than if you know every single algorithm, but can't use them.
- Most of the time, when you use an algorithm, it will be a version from a library (you'll rarely be coding your own SVM implementations -- it takes too long).
What all of this means is that the best way to learn is to work on projects. By working on projects, you gain skills that are immediately applicable and useful. You also have a nice way to build a portfolio.
One technique to start projects is to find a dataset you like. Answer an interesting question about it. Rinse and repeat.
Here are some good places to find datasets to get you started:
Another technique (and my technique) was to find a deep problem, predicting the stock market, that could be broken down into small steps. I first connected to the yahoo finance API, and pulled down daily price data. I then created some indicators, like average price over the past few days, and used them to predict the future (no real algorithms here, just technical analysis). This didn't work so well, so I learned some statistics, and then used linear regression. Then I connected to another API, scraped minute by minute data, and stored it in a SQL database. And so on, until the algorithm worked well.
The great thing about this is that I had context for my learning. I didn't just learn SQL syntax -- I used it to store price data, and thus learned 10x as much as I would have by just studying syntax. Learning without application isn't going to be retained very well, and won't prepare you to do actual data science work.
This guy's trying to predict the stock market, but needs some data science, apparently (via DailyMail)
3. Learn to communicate insights
Data scientists constantly need to present the results of their analysis to others. Skill at doing this can be the difference between an okay and a great data scientist.
Part of communicating insights is understanding the topic and theory well. Another part is understanding how to clearly organize your results. The final piece is being able to explain your analysis clearly.
It's hard to get good at communicating complex concepts effectively, but here are some things you should try:
- Start a blog. Post the results of your data analysis.
- Try to teach your less tech-savvy friends and family about data science concepts. It's amazing how much teaching can help you understand concepts.
- Try to speak at meetups.
- Use github to host all your analysis.
- Get active on communities like Quora, DataTau, and the machine learning subreddit.
4. Learn from your peers
It's amazing how much you can learn from working with others. In data science, teamwork can also be very important in a job setting.
Some ideas here:
- Find people to work with at meetups.
- Contribute to open source packages.
- Message people who write interesting data analysis blogs seeing if you can collaborate.
- Try out Kaggle, a machine learning competition site, and see if you can find a teammate.
5. Constantly increase the degree of difficulty
Are you completely comfortable with the project you're working on? Was the last time you used a new concept a week ago? It's time to work on something more difficult. Data science is a steep mountain to climb, and if you stop climbing, it's easy to never make it.
If you find yourself getting too comfortable, here are some ideas:
- Work with a larger dataset. Learn to use spark.
- See if you can make your algorithm faster.
- How would you scale your algorithm to multiple processors? Can you do it?
- Understand the theory of the algorithm you're using more. Does this change your assumptions?
- Try to teach a novice to do the same things you're doing now.
The bottom line
This is less a roadmap of exactly what to do that it is a rough set of guidelines to follow as you learn data science. If you do all of these things well, you'll find that you're naturally developing data science expertise.
I generally dislike the "here's a big list of stuff" approach, because it makes it extremely hard to figure out what to do next. I've seen a lot of people give up learning when confronted with a giant list of textbooks and MOOCs.
I personally believe that anyone can learn data science if they approach it with the right frame of mind.
I'm also the founder of Dataquest, a site that helps you learn data science in your browser. It encapsulates a lot of the ideas discussed in this post to create a better learning experience. You learn by analyzing interesting datasets like CIA documents and NBA player stats. You also complete projects and build a portfolio. It's not a problem if you don't know how to code -- we teach you python. We teach python because it's the most beginner-friendly language, is used in a lot of production data science work, and can be used for a variety of applications.
Some helpful resources
As I worked on projects, I found these resources helpful. Remember, resources on their own aren't useful -- find a context for them:
- Dataquest -- learn data science in your browser, complete projects, and build a portfolio.
- Khan Academy -- good basic statistics and linear algebra content.
- Introduction to Linear Algebra, 4th Edition -- Great linear algebra book by Gilbert Strang.
- Calculus Online Textbook -- also by Gilbert Strang, great calculus book.
- Elements of statistical learning -- good machine learning book.
- Andrew Ng’s Machine Learning Class -- the original coursera machine learning class. Mostly video-based.
- OpenIntro Statistics -- Good basic stats book.
- Google Scholar -- A paper can be a great way to learn about a topic. For example, here's Breiman's original random forest paper.
- Statsoft statistics textbook -- Good for looking up statistics concepts.
This post is adapted from my Quora answer on how to become a data scientist.