Many aspiring data scientists focus on doing Kaggle competitions as a way to build their portfolios. Kaggle is an excellent way to practice, but it should only be one of many avenues you use to work on data science projects. This is because Kaggle competitions only focus on a narrow part of data science work. To be more specific: Kaggle mostly deals with machine learning, which is only one aspect of Data Science. When you... »
If you’re in the early phases of learning SQL and have completed one or more introductory-level courses, you’ve probably learned most of the basic fundamentals and possibly even some high-level database concepts. As you prepare to embark on the next phase of learning SQL, it’s important to not only understand SQL itself, but also the engine that makes it all possible: the database. In most introductory-level courses, you’ll typically use some sort of embedded database... »
Tips for reducing memory usage by up to 90% When working using pandas with small data (under 100 megabytes), performance is rarely a problem. When we move to larger data (100 megabytes to multiple gigabytes), performance issues can make run times much longer, and cause code to fail entirely due to insufficient memory. While tools like Spark can handle large data sets (100 gigabytes to multiple terabytes), taking full advantage of their capabilities usually requires... »
The printable version of this cheat sheet It’s common when first learning Python for Data Science to have trouble remembering all the syntax that you need. While at Dataquest we advocate getting used to consulting the Python documentation, sometimes it’s nice to have a handy reference, so we’ve put together this cheat sheet to help you out! If you’re interested in learning Python, we have a free Python Programming: Beginner course which can start you... »
Image Credit: DigitalOcean One of the biggest sources of confusion and misinformation for people wanting to learn Python is which version they should learn. Should I learn Python 2.x or Python 3.x? Indeed, this is one of the questions we are asked most often at Dataquest, where we teach Python as part of our Data Science curriculum. This post gives some context behind the question, explains the pespective, and tells you which version you should... »
SettingWithCopyWarning is one of the most common hurdles people run into when learning pandas. A quick web search will reveal scores of Stack Overflow questions, GitHub issues and forum posts from programmers trying to wrap their heads around what this warning means in their particular situation. It’s no surprise that many struggle with this; there are so many ways to index pandas data structures, each with its own particular nuance, and even pandas itself does... »
To source data for data science projects, you’ll often rely on SQL and NoSQL databases, APIs, or ready-made CSV data sets. The problem is that you can’t always find a data set on your topic, databases are not kept current and APIs are either expensive or have usage limits. If the data you’re looking for is on an web page, however, then the solution to all these problems is web scraping. In this tutorial we’ll... »
I learned machine learning through competing in Kaggle competitions. I entered my first competitions in 2011, with almost no data science knowledge. I soon ended up in fifth place out of a hundred or so in a stock trading competition. Over the next year, I won several competitions on automated essay scoring and bond price prediction, and placed well in others. Kaggle competitions require a unique blend of skill, luck, and teamwork to win. The... »
Data is increasingly shaping the systems that we interact with every day. Whether you’re using Siri, searching Google, or browsing your Facebook feed, you’re consuming the results of data analysis. Given its transformational ability, it’s no wonder that so many data-related roles have been created in the past few years. The responsibilities of these roles range from predicting the future, to finding patterns in the world around you, to building systems that manipulate millions of... »
SQL, pronounced “sequel” (or ess-cue-ell, if you prefer), is a very important tool for data scientists to have in their repertoire. You may well have heard the name and wondered what it is, how it works and whether you should learn it. To put it simply, SQL (Structured Query Language) is the language of databases and almost all companies use databases to store their data. Because of this, no matter whether you prefer to use... »
Founded in 2010, Kaggle is a Data Science platform where users can share, collaborate, and compete. One key feature of Kaggle is “Competitions”, which offers users the ability to practice on real world data and to test their skills with, and against, an international community. This guide will teach you how to approach and enter a Kaggle competition, including exploring the data, creating and engineering features, building models, and submitting predictions. We’ll use Python 3... »
Our version 1.19 release includes new features designed to improve your learning experience. The first thing you may notice is a new look. We’ve made some design tweaks, including a new mission-text font which we think you’ll agree makes everything easier to read. Other big changes in v1.19 include: Multiscreen, so you can work more efficiently in-mission. Concepts, designed to help you take your learning to the next level. In-Mission Dataset Preview and Download. You... »
Data science is one of the most buzzed about fields right now, and data scientists are in extreme demand. And with good reason – data scientists are doing everything from creating self-driving cars to automatically captioning images. Given all the interesting applications, it makes sense that data science is a very sought-after career. Data science is applied in many field, including in developing self-driving cars. If you’re reading this post, I’m assuming that you’d like... »
NumPy is the library that gives Python its ability to work with data at speed. Originally, launched in 1995 as ‘Numeric,’ NumPy is the foundation on which many important Python data science libraries are built, including Pandas, SciPy and scikit-learn. The printable version of this cheat sheet It’s common when first learning NumPy to have trouble remembering all the functions and methods that you need, and while at Dataquest we advocate getting used to consulting... »