Adam

“Getting a data science job would have been much harder without Dataquest. It’s a great product. I recommend it to anyone who asks me about how to get started.”

Adam Zabrodski

Data Science Manager @ Later

Course overview

In this course, you’ll learn how to reduce the memory footprint of a pandas DataFrame while working with data from the Museum of Modern Art. You’ll learn how to work with DataFrame chunks, how to use them to increase processing speed in pandas, and how to optimize DataFrame types while exploring data from the Lending Club. You’ll also learn how to augment pandas with SQLite to combine the best of both tools. Finally, you’ll learn when to use disk space over in-memory space, as well as how to run SQL queries using pandas.

Best of all, you’ll learn by doing — you’ll practice and get feedback directly in the browser. At the end of the course, you’ll complete a project that asks you to work on a real-life example — using the pandas SQLite workflow to analyze startup fundraising deals using data from CrunchBase.

Key skills

  • Reducing the memory footprint of a pandas DataFrame
  • Processing large DataFrames in chunks using SQLite

Course outline

Processing Large Datasets In Pandas [7 lessons]

Introduction to Pandas 1h

Lesson Objectives
  • Identify how Pandas uses DataFrames to store data
  • Read a CSV into a DataFrame
  • Access data from a DataFrame

Calculating With Pandas 1h

Lesson Objectives
  • Represent 1-dimensional data using pandas
  • Perform computations
  • Select a subset of data using Boolean masks
  • Add new columns to a dataframe

Optimizing DataFrame Memory Footprint 2h

Lesson Objectives
  • Employ pandas with large datasets
  • Identify how much memory pandas datasets use
  • Optimize pandas datatypes

Processing Dataframes in Chunks 1h

Lesson Objectives
  • Employ DataFrame chunks
  • Increase processing speed in pandas

Guided Project: Practice Optimizing DataFrames and Processing in Chunks 1h

Lesson Objectives
  • Expland your portfolio with a DataFrame chunking project

Augmenting Pandas with SQLite 1h

Lesson Objectives
  • Employ pandas with SQLite
  • Determine whether to use disk space or in-memory space
  • Run SQL queries using pandas

Guided Project: Analyzing Startup Fundraising Deals from Crunchbase 1h

Lesson Objectives
  • Run the pandas and SQLite workflow on a new dataset
  • Analyze startup fundraising deals using a new workflow

Projects in this course

Guided Project: Practice Optimizing DataFrames and Processing in Chunks

Practice optimizing DataFrame types and working in chunks.

Guided Project: Analyzing Startup Fundraising Deals from Crunchbase

Practice analyzing data using the pandas SQLite workflow.

The Dataquest guarantee

Guarantee

Dataquest has helped thousands of people start new careers in data. If you put in the work and follow our path, you’ll master data skills and grow your career.

Money

We believe so strongly in our paths that we offer a full satisfaction guarantee. If you complete a career path on Dataquest and aren’t satisfied with your outcome, we’ll give you a refund.

Master skills faster with Dataquest

Go from zero to job-ready

Go from zero to job-ready

Learn exactly what you need to achieve your goal. Don’t waste time on unrelated lessons.

Build your project portfolio

Build your project portfolio

Build confidence with our in-depth projects, and show off your data skills.

Challenge yourself with exercises

Challenge yourself with exercises

Work with real data from day one with interactive lessons and hands-on exercises.

Showcase your path certification

Showcase your path certification

Impress employers by completing a capstone project and certifying it with an expert review.

Grow your career with
Dataquest.

98%
of learners recommend
Dataquest for career advancement
4.9
Dataquest rating on
G2Crowd and SwitchUp
$30k
Average salary boost
for learners who complete a path
Aaron

Aaron Melton

Business Analyst at Aditi Consulting

“Dataquest starts at the most basic level, so a beginner can understand the concepts. I tried learning to code before, using Codecademy and Coursera. I struggled because I had no background in coding, and I was spending a lot of time Googling. Dataquest helped me actually learn.”

Jessi

Jessica Ko

Machine Learning Engineer at Twitter

“I liked the interactive environment on Dataquest. The material was clear and well organized. I spent more time practicing then watching videos and it made me want to keep learning.”

Victoria

Victoria E. Guzik

Associate Data Scientist at Callisto Media

“I really love learning on Dataquest. I looked into a couple of other options and I found that they were much too handhold-y and fill in the blank relative to Dataquest’s method. The projects on Dataquest were key to getting my job. I doubled my income!”

Join 1M+ data learners on
Dataquest.

1

Create a free account

2

Choose a learning path

3

Complete exercises and projects

4

Advance your career

Start learning today