In this course, you’ll learn how to reduce the memory footprint of a pandas DataFrame while working with data from the Museum of Modern Art. You’ll learn how to work with DataFrame chunks, how to use them to increase processing speed in pandas, and how to optimize DataFrame types while exploring data from the Lending Club. You’ll also learn how to augment pandas with SQLite to combine the best of both tools. Finally, you’ll learn when to use disk space over in-memory space, as well as how to run SQL queries using pandas.
Best of all, you’ll learn by doing — you’ll practice and get feedback directly in the browser. At the end of the course, you’ll complete a project that asks you to work on a real-life example — using the pandas SQLite workflow to analyze startup fundraising deals using data from CrunchBase.
- Reducing the memory footprint of a pandas DataFrame
- Processing large DataFrames in chunks using SQLite
Processing Large Datasets In Pandas [7 lessons]
- Expland your portfolio with a DataFrame chunking project
Projects in this course
Guided Project: Practice Optimizing DataFrames and Processing in Chunks
Practice optimizing DataFrame types and working in chunks.
The Dataquest guarantee
Dataquest has helped thousands of people start new careers in data. If you put in the work and follow our path, you’ll master data skills and grow your career.
We believe so strongly in our paths that we offer a full satisfaction guarantee. If you complete a career path on Dataquest and aren’t satisfied with your outcome, we’ll give you a refund.