MISSION 163

Optimizing DataFrame Memory Footprint

Learn how to reduce a DataFrame's memory footprint be selecting the correct data types.

Objectives

  • Learn how to use pandas for large data sets.
  • Learn how much memory pandas' datasets use.
  • Learn how to optimize pandas data types.

Mission Outline

1. Introduction
2. How Pandas Represents Values in a Dataframe
3. Different Types Have Different Memory Footprints
4. Calculating the True Memory Footprint
5. Optimizing Integer Columns With Subtypes
6. Optimizing Float Columns With Subtypes
7. Converting To DateTime
8. Converting to Categorical to Save Memory
9. Selecting Types While Reading the Data In
10. Next Steps
11. Takeaways

pandas-large-datasets

Course Info:

Processing Large Datasets In Pandas

Intermediate

The average completion time for this course is 10-hours.

This course requires a premium subscription and includes 2 free missions, 2 paid missions, and 1 guided project.  It is the 3rd course in the Data Engineer path.

START LEARNING FREE

Take a Look Inside