MISSION 91

Spark DataFrames

Learn the basics of Spark DataFrames by working with census data.

Objectives

  • Learn how to work with Spark dataframes.
  • Learn the difference between pandas and Spark dataframes.
  • Learn to perform basic filters with Spark dataframes.

Mission Outline

1. The Spark DataFrame: An Introduction
2. Reading in Data
3. Schema
4. Pandas vs Spark DataFrames
5. Row Objects
6. Selecting Columns
7. Filtering Rows
8. Using Column Comparisons as Filters
9. Converting Spark DataFrames to pandas DataFrames
10. Next Steps
11. Takeaways

spark-map-reduce

Course Info:

Intermediate

The median completion time for this course is 6 hours. View Details

This course requires a premium subscription and includes five missions, and one installation tutorial.  It is the 31st course in the Data Scientist In Python path.

START LEARNING FREE

Take a Look Inside