MISSION 91

Spark DataFrames

Learn the basics of Spark DataFrames by working with census data.

Objectives

  • Learn how to work with Spark dataframes.
  • Learn the difference between pandas and Spark dataframes.
  • Learn to perform basic filters with Spark dataframes.

Mission Outline

1. The Spark DataFrame: An Introduction
2. Reading in Data
3. Schema
4. Pandas vs Spark DataFrames
5. Row Objects
6. Selecting Columns
7. Filtering Rows
8. Using Column Comparisons as Filters
9. Converting Spark DataFrames to pandas DataFrames
10. Next Steps
11. Takeaways

Course Info:

Spark and Map-Reduce

Intermediate

The average completion time for this course is 10-hours.

This course requires a premium subscription and includes four paid missions, one challenge, and one installation tutorial.  It is the 31st course in the Data Scientist In Python path.

START LEARNING FREE

Take a Look Inside

Share On Facebook
Share On Twitter
Share On Linkedin
Share On Reddit