In our discussion of data engineering, we have ventured from production-grade databases, the optimization of SQL tables, learning about data structures and algorithms, and then using these concepts to write better code. Within each of these topics, we have learned how they work in isolation, but not how they fit together as a whole.
This piece of the data engineering puzzle is building a data pipeline. A data pipeline is a sequence of tasks. Each task takes an input, and then returns an output that is used in the next task. There is a familiar programming concept that naturally follows the task of taking an input and returning an output: functions.
In this lesson, we will describe a new paradigm of programming called functional programming. We will compare it object-oriented programming (classes, objects, and state), and show how Python gives you the ability to switch between the two. Finally, we will finish the discussion by linking functional programming with data pipelines.
Functional programming can be a difficult programming paradigm to get into. Because it's difficult to understand functional programming without any hands-on practice, you’ll get to apply what you’ve learned from within your browser; there's no need to use your own machine to do the exercises. The Python environment inside of this course includes answer-checking to ensure you've fully mastered each concept before learning the next.
2. Comparing Object-Oriented to Functional
3. Understanding Pure Functions
4. The Lambda Expression
5. The Map Function
6. The Filter Function
7. The Reduce Function
8. Rewriting with List Comprehension
9. Writing Function Partials
10. Using Functional Composition
11. Next Steps