CELEBRATE PYTHON’S BIRTHDAY WITH FREE LEARNING – ALL WEEK LONG
Project overview
In this project, you’ll take on the role of a data analyst and process over 54 MB of Wikipedia articles to find specific text matches. Using Python and MapReduce, you’ll build a parallel solution to efficiently search the dataset and return match details.
You’ll develop a simplified version of the grep command-line utility to find strings across multiple files. Through hands-on practice with text processing, parallel computing, and data engineering, you’ll gain valuable skills in analyzing large unstructured datasets.
Objective: Efficiently search a large text dataset using MapReduce and Python to find specific strings and build valuable big data skills.
Key skill required
To complete this project, it's recommended to build these foundational skills in Python
- Starting multiple processes in Python to parallelize analysis of Wikipedia pages
- Running functions on several processes simultaneously to efficiently process Wikipedia articles
- Sharing data between multiple processes for coordinated analysis of Wikipedia content
- Implementing the MapReduce framework to distribute processing of Wikipedia pages
Projects steps
Step 1: Introducing Wikipedia Data
Step 2: Adding the MapReduce Framework
Step 3: Grep Exact Match
Step 4: Grep Case Insensitive
Step 5: Checking the Implementation
Step 6: Finding Match Positions on Lines
Step 7: Displaying the Results
Step 8: Next Steps
Master skills faster with Dataquest
Go from zero to job-ready
Learn exactly what you need to achieve your goal. Don’t waste time on unrelated lessons.
Build your project portfolio
Build confidence with our in-depth projects, and show off your data skills.
Challenge yourself with exercises
Work with real data from day one with interactive lessons and hands-on exercises.
Showcase your path certification
Share the evidence of your hard work with your network and potential employers.
The Dataquest guarantee
Dataquest has helped thousands of people start new careers in data. If you put in the work and follow our path, you’ll master data skills and grow your career.
We believe so strongly in our paths that we offer a full satisfaction guarantee. If you complete a career path on Dataquest and aren’t satisfied with your outcome, we’ll give you a refund.
Recommended projects
Investigating Fandango Movie Ratings
Practice using R to analyze movie ratings data, compare 2015 vs 2016 ratings, and apply sampling and distributions to investigate bias.
Investigative Statistical Analysis – Analyzing Accuracy in Data Presentation
Practice statistical analysis in Python to investigate movie rating bias and determine if Fandango inflated ratings.
NYC Schools Perceptions
Practice data cleaning, analysis, and visualization in R to explore survey data and showcase your skills with R Notebooks.
Building Fast Queries on a CSV
Practice implementing an inventory system for a laptop store using Python classes, dictionaries, and binary search.
Garden Simulator Text Based Game
Practice using OOP, error handling, and randomness in Python to create an interactive gardening game simulator.
Predicting Heart Disease
Practice building a K Nearest Neighbors classifier in Python to predict heart disease risk from patient data.
Word Raider
Practice using Python variables, lists, loops, conditionals, and file handling to build an interactive word-guessing game.
Predicting Condominium Sale Prices
Practice using linear regression in R to predict condominium sale prices based on size and location in New York City.
Kaggle Data Science Survey
Practice analyzing survey data in Python to uncover insights about data science careers and skills.
Grow your career with
Dataquest.


