Alla Bannikova

“Each step prepares you for the next one. The projects helped me to gain confidence and the community is very beneficial.”

Alla Bannikova

Data Analyst

Project overview

In this project, you’ll take on the role of a data analyst and process over 54 MB of Wikipedia articles to find specific text matches. Using Python and MapReduce, you’ll build a parallel solution to efficiently search the dataset and return match details.

You’ll develop a simplified version of the grep command-line utility to find strings across multiple files. Through hands-on practice with text processing, parallel computing, and data engineering, you’ll gain valuable skills in analyzing large unstructured datasets.

Objective: Efficiently search a large text dataset using MapReduce and Python to find specific strings and build valuable big data skills.

Key skill required

To complete this project, it's recommended to build these foundational skills in Python

  • Starting multiple processes in Python to parallelize analysis of Wikipedia pages
  • Running functions on several processes simultaneously to efficiently process Wikipedia articles
  • Sharing data between multiple processes for coordinated analysis of Wikipedia content
  • Implementing the MapReduce framework to distribute processing of Wikipedia pages

Projects steps

Step 1: Introducing Wikipedia Data

Step 2: Adding the MapReduce Framework

Step 3: Grep Exact Match

Step 4: Grep Case Insensitive

Step 5: Checking the Implementation

Step 6: Finding Match Positions on Lines

Step 7: Displaying the Results

Step 8: Next Steps

Master skills faster with Dataquest

Go from zero to job-ready

Go from zero to job-ready

Learn exactly what you need to achieve your goal. Don’t waste time on unrelated lessons.

Build your project portfolio

Build your project portfolio

Build confidence with our in-depth projects, and show off your data skills.

Challenge yourself with exercises

Challenge yourself with exercises

Work with real data from day one with interactive lessons and hands-on exercises.

Showcase your path certification

Showcase your path certification

Share the evidence of your hard work with your network and potential employers.

The Dataquest guarantee

Guarantee

Dataquest has helped thousands of people start new careers in data. If you put in the work and follow our path, you’ll master data skills and grow your career.

Money

We believe so strongly in our paths that we offer a full satisfaction guarantee. If you complete a career path on Dataquest and aren’t satisfied with your outcome, we’ll give you a refund.

Recommended projects

Free

Guided Project: Investigating Fandango Movie Ratings

Practice using R to analyze movie ratings data, compare 2015 vs 2016 ratings, and apply sampling and distributions to investigate bias.

R Intermediate 8 steps
Free

Guided Project: NYC Schools Perceptions

Practice data cleaning, analysis, and visualization in R to explore survey data and showcase your skills with R Notebooks.

R Beginner 3 steps
Free

Guided Project: Investigating Fandango Movie Ratings

Practice statistical analysis in Python to investigate movie rating bias and determine if Fandango inflated ratings.

Python Intermediate 8 steps
Free

Guided Project: Building Fast Queries on a CSV

Practice implementing an inventory system for a laptop store using Python classes, dictionaries, and binary search.

Python Intermediate 10 steps
Free

Guided Project: Garden Simulator Text Based Game

Practice using OOP, error handling, and randomness in Python to create an interactive gardening game simulator.

Python Beginner 11 steps
Free

Guided Project: Predicting Heart Disease

Practice building a K Nearest Neighbors classifier in Python to predict heart disease risk from patient data.

Python Intermediate 10 steps
Free

Guided Project: Word Raider

Practice using Python variables, lists, loops, conditionals, and file handling to build an interactive word-guessing game.

Python Beginner 11 steps
Free

Guided Project: Predicting Condominium Sale Prices

Practice using linear regression in R to predict condominium sale prices based on size and location in New York City.

R Intermediate 8 steps
Free

Guided Project: Kaggle Data Science Survey

Practice analyzing survey data in Python to uncover insights about data science careers and skills.

Python Beginner 8 steps

Grow your career with
Dataquest.

98%
of learners recommend
Dataquest for career advancement
4.85
Dataquest rating
SwitchUp Best Bootcamps
$30k
Average salary boost
for learners who complete a path
Aaron

Aaron Melton

Business Analyst at Aditi Consulting

“Dataquest starts at the most basic level, so a beginner can understand the concepts. I tried learning to code before, using Codecademy and Coursera. I struggled because I had no background in coding, and I was spending a lot of time Googling. Dataquest helped me actually learn.”

Jessi

Jessica Ko

Machine Learning Engineer at Twitter

“I liked the interactive environment on Dataquest. The material was clear and well organized. I spent more time practicing then watching videos and it made me want to keep learning.”

Victoria

Victoria E. Guzik

Associate Data Scientist at Callisto Media

“I really love learning on Dataquest. I looked into a couple of other options and I found that they were much too handhold-y and fill in the blank relative to Dataquest’s method. The projects on Dataquest were key to getting my job. I doubled my income!”

Join 1M+ data learners on
Dataquest.

1

Create a free account

2

Choose a learning path

3

Complete exercises and projects

4

Advance your career

Start learning today