/ Updates

What's New in v1.14: Data Engineering Path & Performance Improvements!

Our latest Dataquest release has over 20 new features, including many major performance improvements and the launch of our much-anticipated data engineering path.

New Path: Data Engineering

The first course in our Data Engineering Path is here!

Data Engineering is a broad field which includes:

  • Working with Big Data
  • Architecting distributed systems
  • Creating reliable pipelines
  • Combining data sources
  • Collaborating with data science teams and building the right solutions for them.

If you'd like to find out more about Data Engineering, you can read our guide: What is a Data Engineer?

The first course in this new path is Processing Large Datasets In pandas, which includes a variety of techniques, which expand on our existing pandas courses and shows you how to use pandas with larger datasets.

Course Outline

The course contains five missions including two guided projects:

Like all our paths, the data engineering path will be a continuous iterative rollout — we'll keep adding more missions and courses over the course of the year!

Performance Improvements

Our tech team have been hard at work as they continue to make code running faster and improve the stability of our various mission types, making your learning experience better.

Decreased Code Run Times

Building on the changes we made in September, we have moved all code running to be websockets-based, which has brought a 21% reduction in code running times.

In the last six months our median code running times have gone from 4.2 to 1.7 seconds, a 59% reduction.

Code running times over the past six months.

Stability improvements to Console and Command Line Missions

Additionally, we have re-architected communication for our command line missions and console, which will increase the reliability and stability of these missions.

We've also introduced a network and container status panel which will let you know of the status of your code running container and connection.

Network and container status panel.

New in Version 1.14

The full list of features in today's release are below:

  • Launch of the first course of our new data engineering path.
  • Improved code running times.
  • Enabled pooling for code running containers to reduce wait times due to containers initializing.
  • Improvements to the stability of the console and command line missions.
  • A new notifications feed which shows you when someone responds or upvotes your Q&A posts.
  • Moved intercom messaging system to the header to make the interface cleaner.
  • Improved the design and functionality of our payment page.
  • Added a network and container status panel to help with troubleshooting.
  • Added a reset button to command line missions that will help you restore the step to the correct state if you get stuck.
  • We now display a more informative message when your payment is declined due to the wrong zip/postal code.
  • Rewrote our Python Basics mission to be clearer.
  • Made our first course — Python Programming: Beginner — entirely free, and added two guided projects.
  • Added a new guided project: Working With Downloaded Data.
  • Removed the old data visualization course that was replaced with two new courses in November.
  • Fixed other minor errors in various missions.
  • Fixed a bug where certificates are not generated for some users.
  • Fixed a bug where users couldn’t update their email address.
  • Added a prompts for users who signup through facebook to enter their email address.
  • Fixed a bug where certain users behind firewalls could not run code.
  • Fixed an issue with Microsoft Edge v14 that prevented users from logging in and creating accounts.
  • Other minor bugfixes and stability improvements.

Coming Soon

If you're interested in what's coming in the next few weeks:

  • The second course in the Data Engineering path
  • A new 'Machine Learning Fundamentals' course
  • Ability to download the source dataset inside each mission.
  • A new mission end screen to make it easier for you to give us feedback.
  • A new mission interface, and much more!

As always, we'd love to hear your feedback on what we should build next. If you have a great idea, e-mail us!

Josh Devlin

Josh Devlin

Data Scientist at Dataquest.io. Loves Data and Aussie Rules Football. Australian living in Texas.

Read More