What’s New in v1.14: Data Engineering Path & Performance Improvements!
Our latest Dataquest release has over 20 new features, including many major performance improvements and the launch of our much-anticipated data engineering path.
New Path: Data Engineering
The first course in our Data Engineering Path is here!
Data Engineering is a broad field which includes:
- Working with Big Data
- Architecting distributed systems
- Creating reliable pipelines
- Combining data sources
- Collaborating with data science teams and building the right solutions for them.
If you’d like to find out more about Data Engineering, you can read our guide: What is a Data Engineer?
The first course in this new path is Processing Large Datasets In pandas, which includes a variety of techniques, which expand on our existing pandas courses and shows you how to use pandas with larger datasets.
Course Outline
The course contains five missions including two guided projects:
- Optimizing DataFrame Memory Footprint FREE
- Processing DataFrames In Chunks
- Guided Project: Practice Optimizing DataFrames And Processing In Chunks
- Augmenting pandas With SQLite
- Guided Project: Analyzing Startup Fundraising Deals From Crunchbase
Like all our paths, the data engineering path will be a continuous iterative rollout — we’ll keep adding more missions and courses over the course of the year!
Start our new Data Engineering path for free!
Performance Improvements
Our tech team have been hard at work as they continue to make code running faster and improve the stability of our various mission types, making your learning experience better.
Decreased Code Run Times
Building on the changes we made in September, we have moved all code running to be websockets-based, which has brought a 21% reduction in code running times.
In the last six months our median code running times have gone from 4.2 to 1.7 seconds, a 59% reduction.
Code running times over the past six months.
Stability improvements to Console and Command Line Missions
Additionally, we have re-architected communication for our command line missions and console, which will increase the reliability and stability of these missions.
We’ve also introduced a network and container status panel which will let you know of the status of your code running container and connection.
Network and container status panel.
New in Version 1.14
The full list of features in today’s release are below:
- Launch of the first course of our new data engineering path.
- Improved code running times.
- Enabled pooling for code running containers to reduce wait times due to containers initializing.
- Improvements to the stability of the console and command line missions.
- A new notifications feed which shows you when someone responds or upvotes your Q&A posts.
- Moved intercom messaging system to the header to make the interface cleaner.
- Improved the design and functionality of our payment page.
- Added a network and container status panel to help with troubleshooting.
- Added a reset button to command line missions that will help you restore the step to the correct state if you get stuck.
- We now display a more informative message when your payment is declined due to the wrong zip/postal code.
- Rewrote our Python Basics mission to be clearer.
- Made our first course — Python Programming: Beginner — entirely free, and added two guided projects.
- Added a new guided project: Working With Downloaded Data.
- Removed the old data visualization course that was replaced with two new courses in November.
- Fixed other minor errors in various missions.
- Fixed a bug where certificates are not generated for some users.
- Fixed a bug where users couldn’t update their email address.
- Added a prompts for users who signup through facebook to enter their email address.
- Fixed a bug where certain users behind firewalls could not run code.
- Fixed an issue with Microsoft Edge v14 that prevented users from logging in and creating accounts.
- Other minor bugfixes and stability improvements.
Checkout the new features now!
Coming Soon
If you’re interested in what’s coming in the next few weeks:
- The second course in the Data Engineering path
- A new ‘Machine Learning Fundamentals’ course
- Ability to download the source dataset inside each mission.
- A new mission end screen to make it easier for you to give us feedback.
- A new mission interface, and much more!
As always, we’d love to hear your feedback on what we should build next. If you have a great idea, e-mail us!
Data Scientist at Dataquest.io. Loves Data and Aussie Rules Football. Australian living in Texas.