The Data Engineer Roadmap 🗺

Introduction

This has been inspired by the Full Stack Developer's Roadmap post written by @ender_minyard 🙌

With the ever growing data volumes and demands, the data engineering career has been one of the fastest growing jobs for the past few years.

According to the 2021 Stack Overflow survey, data engineers are one of the top 5 highest paid professionals right after SREs and DevOps engineers:

If you are looking to become a data engineer, here are some resources for data engineering that you can save for later.

Table Of Contents

💻 Fundamentals

👩‍💻 Programming basics

🧪 Testing

📊 Database Fundamentals

🏠 Data warehouses

📦 Object storage

⚡ Data processing

📩 Messaging

💽 Cluster computing

⏲ Workflow Scheduling

📺 Monitoring data pipelines

👨‍💻 Infrastructure as Code

🛫 CI/CD

💻 Fundamentals

Having a solid understanding of the Linux operating system could be a must in many IT related roles. You are going to benefit a lot if you know the basics of the following:

Basic Terminal Usage https://devdojo.com/course/linux-command-line-basics

Shell Scripting

Git and GitHub

Networking

👩‍💻 Programming basics

As with any IT related role it is essential to have fundamental knowledge of programming in general. The programming language itself does not matter that much, but you need to have good understanding of programming paradigms and best practices.

🧪 Testing

Functional testing

📊 Database Fundamentals

Having a solid understanding of SQL, data normalization and ACID transactions is a must for all data engineers.

SQL basics

OLTP vs OLAP

Horizontal vs vertical scaling

Relational Databases