29
How I Went From Analyst to Data Engineer
Photo by Pakata Goh on Unsplash
A common career path question in the data field is how to make the transition from a data analyst to a data engineer.
Of course, this question is probably overshadowed by the question of how to go from data analyst to data scientist.
While both data analyst and data engineer can be equally rewarding professions, you may find that you prefer the more technical and code-heavy side of data engineering, or perhaps you have many years of experience as an analyst and want to diversify your skills. In either scenario, it is possible to build on your skills and knowledge as an analyst to go into a new role as a data engineer. So what is the most effective way to grow your skills?
Here is how I went from analyst to data engineer.
As in most industries, experience, education, and overall know-how all play a role in getting a job.
If you are a recent college graduate or otherwise looking for an entry-level position in the data field, it's not unlikely that you'll start in some form of data analyst position. This is the most common scenario because most of these roles only require a bachelor's degree, some Excel experience, and maybe a little SQL. While the work is challenging for those with no working data or statistical knowledge, it is far less complex and requires less technical ability than the work of other members of a data team, such as a data engineer or data scientist.
Your key role as an analyst is essentially to turn complicated numeric data into a format that those in the organization with no statistical knowledge can understand. The knowledge that you will gain on the job as an analyst will differ depending upon your employer and specific area of expertise; however, there are steps that you can take to ensure that you are positioned to move towards the title of a data engineer.
First, be sure to make the most of your position as an analyst. During your first job, you will learn a lot of valuable skills that will lay the groundwork for an engineering position.
For starters, you will learn the ins and outs of how businesses work as well as how to operate professionally in the data field. Despite this, not all analyst positions will allow you to continually learn and develop the skills to become a data engineer. If you are not learning in your first position, be sure to seek other opportunities that hone in on the technical aspects of data analysis. This may be in the form of courses outside of work, working with other teams (like the data science team) within your existing company, or seeking employment with a new company entirely. These projects can also help you get a feel for whether you will excel in the coding/programming aspect of data engineering before making any major employment changes.
After you have learned the ins and outs of being an analyst in a more general position and have also learned some of the basics of data engineering, it is wise to find a new position where you work directly under the guidance of an engineering team.
Here, you will learn the best practices and drastically transform your technical skills while working as part of the team. Although you may still be hired on as an analyst or to perform more of a data operations role, you will have the opportunity to take the initiative to prove yourself in coding and programming, and eventually, you will be ready to take on the full duties of a data engineer.
Interviewing for a new position can be daunting, especially when you first start out as an analyst and are looking to make the leap to engineer. In this industry, there is a wide range of expectations for those in the data field. You may be required to perform tasks you have never even heard of at one organization, while in others, the bulk of your work may be less technical than you would prefer.
By interviewing with many employers, you can identify areas where your skills or knowledge levels are lacking and work on these aspects to ensure that your experience is well-rounded. Eventually, however, you will find a position that is in line with your skillset, and you can begin your career as a data engineer.
I began my data journey as an analyst in the health care field. After college, I had taken several courses like bioinformatics, computation neurobiology, and epidemiology that unveiled my passion for data. Likewise, these courses taught me the importance of programming, and I began to understand the value that these technical skills could provide to someone working in the field of data. I began to use my programming knowledge more, but as a recent college graduate, opportunities as a data engineer were hard to come by. Instead, I began my career as a project analyst at a hospital.
In this position, I found that my role did not involve much in the form of project analytics and focused more on tasks like website development, managing data warehouses, developing ETLs, and basic analytics. Despite this, I never had an "engineering" title and did not have the opportunity to build a solid foundation in the position because my team was strictly analytics-focused. I realized that this job did not offer learning opportunities that would allow me to advance my career and transition into a data engineer role. I turned to the data science team at the organization and was given an opportunity to develop a data model that was out of my typical wheelhouse, which proved to be a learning experience.
The biggest and most crucial step of my transition from analyst to data engineer came when I left the hospital position behind and began to work with a health care startup that focused on data analytics. I recognized the value of working with an engineering team and thrived under their guidance while learning the best practices in terms of data analytics and engineering.
While I first started out performing data operations and manually loading data into the warehouse, I took the initiative to build a website that would enable the process to be automated. Although the amount of data being entered did not necessarily require automation, this venture proved that I was able to perform more complex tasks, such as developing.
My manager began to assign me engineering tasks, and I was able to access a plethora of information and programs that helped me gain real-life experience as a data engineer. After honing my data engineer skills for over two years at the health care startup, I realized that there was little room for personal and professional growth in such a small establishment and began to seek out other employment opportunities as a data engineer in a larger business.
Through the interviewing process, I discovered that there were gaps in my knowledge as a data engineer. While some companies were focused on integration and unit tests, others were more concerned with data structures and algorithms. The process of interviewing with several establishments in the data field is hit or miss but allows you to identify new concepts and areas that you need to improve upon, as well as your personal strengths as a data engineer. Using these knowledge gaps, I was able to work on these areas to improve my skills.
Eventually, I landed a job at a FAANG (Facebook, Amazon, Apple, Netflix, and Google). This proved to be a drastic change from the startup I formerly worked at, as there are thousands of engineers employed at these organizations. The data infrastructure was much more mature and streamlined, which made it easier to learn new practices and concepts while also gaining valuable experience in working collaboratively with larger teams.
While your journey from analyst to data or software engineer may look a lot different from my own path, it is important to remember to use your position as an analyst to gain skills, knowledge, and experience that will make switching into engineering possible in the future.
Transitioning from an entry-level position, like analyst, to a senior role in any industry can take several years and requires immense dedication. Although you are likely to encounter setbacks and struggle with certain aspects along the way, be sure to use each experience as a learning opportunity from which to grow.
29