What does a Devops Engineer do?

Yesterday I was watching Michael DeHaan (Creator of Ansible) answering questions from network engineers about automation. Michael made the comment that the term devops has been overused and that no one really knows what devops means anymore. I do believe he has a point, but I'd like to set the record straight.

You see, I am a devops engineer. I know what devops means to me. Even though they have once again given me the devops job title; automation engineer would work just as well. To me devops is about a philosophy more than a job title or magic bullet. It is important that everyone understands what role a "Devops Engineer" plays at a company.

First of all in order to be an effective devops engineer you need to be a people person and you need to be able to deploy an application working with the developer(s) that created it. Often times you will find yourself having to create a wrapper script to modify the application, change variables, build code, etc.

Most people are inherently resistant to change and old school developers tend to be far more resistant than your average bear. You will likely find yourself helping a developer pushing code from their Windows laptop to a virtual machine. In this type of situation you may be expected to deploy an application and create a framework for future developers to seamlessly deploy new code.

If you work at a company that practices safe agile you may have evangelist among others driving the push to better development practices. Without support from upper management your developers are not likely to adapt to more modern practices. You will work with many developers who don't understand what a bridge network is, what a Kubernetes Deploymentconfig is, or even what an AWS CloudFormation script is. Your job is to understand all of the nuances.

The devops engineer's job is to fill the knowledge gap and try to encourage more productive behaviors. Often this means spending hours writing documentation, having a demo session with the team, and paired programming directly with a developer. That is all on top of writing scripts, maintaining infrastructure, continuously learning, CI/CD (continuous integration/continuous deployment), backups, performance enhancing, setting up monitoring, pen testing, security audits, and everything else. You are typically isolated on a team with developers helping every one of them from time to time.

The second major part of devops beyond working with people is automation. Many different things fall in this bucket and a Devops engineer needs to understand them. Once you have deployed a new application you will want to automate this process with a CI/CD pipeline. Next you can add automated tests created by both you and the developer to ensure everything is running as expected. If you are working with a database like PostgreSQL or MongoDB chances are you will also be a part time DBA unless it is fully managed such as running in the Cloud.

After a development team is created we tend to follow what is called the crawl, walk, run model. This involves getting something to work, then iterating over and over again to automate and improve it. After a few months things start to get fully automated and Jenkins or another CI/CD tool will handle manual steps that used to be the process. In fact if the team keeps pushing the same application a devops engineer can ultimately have the app running smoothly on its own. At this point the devops engineer job shifts to being ready when something goes wrong and, creating robust monitoring to alert key people before or when issues occur.

Tools

A devops engineer needs to learn many different technologies and be really good at those skills. At a minimum they should be able to write bash, they should understand Linux to the point they can mount iscsi volumes, setup a new smtp gateway, or tune Postgres. You must be a fast learner and highly adaptable as every place you go will use new or different tools. Above all else you need to be a really good problem solver and enjoy finding ways to make things more efficient.

If you are considering becoming a devops engineer you need to understand the operating system you support and how to fix it when things go wrong. Below is a minimal list of valued skill I would advise every devops engineer to learn. Remember, the more you know the better. At some point you could easily find yourself architecting a new infrastructure, in order to, build a new hybrid cloud solution for the application you've been asked to support.

Valued Skills

• Ansible **
• Apache/Nginx/HAProxy
• Bash **
• CDN like Akamai
• CI/CD **
• Cloud AWS, GCP, and/or Azure (at least one) **
• DNS/Bind **
• Docker **
• Git **
• Hardware (Dell and/or HP) **
• HA proxy
• Jenkins **
• Kubernetes or Openshift
• Linux **
• Load Balancing
• MongoDB
• Networking **
• PostgresSQL and maybe MySQL
• Python and familiarity with Java
• Storage configuration (nfs and iscsi)
• Storage server concepts
• Tomcat - know how deploy war files and tune the server.
• VI or emacs (there is no IDE in a container) **
• VMWare or RHEV
• Yaml/JSON (learn both) **
• Microsoft Windows

** Focus on mastering these first as they are very common.

As you can see in the list above, by the time you become a devops engineer, you will be a full stack infrastructure engineer supporting developers. A developers job is to work on large complex applications. A devops engineer understands how infrastructure works, helps the developers deploy and automate their apps, and maintains "all the things" a system administrator would traditionally support.

At my current job I support developers using Openshift and CI/CD. My last job I mostly worked as a Platform Engineer installing and supporting Openshift as a service (PaaS). I still had to work with developers while maintaining storage, hardware, virtual machines, 10 Openshift clusters deployed globally, and 100s of other things.

Being a devops engineer you never know what you are going to get into next. As you gain more skills you will become highly desired and be able to work where you choose. I personally never plan to go into another office again as I enjoy working remote. I have seen studies that claim bash scripters are actually paid more than developers of any language. I am guessing that is because, these days, most bash experts are doing devops in one form or fashion.

Devops engineers wear many different hats and must learn to use a vast array of tools. They must constantly solve difficult and sometimes very frustrating problems. Being a devops engineer requires a large amount of patience and a love of what you are doing. I enjoy my job now more than I ever enjoyed being a system administrator, IT manager, or product owner. I highly recommend taking the time to invest in yourself and follow the path down devops lane.