27
2 Widespread Attacks on Your Containerized Environment and 7 Rules to Prevent it.
How to Apply Common Security Principles and What tool to Choose to Prevent Attacks on Your Docker Containers and Kubernetes clusters.
The MITM attack is widespread in the Kubernetes and Docker. This attack includes additional malicious parts between the component that sends data and the component that receives this data. It can be a fake container, service, middleware, or even a human. For example:
- CVE-2020–8554 — vulnerability that allows attackers to obtain access to the cluster by creating ClisterIPs service.
- Siloscape — malware inside windows containers. The Silocape creates a backdoor to the whole Kubernetes cluster, including sensitive data and CPU, GPU, and resources
This attack allows an attacker to run the malicious code to use the CPU, GPU, and Memory of the PC for mining cryptocurrencies. Example:
- CVE-2018–15664 — gives access to the docker system with root permission.
The container itself is a small OS that can be susceptible to attack with malicious code. In this article, we talk about
- What you need to do to secure your app in the container.
- Security tools.
- Security rules, policies.
- How to apply common security principles to prevent attacks.
I consider using Docker and Kubernetes as two current leaders in container engines and orchestration.
This principle means that you should not execute containers using admin users. You should create users that have admin access and can only operate with this particular container. You can also make groups add users there. You can read more about it in “Isolate containers with a user namespace”. Below is an example of how to create the user and group.
First of all, you should use official verified and signed images. To find and check images you can use docker trust inspect.
For example docker trust inspect — pretty google/apigee-mart-server:1.3.6.
Docker Content Trust (DCT) can help with digital signatures.
It allows verification of images and publishers during runtime. This process is based on (Docker Content Trust Keys)[https://docs.docker.com/engine/security/trust/#docker-content-trust-keys], which generates several keys during the first interaction with DCT.
You should set up Memory and CPU limits for your docker container because docker has a container that does not have this option by default. This principle is a way to prevent DoS attacks. For example, you can set up a Memory limit to prevent your container from consuming all memory. The same applies to CPU limits.
There is also an option to set up resource limits on a Kubernetes level. I cover this topic in the Kubernetes Security section.
This principle is essential to understand Docker’s networking principles. You should understand what Docker Network Drivers are, for example:
- bridge
- host
- overlay
By default, one container network stack does not have access to another container. However, if you configure bridged or host to accept traffic from any other containers or external networks, you can create a potential security backdoor for an attack. You can also disable inter container communication using just set flag — icc=false within docker daemon.
Security monitoring is essential for the detection of malicious code and attacks on your containers.
With a proper monitoring tool, you should be able to detect the issues.
The tool allows you:
- Build a real-time dashboard.
- Set up alerts to send you messages via Email, SMS, or even your preferable chat platform.
To find the vulnerability in Docker containers, you can use
-
docker scan command. - CAdvisor, which is also a pretty powerful tool to monitor your container. Moreover, you can also run it in the Kubernetes cluster.
Docker scan or Caadvisor is a simple solution that applies to only one particular container. For more complex scenarios, for example, when you run 50+ containers in the Kubernetes, you need complex monitoring tools such as:
- Prometheus and Grafana. Prometheus is a logging component that “scrapes” information from your container and puts it into a data source. The data sources can be SQL, NoSQL data storage. Also, Prometheus has an Alert Manager component. It allows users to create rule-based alerts. Grafana is a framework that helps to build complex UI dashboards. The dashboards can be easily configured to get data from Prometheus.
- Datadog, which is a comprehensive, all-in-one monitoring tool that contains logging component subsystems and sidecars. It also includes a complex and interactive UI framework.
- Azure Log Analytics. It is a complex tool that monitors your containers, especially in Azure Container Registry, Azure Kubernetes Services. It is a handy option if you have your container solution under Azure Cloud Services as it supports it out of the box.
I have listed some pretty popular solutions. However, there are many others in the market worth mentioning, like Sysdig, Sematext, Dynatrace.
I like to “cook” using the combination of Prometheus + CAdvisor + Grafana.
- Prometheus is a powerful and open-source option for monitoring CPU, GPU, Memory, Images, and other metrics.
- CAdvisor is quite good at detecting vulnerabilities.
- Grafana is good at building, configuring dashboards and alerts, and imports all components together.
Also, I use the security tool Kube-bench that covers vulnerabilities scanning only. The Kubebench brings an additional layer to your cluster security monitoring. There are plenty of security tools available for Kubernetes.
This principle is quite important to move all sensitive data out of the container. You can use different options to manage your secrets and other sensitive data:
- Docker secret allows you to store your secrets outside of the image.
- If your run docker containers in Kubernetes, you can use Secrets to store your passwords, certificates, or any other sensitive data
- Use cloud-specific storage for sensitive data — for example, Azure Key Vault or AWS Secret Manager.
Vulnerability scanning tools are the essential part of detecting images that may have security holes. Moreover, you can also integrate properly selected tools into the CI/CD process. Below I have listed some scanning tools:
- Dagda uses a static analysis approach to find viruses, malware, and fake sub-images and trojans. It is based on Red Hat Security Advisories (RHSA) libraries of existing vulnerabilities databases.
- Trivy can detect complex vulnerabilities with high accuracy for OS like Alpine Linux and RHEL/CentOS , Debian, Ubuntu, and others. It is quite powerful, opensource, and free. You can run Trivi in standalone or client/server modes. Therefore you can add it to your CI/CD process.
- Clair is used for static analysis of your images. It supports images that are based on the Open Container Initiative (OCI). You can build your services for scanning images that can be based on Clair API. Clair uses CVE databases to detect vulnerabilities.
You can find more in the Containers Trend Report article.
- Use Secured Docker Registries
To protect your images, you can create an additional security layer and use images from protected registries like:
- Harbor, which is an open source registry with integrated vulnerability scanning. It is based on security policies that apply on docker artifacts.
- Quay, which scans your images for vulnerability. It is an image registry powered by RedHat. Quay also offers a standalone image repository that you can install and use internally in your organization. Below you can see how it scans for vulnerabilities.
But what if you are already using other registries like Azure Container Registry or Docker Hub? You can find how to do it in the Containers Tend Report article.
Kubernetes security is a big and important topic. The main security rules are:
- Networking and Network Policies. You should understand how the Kubernetes networking model works. It will help you set up proper network communication between pods, and pretend creating open ports or direct access to the nodes. The Network Policy can help you to organize this communication.
- Secure Ingress and Egress traffic to your pod. Here you can also use Network Policies. You can use strategy to deny all Egress and Ingress traffic and then start opening. You can also use service mesh like Istio. It adds additional service layers, automates traffic and helps with monitoring. However, you should be careful to use the service mesh as it may add additional complexity.
- Transport Layer Security. You should enable TLS if it is not opened. TLS should be used for communication between Kubernetes cluster services.
- Restrict Access To Kubernetes Dashboard.
- Use RBAC and follow the principle of least privilege.
- Restrict access to Kubelet. You should enable the authentication and authorization to use this tool. Only admins should have access to Kubelet.
The security principles that are mentioned in the docker container section
To find security and misconfiguration in kubernetes, you can use the following tools:
- Kube-bench. It is one of the most powerful tools in the section. Its security check is based on CIS Kubernetes Benchmark.
- Kubeaudit
- Kubesect.io
Security Topic is very important and complex, especially in the Docker and Kubernetes world. This article contains the most important recommendations to be taken into account. You can find more information about Containers Trends in the Containers Tend Report.
27