Use Kool to run Multiple Docker Applications at the same time in your Local Development Environment

More and more, we find ourselves working with Docker containers in the context of microservice architectures comprising multiple, loosely-coupled applications and services, and/or projects made up of two or more distinct apps running side-by-side talking to each other via APIs. As developers who work on these types of projects know all too well, setting up your local development environment to run multiple Docker applications at the same time can be a real hassle. Developing each app on its own is a breeze using Kool and Docker Compose. However, when you need to run multiple apps at the same time, configuring the routing and intercommunication between different containers can be challenging.

In this tutorial, we'll show you how to set up your local environment to run more than one Docker application at the same time using Kool and a Caddy web server. No doubt, this is just one of several ways to solve for this use case, but we think it's an approach that works really well for most projects.

Requirements

Before you start, if you haven't done so already, you need to install Docker and Kool CLI. It also wouldn't hurt to quickly get up-to-speed with the Kool CLI commands.

kool is a CLI tool that makes local development with Docker super easy. Kool CLI will level up your development workflow, and help you and your team improve the way you develop and deploy cloud native applications. Make sure you're always using the latest version by running kool self-update.

It always starts off simple

You have a single application with its Docker Compose environment. Let's call it App 1.

$ mkdir -p ~/multiple-apps/app-1
$ cd ~/multiple-apps/app-1

Create the two files we need:

# App 1
# ~/multiple-apps/app-1/docker-compose.yml
services:
  app:
    image: kooldev/php:8.0-nginx
    ports:
      - 80:80
    volumes:
      - .:/app/public

# App 1
# ~/multiple-apps/app-1/index.php
<?php
    echo "Welcome to App 1!\n";

With these two files, you can now get App 1 up and running using kool start, and check the status of its service container using kool status:

$ kool start
Creating network "app-1_default" with the default driver
Creating app-1_app_1 ... done

$ kool status
+---------+---------+---------------------------------------------+---------------+
| SERVICE | RUNNING | PORTS                                       | STATE         |
+---------+---------+---------------------------------------------+---------------+
| app     | Running | 0.0.0.0:80->80/tcp, :::80->80/tcp, 9000/tcp | Up 4 seconds  |
+---------+---------+---------------------------------------------+---------------+
[done] Fetching services status

$ curl localhost
Welcome to App 1!

Awesome! Your app service container is running, kool status shows that port 80 is mapped from your host to the container, and curl localhost successfully returns the output of App 1.

But then it starts getting tricky

As your project evolves over time, you need to add a second application called App 2, which runs alongside App 1. In other words, to work on the project, you need to run both apps at the same time.

Let's quickly set up App 2.

$ mkdir -p ~/multiple-apps/app-2
$ cd ~/multiple-apps/app-2

Once again, let's create the two files we need (inside the app-2 directory):

# App 2
# ~/multiple-apps/app-2/docker-compose.yml
services:
  app:
    image: kooldev/php:8.0-nginx
    ports:
      - 80:80
    volumes:
      - .:/app/public

# App 2
# ~/multiple-apps/app-2/index.php
<?php
    echo "Welcome to App 2!\n";

This time, when you try to get App 2 up and running (using kool start), you run into a problem.

$ kool start
Creating network "app-2_default" with the default driver
Creating app-2_app_1 ...
Creating app-2_app_1 ... error

ERROR: for app-2_app_1  Cannot start service app: driver failed programming external connectivity on endpoint app-2_app_1 (24719704f55491122a18f051d3f1e789b6afc3f34ccf7bfe3d7eac510117ef42):
  Bind for 0.0.0.0:80 failed: port is already allocated

ERROR: for app  Cannot start service app: driver failed programming external connectivity on endpoint app-2_app_1 (24719704f55491122a18f051d3f1e789b6afc3f34ccf7bfe3d7eac510117ef42):
  Bind for 0.0.0.0:80 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.

As per the error message, you have a port conflict (Bind for 0.0.0.0:80 failed: port is already allocated). You cannot have two different containers bound to the same port on your host.

Before you continue, let's stop your App 2 and App 1 containers:

$ cd ~/multiple-apps/app-2 # you should already be here
$ kool stop

$ cd ~/multiple-apps/app-1
$ kool stop

Not so fast!

To fix this error, your first impulse is probably to use different ports for each service. For example, you can run App 1 on localhost:8081 and App 2 on localhost:8082. However, you'll quickly realize this solution isn't viable because it's not flexible enough, and doesn't provide intercommunication between applications over a shared Docker network.

Proxy to the rescue

The proxy design pattern provides a much better solution for running multiple Docker applications at the same time in your local development environment.

Add a global network to Docker Compose

First, you need to improve the Docker Compose environments used by each of your apps. Using one of the many best practices built into the Docker configurations included with Kool Presets, let's create a shared network between Docker containers.

By default, all containers in a docker-compose.yml file will share the same virtual network. This means two different applications will not have a channel of communication. For this reason, Kool Presets will usually have two networks for each container: kool_local and kool_global.

kool_local is a local network that's only available to the group of containers from that docker-compose.yml file. It's the same as the default network (if we didn't specify the network ourselves).
kool_global is a global network created outside the scope of any particular docker-compose.yml file. It's available system-wide, and any containers running on the host can join it.

Let's create an external kool_global network inside each Docker Compose environment, and add each app to it.

# App 1
# ~/multiple-apps/app-1/docker-compose.yml
services:
  app:
    image: kooldev/php:8.0-nginx
    expose:
      - 80
    volumes:
      - .:/app/public
    networks:
      kool_global:
        aliases:
          - app-1

networks:
  kool_global:
    external: true

# App 2
# ~/multiple-apps/app-2/docker-compose.yml
services:
  app:
    image: kooldev/php:8.0-nginx
    expose:
      - 80
    volumes:
      - .:/app/public
    networks:
      kool_global:
        aliases:
          - app-2

networks:
  kool_global:
    external: true

Notice that we replaced the ports configuration with expose. You don't want these app containers bound to the host anymore, in order to avoid a conflict. Instead, you want to bind a single container to the host, which proxies each request internally over the kool_global network to the correct service container using its network alias.

The aliases key on the container network works like a domain name that resolves to that container's address when used within the same network. This is a great way to normalize names for service containers that need to talk to each other.

Set up the proxy

As mentioned earlier, we're going to use a Caddy web server as our reverse proxy. It's our first choice because it has a simple configuration interface and a rich feature set. Traefik or Nginx would work great too, so feel free to use what you like best when you implement this solution in a real project.

Let's start by creating a Caddyfile configuration file in a new proxy directory:

$ mkdir ~/multiple-apps/proxy
$ cd ~/multiple-apps/proxy

# ~/multiple-apps/proxy/Caddyfile
{
  auto_https off
}

http://a.localhost {
  reverse_proxy / http://app-1
}

http://b.localhost {
  reverse_proxy / http://app-2
}

Notice that we use the app-1 and app-2 container network aliases to point to each destination, based on the incoming Host request (a.localhost vs. b.localhost). You should also add these local domains to your /etc/hosts file: echo "127.0.0.1 a.localhost b.localhost" | sudo tee -a /etc/hosts.

By default, Caddy tries to use HTTPS for all hosts. For this tutorial, we're disabling it. We'll cover local TLS usage in a future article.

Next, let's create a new docker-compose.yml for Caddy itself:

# ~/multiple-apps/proxy/docker-compose.yml
services:
  proxy:
    image: caddy:2-alpine
    ports:
      - 80:80
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
    networks:
      - kool_global

networks:
  kool_global:
    external: true

And that's it. You're done configuring your proxy.

Now, all you need to do is spin up your App 1, App 2 and Proxy service containers (using kool start of course), and verify it works by sending a couple of test requests.

# Proxy
$ cd ~/multiple-apps/proxy # you should already be here
$ kool start
Creating proxy_proxy_1 ... done

# App 1
$ cd ~/multiple-apps/app-1
$ kool start

# App 2
$ cd ~/multiple-apps/app-2
$ kool start

$ curl -H "Host: a.localhost" http://localhost
Welcome to App 1!

$ curl -H "Host: b.localhost" http://localhost
Welcome to App 2!

W00t! You now have both apps up and running at the same time. You can access them using different host names, and they can also communicate with each other.

To clean up your local environment, you'll need to move into each directory (app-1, app-2, and proxy) and run kool stop, and then remove the directories you created: rm -r ~/multiple-apps.

Next steps

Kool's core team is already working on a new set of commands to make the above steps seamless and transparent, so you don't have to worry about the details. We invite you to join the discussion and contribute.