Kamal 2: What's New and How to Easily Upgrade Your Apps

37signals, the team behind Kamal, released Kamal v2.0 during the 2024 Rails World conference in September 2024. This release has a lot of improvements that resolve many common complaints about the tool, and it should make your deployments go much smoother than before.

Main Changes in Kamal 2.0

Let’s talk about some of the main changes included in this release. Check out Kamal’s documentation for more details about all the new updates.

Goodbye, Traefik—Hello, Kamal Proxy

The most significant change in Kamal 2.0 is the elimination of Traefik as the default proxy service for your deployed application. Instead, Kamal now uses a new proxy server called Kamal Proxy, built from scratch by the team at 37signals.

This proxy server is simpler to configure for most use cases, provides clearer error messages to help debug problems, and makes it easier to set up a ton of functionality like automatic SSL with Let’s Encrypt and the ability to deploy multiple web applications on a single server. You could handle most of this functionality with Traefik, but Kamal Proxy simplifies the process.

Something to keep in mind is that Kamal Proxy is not a direct replacement for Traefik. Kamal Proxy doesn’t have a lot of Traefik’s built-in functionality, like middleware (which I have covered previously on YouTube) and other advanced features. If you were using Traefik for response compression, rate-limiting, or other functions, you wouldn’t have those using the new replacement proxy.

If you need or prefer to use Traefik as your proxy server when deploying with Kamal 2.0, you can still continue to use it by setting up Traefik as a Kamal accessory and adding some configuration to route requests through Kamal Proxy. The Kamal documentation has more details on how to do this.

New Secrets Management

Another significant change in Kamal 2.0 is how it manages secrets. In previous versions of Kamal, we used the .env file to set up environment variables and sensitive configuration settings like your container registry password. However, many tools already use .env, which could clash with Kamal’s settings and create some confusion.

Kamal 2.0 fixes this by changing the file location to set secrets. Instead of using the .env file, Kamal will check the .kamal/secrets file instead, preventing potential conflicts between Kamal and other tools that rely on a .env file.

Another welcome change to secrets in Kamal is that they no longer have a separate lifecycle event, so your environment variables will be automatically set when running commands like kamal deploy. That means you don’t need to run separate commands to update environment variables on your servers. If you ever deployed your application using Kamal and it didn’t work because you forgot to run the kamal env push command to update environment variables on the host, you’ll be very happy with this change.

The Kamal command line tool also has a few additional commands to make it easier to manage secrets, like kamal secrets fetch to read secrets from common password managers like Bitwarden and 1Password, and kamal secrets print to make sure you have the right data set up for your application.

Updates to How Kamal Uses Docker

Beyond the new Kamal Proxy and secrets setup, Kamal 2.0 has revamped how it uses Docker, both when building your application’s Docker image and running Docker containers on your hosts.

First, the default build driver that Kamal uses to build the Docker image of your application on deployment has changed. In older versions, Kamal would check the builder settings in the configuration to determine which Docker build driver to use, with the likely choice being the default docker driver.

Kamal 2.0 uses the docker-container build driver by default. Now, for most people, this change likely won’t mean much, but it will require that you specify the image architecture to build for your deployment, unlike older versions where it automatically builds multi-architecture images unless specified.

The docker-container driver also has a few other advantages, like configuring how to export the build cache, and it skips loading the image into your local image store to prevent all those deployment builds from piling up on your development system.

Another Docker-related change in Kamal 2.0 is that setting up your application and accessories will create a custom Docker network called kamal. Before, Kamal would use the default bridge network, which assigns an IP address for each container it spins. However, these IP addresses aren’t stable across restarts, and some services need a way to ensure they can reach other containers consistently.

With the custom Docker network, you can communicate with each container in that network directly by their hostname. In fact, Kamal Proxy uses this new network to ensure it routes traffic to the right place, making application restarts more stable than before.

Upgrading from Kamal 1.x

Let’s see how we can take an existing deployed application that uses an older version of Kamal and bring it up to date with the latest and greatest. The modifications made in this article might differ from the ones you need to make to other applications using Kamal 1.x, so it’s a good idea to review the latest Kamal documentation to know what older configuration settings have changed or no longer exist.

As an example for this article, I’ll use a Ruby on Rails application called Airport Gap that I have deployed on a Hetzner Cloud server using Kamal version 1.9.2.

The Kamal configuration file for this example consists of a single server running the application, a worker process for asynchronous jobs, and two accessories—a PostgreSQL database and a Redis server. The configuration also contains some modifications to the default settings for the Traefik proxy service, and the web application uses a few container labels to configure middleware and set up SSL with Let’s Encrypt.

Step #1: Migrate secrets and environment variables

Let’s start updating this configuration to get it ready to upgrade to the latest version of Kamal. The first thing I will do is migrate all my environment variables to the new secrets location. For this, I will create a new file under the .kamal directory that Kamal creates when running kamal init in your project and use the file name secrets.

Setting secrets in this file uses the same format as when setting them in .env, so in most cases, all you need to do is copy and paste the variables you need from your existing .env file when upgrading to Kamal 2.0. Once you do that, you can get rid of the .env file or keep it if you’re using it for your application or other tooling.

The latest versions of Kamal suggest using password managers to read secrets, but you can also set the values directly in the .kamal/secrets file. If you set sensitive data in this file, remember to include it in your project’s .gitignore or similar file for whichever source code management tool you’re using so you don’t accidentally commit this information to your repository. Also, add this file in the .dockerignore file so it’s not included when building the Docker image.

One gotcha I wanted to mention here is that Kamal no longer loads environment variables automatically when running Kamal commands. That means if you were setting up environment variables in your .env file and referred to their values using ERB in the configuration settings like <%= ENV['SERVER_IP'] %>, for instance, you’ll need to manually load those values first by requiring Ruby’s Dotenv library. Check out the “Secrets changes” section of the Kamal documentation for more information.

Step #2: Remove Traefik-related settings

We’ll go to the main config/deploy.yml file to make some necessary changes before upgrading to Kamal 2.0. In this configuration file, I’ll remove all the Traefik-related settings since I’ll migrate over to using Kamal Proxy instead.

For the example configuration, we’ll need to remove all the Traefik labels set up under servers → labels since they no longer have any use. I’ll also remove the entire traefik section at the end of the configuration file since it’s no longer a valid setting in Kamal 2.0.

Step #3: Set up Kamal Proxy

Because we’re now going to use Kamal Proxy, there are some changes to be aware of for your applications. Kamal Proxy assumes that the Docker image exposes the web application on port 80 and will set it as the default target port for accessing your app.

There’s a good chance your Docker image exposes a different port. In this example, the Airport Gap Docker image exposes and runs the server on port 3000, so I need to make sure that Kamal Proxy sets this port as the target for my web application.

To do this, I’ll configure a new proxy section in my Kamal configuration, which handles configuration settings for Kamal Proxy. Under this section, I’ll add the app_port setting. This setting lets you specify which port the proxy should use to route traffic to the application that’s running on a Docker container. I’ll set the port to 3000 here.

Another proxy-related setting I’ll include under proxy is the host. This setting allows you to specify which host Kamal Proxy should use to route to your web application. For this example, I’ll set the host to airportgap.com. The setting is optional, and if you don’t set it, Kamal will route all incoming requests straight to your application.

While setting the host isn’t necessary, it’s important to set it for two reasons:

• It allows you to deploy multiple web applications on the same server using Kamal. Setting a host for each application will let Kamal detect where to route traffic when you have more than one web application deployed on a server. Even if you don’t plan on deploying another application on the same server, you should set a host to limit which requests go through to your web app.
• It lets Kamal Proxy set up HTTPS automatically for you. Kamal Proxy supports automatic SSL certificate generation using Let’s Encrypt, and it requires a host setting to allow Let’s Encrypt to run the validation challenge for your domain.

However, the automated SSL process doesn’t happen by default. For that, we’ll also need to include the ssl setting under proxy to equal true. Adding the host and ssl settings is all you need to have secure HTTPS connections to your web application.

Step #4: Update the Docker build process

Finally, one last change I need to make in this configuration file is under the builder section. In this older configuration, I set the multiarch setting to false because my local development system and the server where I deploy this application use the same processor architecture (AMD64), so I don’t build the image for Arm-based processors like newer Apple computers.

In Kamal 2.0, the multiarch setting no longer exists, and we must specify the architecture for which we’re building the image. So I’ll remove it from the configuration and add the new arch setting in its place.

This setting accepts a string if you plan to build the Docker image with a single architecture or an array to specify you want to build the image for AMD64 and Arm-based processors. In my case, I’m only going to use a single architecture, amd64.

These are the only changes I need to make for my configuration. Again, depending on your existing Kamal configuration, you might have to modify a lot more before upgrading to Kamal 2.0, so make sure to read the documentation to understand what needs to change for other deployment configurations.

The updated config/deploy.yml file for Kamal 2.0 will now look like the following:

Upgrading Existing Kamal 1.x Deployments

Now that my configuration is all squared away, we can proceed with using Kamal 2.0 for our deploys. The first step is to update the Kamal gem to the latest version. You can do this by running the gem update kamal command to update it locally. If you’re using Bundler and have Kamal as a project dependency, update the version in your Gemfile and run bundle update.

With Kamal version 2 installed, we can begin the upgrade process. Thankfully, a new command called kamal upgrade automatically handles all the underlying changes for us. This command does a couple of things:

1. It stops and removes Traefik to prepare for Kamal Proxy.
2. It creates the kamal Docker network if it doesn’t exist, which it will use to communicate between all containers set up for your deployments.
3. Kamal Proxy will be set up as a container using the kamal Docker network.
4. It reboots the currently-running application and sets it to run in the kamal Docker network.
5. Once the application spins up again, Kamal Proxy gets updated to route traffic to it.
6. It reboots all your accessories and sets them in the kamal Docker network.

The kamal upgrade command prints out a warning that it will do what I just explained. That means there will be some slight downtime while it switches all these processes, so keep that in mind when performing the upgrade. This process should take about a minute or so to go through the upgrade and check that the application is still healthy.

When the command finishes its execution, congratulations! Your application is now working with the latest version of Kamal.

We can continue deploying our application using the kamal deploy command, interact with our servers using the kamal app exec command, and everything you’ve come to expect with older versions of Kamal. Most of the commands from older versions of Kamal work pretty much the same.

The command line tool does have a couple of new commands for managing secrets and Kamal Proxy, and it eliminates some commands like kamal env push and kamal envify. But beyond that, if you’ve been using Kamal to deploy your web app for a while, there’s not much to learn.

Wrap Up

The latest version of Kamal has a lot of exciting changes that make deploying your web application simpler than ever. The introduction of Kamal Proxy eliminates a lot of the complexity we had to deal with before for setting up SSL and deploying more than one web application on a server. Other updates are welcome introductions that improve the deployment experience, like how Docker works under the hood and avoiding the reliance on the .env file for secrets.

As a bonus, the upgrade process couldn’t be simpler. In most cases, all you need is to set up a new .kamal/secrets file, make a few minor modifications to your configuration files, run the kamal upgrade command, and you’re good to go. For a major software upgrade, this is one of the most painless ones I’ve experienced.

While all of this makes deploying web apps easier, Kamal 2.0 will help the team behind Kamal develop the tool and open up many opportunities for them that they didn’t have before. I’m sure we’ll see many more exciting improvements that will keep improving Kamal, which I’m looking forward to.

Need help with your Kamal setup?

If you’re stuck with your existing Kamal deployments or want to begin using Kamal to deploy your web applications, I can help you set up your Kamal configuration, deploy your application, and ensure your web app is running smoothly on your servers. Reach out and let’s start a conversation today.

Screencast

If this article or video helped you upgrade your Kamal-deployed web applications, consider subscribing to my YouTube channel for similar videos containing tips on helping Rails developers ship their code with more confidence, from development to deployment.