Since hitting the scene back in 2013 it has really transformed how the entire industry packages, publishes, and deploys services in “The Cloud”. With a Docker image you’re able to bundle up everything your service needs (the code, runtime dependencies, system dependencies, and the OS dependencies) into one package that can be easily run by any Docker host. If done right, Docker images also make it easier to rebuild the service in a repeatable way.

However, actually defining that Docker image can be tricky and there are many ways it can be done wrong. For now though, I just want to focus on how to make your Docker image smaller.

ℹ️ What does the “size” of a Docker image mean? It’s referring to the actual storage costs of the image on disk and over the wire. This is going to encompass everything that you installed or copied over into your image or was included in any base images you’re using.

Why do we care how big an image is?⌗

You might be wondering why it matters how big your Docker image is. That’s a fair question. There are a few motivations for keeping the image as small as possible.

  1. 💽 It takes up less disk space on the host machine. The more images you have the more likely you’re going to run out of disk space on that partition.
  2. 💰 It can cost more to store images in your registry if your registry charges per gigabyte. This gets worse over time as the number of different versions of your image increases.
  3. ⏱️ It takes longer to send that image over the network. This can mean that your service is going to take longer to scale up or deploy.
  4. 🦹 It can mean that you have a much larger attack surface because you likely have more things present on your image than you need and the more things you have installed the more likely it is that at least one of them has a vulnerability.
  5. ⏱️ You might be artificially increasing the build times of your images by including way more than you need.

Our sample application⌗

Let’s quickly define an application that we can use to show how you can decrease your Docker image size.

I made an incredibly minimal application in Go that we’ll be Dockerizing for the purposes of this post. You can find it here:

https://github.com/durandj/minifying-docker-images-demo

Each stage of this post is a different tag in the repository.

This is a simple web server that returns the HTTP 418 status code when you hit the root path.

It also has two runtime dependencies and because it’s a Go application we’ll need to actually build it before we finish with the Docker image.

ℹ️ While this may be a Go repository, all the same recommendations and suggestions apply. In some languages the recommendations become even more important! I’ll do my best to call some of these differences between languages out where I can.

Creating a Docker image⌗

v0 - The Naive Approach⌗

The easiest way to create a Docker image for our application would look something like this:

FROM golang:1.21

WORKDIR /code

COPY . .
RUN CGO_ENABLED=0 go build -o service cmd/main.go

CMD ["/code/service"]

ℹ️ You could also just build the Go binary outside of the Docker image and add it in. That would work however it doesn’t fully solve the small image problem unless you really only copy in the one file very specifically. It also introduces some new problems. Namely, how do you make sure that the binary is being built in a repeatable way which isn’t sucking in some unexpected dependencies on the build environment.

You may already know what each step is going on here but we’ll go through it again anyway just to make extra sure.

FROM golang:1.21

Here we’re telling Docker that we want to switch to the golang:1.21 image for any subsequent commands that we perform. This is a great way to pin the specific runtime/OS dependencies that we need for building our application. Specifically here we are saying that we want the Debian base image and Go 1.21.

WORKDIR /code

Now we’re telling Docker that want to switch to the /code directory for all the following commands. At least until another WORKDIR command is encountered. Without this Docker always switches to the base Docker image’s working directory on each command.

COPY . .

This copies every file from the Docker context to the current working directory in the image. While this command looks very simple it’s actually got a lot going on and is the most likely thing to cause problems in your Docker image that are also easy to fix. Given that let’s dig a bit deeper into what it’s doing.

So first, what is the Docker context that I mentioned? When you’re building a Docker image you must specify a directory. Often times this looks something like docker image build --tag=<tag> . That . at the end says that you’re taking the current directory as your build context but you could also pass any other directory instead. Docker is then able to enforce that you can only copy files from the given directory into the Docker image. This is partially a security feature for when you’re building an unknown Docker image. Since you picked the root directory for the image to work from, it can’t try and access other files that could potentially compromise the system and since the Docker daemon usually is run as the root user that means it could access a lot of files.

The next thing about Docker context that gets brushed under the rug is that it’s also used to support remote Docker image builds. The Docker CLI doesn’t do any of the work itself. When you ask it to start a container or build a Docker image it’s just making an RPC to the Docker daemon which may or may not be running on the same machine. You might be using a remote machine for compliance reasons, because you need a more powerful build server, or because you’re using a non-Linux machine. Docker is built specifically for Linux and if you want to run a Linux based Docker image (most images fall under this category) then you need to run it in on a Linux machine. If you are using a Mac then you’re going to be running a virtual machine that has Docker installed. When you start a Docker image build, the Docker CLI will send the remote Docker daemon all of the files in the Docker context. If your Docker context is a huge directory containing a lot of files, this will take a long time.

RUN CGO_ENABLED=0 go build -o service cmd/main.go

This command is much simpler, it just runs the given shell command. For our purposes that means building the Go binary.

The main extra note to add here is when you need to install extra dependencies using apt-get or the like. Make sure you clean up any generated cache files. Same goes for downloaded or generated files that you don’t need. Deleting them after you’re done with them reduces file polution.

CMD ["/code/service"]

This final command tells Docker that when this image is run as a container to run the binary that we built with no arguments by default.

Results⌗

So this image totally works and would be able to successfully run our demo service.

However when I look at the size of the resulting image we see that the image itself is 899 MB in size. That’s huge! Especially since the actual Go binary we built is only 9.8 MB.

The reason for this is because we’re pulling in the entire build toolchain and anything else that is normally needed to build a Go binary. We don’t need any of that when we’re actually running the service. Let’s try and get rid of all of that.

v1 - Multi-stage images⌗

One of the great features of Docker is that it supports multi-stage builds for Docker images. This means you can split different parts of the build into different stages that can happen at different times and only include one of them in the final output.

For this incredibly simple application we only need two stages:

  • build
  • runtime

Stages are denoted by this syntax FROM <image:tag> AS <stage name>. Optionally, you can omit the stage name and use the index but that’s not really a good idea. You can also skip the stage name if its the final stage since nothing else is going to refer to it.

Each stage starts basically from scratch. The file system will only contain what is provided by the current base image (from the FROM statement) and omits anything from any previous stages that might have already happened. If we want to include something from a previous stage we can use a modified version of the COPY command.

COPY --from=<stage> <input path> <output path>

This works like a normal COPY command but instead of copying from the Docker build context it will instead copy from a different stage.

Let’s put this together to create a better Dockerfile.

FROM golang:1.21 AS build

WORKDIR /code

COPY . .
RUN CGO_ENABLED=0 go build -o service cmd/main.go


FROM debian:bookworm

WORKDIR /srv

COPY --from=build /code/service ./service

CMD ["/srv/service"]

We now have our two stages with the last one implicitly being the runtime stage. The build stage is in charge of building the Go binary and the runtime stage takes that binary and is in charge of actually running it.

Results⌗

With this change our application works exactly the same but now weighs in at only 127 MB. This is much better but it’s still a lot bigger than just the binary itself.

v2 - Scratch⌗

Normally a Docker image will include some kind of Linux operating system which is used to provide any OS level dependencies required by the application. It also means you have access to a shell should your application need it. If you don’t need any of that though it’s dead weight and brings in a bunch of dependencies that will need patching.

We can remove the OS from within the image by using the scratch base image which is the lowest level base image you can use. It contains absolutely nothing.

FROM golang:1.21 AS build

WORKDIR /code

COPY . .
RUN CGO_ENABLED=0 go build -o service cmd/main.go


FROM scratch

COPY --from=build /code/service ./service

CMD ["/service"]

Our image no longer has an operating system and can only run this one service. We also only need to worry about updating Go dependencies and operating system concerns are no longer an issue.

ℹ️ It’s very important that we compiled this binary with CGO_ENABLED set to false so that a completely static binary is built that doesn’t depend on any C libraries being provided.

Results⌗

This brings the total image size to 10.1 MB in size. This is basically the size of the binary plus some extra metadata.

⚠️The scratch image comes with some downsides and isn’t helpful in every situation. Let’s look at some of those cases.

  1. If you’re using a programming language which requires a runtime (such as Python, Java, Node, Ruby, etc) this won’t work for you. It might be possible to compile the runtime for the scratch environment but it’s probably going to be more work than it’s worth and some parts of the runtime may not even work since there’s no Linux environment.
  2. If you depend on OS resources you’re going to be out of luck. For example, you’re not going to have file system watchers.
  3. If you require extra resources such as SSL or TLS certificates you’re going to have to manually pull those in which is going to be extra work to maintain.
  4. Debugging is almost certainly going to be harder since there’s no shell anymore. This certainly removes possible attack vectors but it also makes things a lot more challenging for you.

Given the challenges with the scratch image let’s look at another alternative.

v3 - Distroless⌗

Another popular option that was created by Google is the Distroless base images. These are meant to be a better alternative to scratch that basically addresses all of it’s short comings.

They do this by being based on a super minimal Debian installation with everything removed that isn’t nailed down. It also comes with variants that have runtimes for common programming languages.

By default there’s no shell provided just like in the scratch image but you can optionally have one included with the debug tag.

We also get the option to use a base image that doesn’t use the root use by default!

FROM golang:1.21 AS build

WORKDIR /code

COPY . .
RUN CGO_ENABLED=0 go build -o service cmd/main.go


FROM gcr.io/distroless/static-debian11:nonroot

WORKDIR /srv

COPY --from=build /code/service /srv/service

CMD ["/srv/service"]

This should look basically idential to the v1 image just with a new base image for the runtime.

Results⌗

With this change we’ve got our image down to 12.5 MB. This is every so slightly bigger than the scratch image but way smaller than any of the other options. It also is more flexible and more secure.

This is a good stopping point for the base image changes we can make so now let’s look at how we can reduce how much we include in the Docker build context.

v4 - Smaller build context⌗

If you look carefully at the build logs you’ll see that the current Docker context when building is currently about 6.36 KB.

=> => transferring context: 6.36kB

This isn’t awful but there’s definitely room for improvement and honestly that amount is going to be much worse depending on the programming language that you’re using. For exampe, Node is going to be significantly worse because of node_modules.

ℹ️ Remember from a previous section that this can really increase the time it takes for a build to even start.

Docker helpfully provides a .dockerignore file that works exactly the same as .gitignore and can be used to tell the Docker CLI to not include files in the build context.

In our case we can use a really simple file to get a ton of benefit.

/.git
/.gitignore
/Makefile

.editorconfig
.tool-versions
Dockerfile

*.md

Results⌗

This one file reduces our context size down to 206 B. This is a major reduction and again it gets even more profound when you have more files in your repository.

This is a good start for making builds start faster but let’s make them finish faster.

v5 - Reducing rerun steps⌗

One of the issues that you may have noticed when we’re making changes to the service is that it causes a full rebuild of the entire image. We’ve reduced how often this has happened by removing some files from the build context (such as the Dockerfile) but if we added a new feature to the service but are still using the same dependency versions, should we redownload the dependencies every time? No.

ℹ️ If you’re not familiar with how go build works, it by default will fetch any missing dependency versions. So it implicitly does the equivalent of npm install on build.

Docker provides a caching mechanism to store the output from each build step which means it should be able to skip entire steps if the input for that step hasn’t changed. We can leverage this to speed up our builds.

Right now the dependency installation step is the slowest part of our build and it’s also expensive for whomever is serving our dependencies so let’s do the right thing for everyone and only do the install step when we don’t have a cached version of them.

FROM golang:1.21 AS build

WORKDIR /code

COPY ./go.mod ./go.sum ./
RUN go mod download

COPY . .
RUN CGO_ENABLED=0 go build -o service cmd/main.go


FROM gcr.io/distroless/static-debian11:nonroot

WORKDIR /srv

COPY --from=build /code/service /srv/service

CMD ["/srv/service"]

Docker now knows when we want to do the dependency install step and inputs we need for it and it can prevent us from redownloading them on every build. Now the only step that needs to happen when we make a feature change is the step to build the binary!

Wrap up⌗

Those are just the basics of what you can do to improve your Docker images. There are further optimizations you can make with how many layers get created when running each command or how you show the logs for each command but that’s a tail for another time.