5 Steps to Compile Yocto Using Docker Containers

Embedded Linux development is never easy. As a software developer, I often run into compatibility issues between my Linux distribution and the dependencies required for a compilation when using the Yocto Project. One potential solution is the use of Docker. In this article, I’ll therefore show you how to create a Docker container for compiling Yocto.

For those reading this in 2024, you may also benefit from leveraging Yocto’s own build system, Poky. Alternatively, you can check out our Yocto Embedded Development Services for end-to-end support.

These days, Docker – and containers in general – are difficult to miss. On the Docker website, a container is defined as “a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.”

Containers are used to address:

security, in particular by making it possible to run an application independently from the rest of the system;
software dependency management, by making it possible to deliver an application with its software dependencies and their configurations;
scalability, by making it possible to precisely configure the hardware resources (CPU, memory, bandwidth, etc.) allocated to each container.

How to create a Docker container for compiling Yocto

Install Docker

First of all, you must install Docker. There are two versions of the software. The Community Edition, which is free of charge, and the Enterprise Edition, for which you’ll have to purchase a license. We’ll be working with the first. You may even find this version in the software repositories of your preferred distribution. This is true for Debian and Ubuntu, where the following command should suffice:

sudo apt-get install docker.io

For Fedora, try:

sudo dnf install docker

If Docker is not included in the software repository of your distro or you wish to install the latest available version, you may need to go through the Docker website.

Useful commands to manage images and containers, the 2 key notions in Docker container

In Docker, there are two distinct notions: images and containers. An image is the sum of all software that is required to run an application. A container is generated by running an image.

You can therefore have several containers for the same image. In this section, you’ll find useful commands for working with images and containers.

Commands to manage images in Docker containers

First of all, you’ll want to list all the images on your machine. To do so, you can run the following command:

docker image ls

or its alias:

docker images

You’ll see the following information:

REPOSITORY: the name of the image
TAG: the version of the image
IMAGE ID: a unique identifier for the image
CREATED: length of time since the image was created
SIZE: the size of the image

You shouldn’t have a lot of images. To pull an image, you can use the following command:

docker image pull <registry>/<image>:<tag>

The “registry” is the server that contains the image. If you omit this information, the “registry” will be defined as “docker.io”. The “tag” is the desired version of the image. If you omit this information, it will be defined as “latest”.

You can therefore pull images from the public registry “docker.io”. Existing images can be viewed on DockerHub website.

If you search for Ubuntu, you should land on the following page of DockerHub website.

On the right-hand side of the page, there’s a command for pulling the image:

docker pull ubuntu

This is short for:

docker image pull docker.io/ubuntu:latest

This page includes all the tags that are available for this image.

Another way to pull an existing image is to import it via a .tar archive. Run the following:

docker image load -i <archive name>.tar

or its alias:

docker load -i <archive name>.tar

To do it that way, you will need to have previously exported the image using the command:

docker image save -o <archive name>.tar <image>:<tag>

or its alias:

docker save -o <archive name>.tar <image>:<tag>

You can also create your own image. To do so, you’ll have to write a Dockerfile (see next section) and ask Docker to generate the image:

docker build -t <image>:<tag> <path to the dockerfile>

Finally, to delete an image, you can use the following command:

docker image rm <image>:<tag>

or its alias:

docker rmi <image>:<tag>

Commands to manage your Docker container

Now that you know how to manage images, it’s time to learn how to manage containers. To create a Docker container and then run an application inside of it, use the following command:

docker run

You’ve got various options here. In our example, you may decide to use:

-t or --tty which lets you allocate a pseudo-TTY;
-i or --interactive which lets you keep the STDIN open. The first of these options gives you an interactive container;
-v or --volume which lets you create a bind mount between the host machine and the container;
--name which lets you name the container;
--rm which lets you have the container deleted automatically once the command is completed.

After using any of these options, you’ll have to enter the image that will be used to create the container and eventually, the command that will be run. Your command will therefore look like this:

docker run -it -v <host directory>:<container directory> <image>:<tag> <command>

You can use the following command to list the running containers:

docker container ls

docker ps

By default, this command only shows the containers that are running. The option “-a” allows the display of all the containers. You’ll see the following information:

CONTAINER ID: the container’s unique identifier
IMAGE: the image (with its tag) from which the container was created
COMMAND: the command that is running or was run
CREATED: length of time since the container was created
STATUS: the status of a container, which can be:

◦ created

◦ restarting

◦ removing

◦ running

◦ paused

◦ exited

PORTS: “exposed” ports, or those on which the container is likely to listen. (For ports, we can use the “–expose” option with the “docker run” command.)
NAMES: the name of the container (a random name is provided if you do not enter one)

It’s also possible to pause or create a container without running it… but I will not cover these scenarios in this article.

Instead, we’ll focus on the “running” status, which is displayed in the preceding command by the “Up” message followed by the amount of time since the container started running, and the “exited” status, which is displayed by the “Exited” message followed by the exit code in brackets and the amount of time since the container stopped running.

To start a container that has stopped running, use:

docker container start <container ID>| <container name>

docker start <container ID>| <container name>

While being created, the container will run the specified command. If the container is interactive, you’ll have to reattach STDOUT, STDERR with the option “-a”, and STDIN with the option “-i”.

To stop a running container, use:

docker container stop <container ID>| <container name>

docker stop <container ID>| <container name>

To delete a stopped container, use the following:

docker container rm <container ID>| <container name>

docker rm <container ID>| <container name>

Create your own image with Dockerfile

If you’re unable to find an appropriate image, then you’ll have to create your own. To do so, we’ll use a “Dockerfile”. A Dockerfile consolidates all the instructions that are necessary for creating a new image using the “docker build” command from above.

This command searches for a Dockerfile in the specified directory. All the other files and folders in the specified directory are sent to the Docker daemon to be used during the creation of the image. The directory should therefore only include those files that will be useful for creating the image (otherwise you’ll have to create a “.dockerignore” file).

To comment a line in the Dockerfile, start the line with the “#” symbol. The other lines should be composed of a command followed by arguments. The command is traditionally written in uppercase letters, but lowercase letters will also do.

The following commands will help us as we go forward:

FROM: starts the build and lets you choose an image to serve as the base
RUN: lets you run a command
ARG: lets you define the arguments that will be passed by the user to the Docker daemon which builds the image
ENV: lets you define the environment variables of the container
USER: lets you define the username to be used for subsequent commands of the Dockerfile
WORKDIR: lets you define the work directory to be used for subsequent commands of the Dockerfile
COPY: lets you copy a file from the host machine in the image
ENTRYPOINT: lets you define the image to be run when starting the container

Compiling Yocto with Docker Container, initial approach

At the time of writing this article, the latest version of Yocto was 2.7 (Warrior). If you are reading this in 2024 then you may be using 4.3 (Nanbield) or even 5.0 (Scarthgap). For those, like us, who prefer to use the Long-term Support (LTS) versions, 4.0 (Kirkstone) may be your version of choice. For more information on Yocto versions and release dates, check out the Yocto project’s releases Wiki.

To find out which Linux distros are supported, head over to the Yocto website. Here’s the link to version 2.7.

If, unfortunately, the distribution you use is not on the list (or not in the right version), to avoid compatibility issues, we’ll create a container with Docker that lets you compile everything.

To compile Yocto, you’ll have to install some packages on a classic distribution. We’ll therefore create an image containing all these packages. Let’s start by choosing a distribution from the list:

FROM ubuntu:18.04

Next, let’s install all the desired packages on the image. The Yocto manual provides the following list:

RUN apt-get update && apt-get install -y gawk wget git-core diffstat unzip
            texinfo gcc-multilib build-essential chrpath socat cpio python
            python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping
            python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm

If you only install the above, you’ll get the following error message when compiling Yocto:

Please use a locale setting that supports UTF-8 (such as LANG=en_US.UTF-8).
Python can’t change the filesystem locale after loading so we need a UTF-8 when Python starts or things won’t work.

You’ll therefore have to install the locales. The command becomes:

Next, we’ll create a user called “dev” to compile Yocto. Let’s use the following command:

RUN groupadd -g 1000 dev
            && useradd -u 1000 -g dev -d /home/dev dev
            && mkdir /home/dev
            && chown -R dev:dev /home/dev

And generate the locales:

RUN locale-gen en_US.UTF-8

We must also define the locale we wish to use:

ENV LANG en_US.UTF-8

And change the username:

USER dev

Finally, we’ll define the work directory:

WORKDIR /home/dev

The full Dockerfile will therefore look like this:

FROM ubuntu:18.04
RUN apt-get update && apt-get install -y gawk wget git-core diffstat unzip
            exinfo gcc-multilib build-essential chrpath socat cpio python
            python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping
            python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm locales

RUN groupadd -g 1000 dev
            && useradd -u 1000 -g dev -d /home/dev dev
            && mkdir /home/dev
            && chown -R dev:dev /home/dev

RUN locale-gen en_US.UTF-8

ENV LANG en_US.UTF-8

USER dev

WORKDIR /home/dev

After saving this file by itself in a separate directory, we can run the command that will create our image:

docker build -t <image name>:<tag> .

We can then use this image to create a container:

docker run -i -t –name first-test :<tag>

To fetch Yocto and compile the core-image-minimal image for the qemux68 architecture, run the following commands:

git clone -b warrior git://git.yoctoproject.org/poky
cd poky
source oe-init-build-env
bitbake core-image-minimal

Once the compilation is complete, you’ll find the results of your compilation in the tmp/deploy/images/qemux86/ directory.

Bringing Improvements to your Yocto Compilation

It won’t take long for you to notice the limitations of the above version.

Installing missing packages

If you look for vim or nano when editing a file, you’re not going to find them. You’ll therefore have to install one of these packages. What’s more, the free version does not come with the auto-completion feature. Instead, you’ll have to install the “bash-completion” package and configure your .bashrc. Finally, you may have tried to configure your Linux kernel with the following command:

bitbake -c menuconfig virtual/kernel

It will probably fail and generate the following message:

ERROR: linux-yocto-5.0.19+gitAUTOINC+31de88e51d_00638cdd8f-r0 do_menuconfig: No valid terminal found, unable to open devshell.
Tried the following commands:
            tmux split-window -c “{cwd}” “do_terminal”
            tmux new-window -c “{cwd}” -n “linux-yocto Configuration” “do_terminal”
            xfce4-terminal -T “linux-yocto Configuration” -e “do_terminal”
            terminology -T=”linux-yocto Configuration” -e do_terminal
            mate-terminal –disable-factory -t “linux-yocto Configuration” -x do_terminal
            konsole –separate –workdir . -p tabtitle=”linux-yocto Configuration” -e do_terminal
            gnome-terminal -t “linux-yocto Configuration” -x do_terminal
            xterm -T “linux-yocto Configuration” -e do_terminal
            rxvt -T “linux-yocto Configuration” -e do_terminal
            tmux new -c “{cwd}” -d -s devshell -n devshell “do_terminal”
            screen -D -m -t “linux-yocto Configuration” -S devshell do_terminal
ERROR: linux-yocto-5.0.19+gitAUTOINC+31de88e51d_00638cdd8f-r0 do_menuconfig:
ERROR: linux-yocto-5.0.19+gitAUTOINC+31de88e51d_00638cdd8f-r0 do_menuconfig: Function failed: do_menuconfig
ERROR: Logfile of failure stored in: /home/dev/poky/build/tmp/work/qemux86-poky-linux/linux-yocto/5.0.19+gitAUTOINC+31de88e51d_00638cdd8f-r0/temp/log.do_menuconfig.1785
ERROR: Task (/home/dev/poky/meta/recipes-kernel/linux/linux-yocto_5.0.bb:do_menuconfig) failed with exit code ‘1’

You can install “screen”, “vim”, and “bash-completion” to fix these problems. The line in our Dockerfile for installing the packages thus becomes:

Next, go ahead and copy the .bashrc in the container:

COPY ./bashrc /home/dev/.bashrc

Then, create a “bashrc” file alongside your Dockerfile that contains:

# enable programmable completion features (you don’t need to enable
# this, if it’s already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if ! shopt -oq posix; then
            if [ -f /usr/share/bash-completion/bash_completion ]; then
                        . /usr/share/bash-completion/bash_completion
            elif [ -f /etc/bash_completion ]; then
                        . /etc/bash_completion
            fi
fi

Securing your work by sharing it between the host machine and container

The result of the compilation only appears in the tree directory of the container. If you delete the container, you’ll lose all of the work done. You may therefore wish to share this data with the host machine:

docker run -i -t –name first-test -v <host directory>:/home/dev yocto-compile

As a result, any changes you make to your container’s /home/dev directory will also take place in the designated host directory. This may cause issues regarding access rights. We only set the PUID in the Dockerfile and the PGID was already set to 1000.

If the user of the host machine uses different IDs, you’re likely to run into problems when trying to write in your host machine’s tree directory from the container. We’ll therefore pass the PUID and PGID as parameters to the Dockerfile.

By the way, if you create the image for yourself, you can use the same username and the same path between the host machine and the container for the project. This will ensure there is a valid absolute path both on your host machine and in your container, thereby making it easier for you.

If a compilation error occurs, Yocto usually provides a path to a log file where the error is stored. As this path is an absolute path, you can open it on your host machine without having to change it. The Dockerfile will therefore look like this:

FROM ubuntu:18.04

RUN apt-get update && apt-get install -y gawk wget git-core diffstat unzip
            texinfo gcc-multilib build-essential chrpath socat cpio python
            python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping
            python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm locales
            vim bash-completion screen

ARG USERNAME=dev

ARG PUID=1000

ARG PGID=1000

RUN groupadd -g ${PGID} ${USERNAME}
            && useradd -u ${PUID} -g ${USERNAME} -d /home/${USERNAME} ${USERNAME}
            && mkdir /home/${USERNAME}
            && chown -R ${USERNAME}:${USERNAME} /home/${USERNAME}

RUN locale-gen en_US.UTF-8

ENV LANG en_US.UTF-8

COPY ./bashrc /home/${USERNAME}/.bashrc

USER ${USERNAME}

WORKDIR /home/${USERNAME}

The command for generating the image is as follows:

docker build -t <image name>:<tag> –build-arg USERNAME=<user name> –build-arg PUID=<puid> –build-arg PGID=<pgid> .

Since the data has been shared with the host machine, you can destroy the container once you have exited it. The command for creating the container therefore becomes:

docker run -it –rm -v <working directory>:<working directory> <image name>:<tag>

You can subsequently compile Yocto by running:

cd <working directory>
git clone -b warrior git://git.yoctoproject.org/poky
cd poky
source oe-init-build-env
bitbake core-image-minimal

Creating scripts to automate and gain time

In this last section, we’ll create a few shell scripts to make our lives easier.

Here they are:

StartBuild.sh, which will allow us to start the compilation by creating a container with the right parameters
yocto-entrypoint.sh, or the script that will let us fetch sources and start the compilation of Yocto
docker-entrypoint.sh, a simple script that serves as the entry point to the container

yocto-entrypoint.sh may need to be updated over time. We’ll use docker-entrypoint.sh (which will serve as the entry point to the container) to avoid having to generate a new image for each modification.

The main role of this script will be to call yocto-entrypoint.sh, which will be passed to the container by using the -v option of the docker run command. StartBuild.sh will execute the docker run with the right arguments.

We’ll start by creating a docker-entrypoint.sh script alongside our Dockerfile. It will contain:

#!/bin/bash
set -e
[ $# -eq 0 ] && set — yocto
if [ “$1” = “yocto” ]; then
shift; set — ${YOCTO_ENTRYPOINT} “$@”
fi
exec “$@”

As you can see, if we do not pass parameters or the first parameter of the script is Yocto, then we’ll call the Yocto compilation script whose path is made available by the environment variable YOCTO_ENTRYPOINT.

We’ll create the yocto-entrypoint.sh script in our work directory (a separate directory that does not include the Dockerfile):

#!/bin/bash

SUPPORTED_MACHINES=”
            qemuarm
            qemuarm64
            qemumips
            qemumips64
            qemuppc
            qemux86
            qemux86-64
            beaglebone-yocto
            genericx86
            genericx86-64
            mpc8315e-rdb
            edgerouter
”

SUPPORTED_YOCTO=”
            thud
            warrior
”
WORKDIR=”/”

yocto_sync()
{
            if [ -d ${WORKDIR}/yocto/sources/poky ]; then
                        cd ${WORKDIR}/yocto/sources/poky
                        git fetch origin
                        git checkout origin/$1
            else
                        mkdir -p “${WORKDIR}/yocto/sources”
                        cd “${WORKDIR}/yocto/sources”
                        git clone -b $1 git://git.yoctoproject.org/poky.git
            fi
}

is_in_list()
{
            local key=”$1″
            local list=”$2″

            for item in $list; do
                        if [ “$item” = “$key” ]; then
                                    return 0
                        fi
            done

            return 1
}

is_machine_supported()
{
            local MACHINE=”$1″

is_in_list “$MACHINE” “$SUPPORTED_MACHINES”
}

is_yocto_supported()
{
            local YOCTO=”$1″

is_in_list “$YOCTO” “$SUPPORTED_YOCTO”
}

show_usage()
{
            echo “Usage: StartBuild.sh command [args]”
            echo “Commands:”
            echo ” sync ”
            echo ” Sync Yocto version”
            echo ” E.g. sync quemux86 warrior”
            echo
            echo ” all”
            echo ” Build Yocto”
            echo
            echo ” bash”
            echo ” Start an interactive bash shell”
            echo
            echo ” help”
            echo ” Show this text”
            echo
            exit 1
}

main()
{
            if [ $# -lt 1 ]; then
                        show_usage
            fi

            if [ ! -d “${WORKDIR}/yocto/sources” ] && [ “$1” != “sync” ]; then
                        echo “The directory ‘yocto/sources’ does not yet exist. Use the ‘sync’ command”
                        show_usage
            fi

            case “$1” in
                        all)
                                    cd ${WORKDIR}/yocto/
                                    source sources/poky/oe-init-build-env build
                                    bitbake core-image-minimal
                        ;;

                        sync)
                                    shift; set — “$@”
                                    if [ $# -ne 2 ]; then
                                                echo “sync command accepts only 2 arguments”
                                                show_usage
                                    fi

                                    if ! is_machine_supported “$1”; then
                                                echo “$1 is not a supported machine: ${SUPPORTED_MACHINES}”
                                                show_usage
                                    fi

                                    if ! is_yocto_supported “$2”; then
                                                echo “$2 is not a supported yocto version: ${SUPPORTED_YOCTO}”
                                                show_usage
                                    fi

                                    yocto_sync $2

                                    cd “${WORKDIR}/yocto”
                                    source sources/poky/oe-init-build-env build
                                    sed -i “s/^MACHINE ??= .*$/MACHINE ??= ”$1”/” conf/local.conf
                        ;;

                        bash)
                                    cd “${WORKDIR}/yocto”
                                    source sources/poky/oe-init-build-env build
                                    bash
                        ;;

                        help)
                                    show_usage
                        ;;

                        *)
                                    echo “Command not supported: $1”
                                    show_usage
            esac
}

main $@

In the main function, there are three primary events:

all, which lets you start the compilation of a core-image-minimal image.
sync, which parameterizes the machine and the desired Yocto version. This lets you fetch or update the sources and modify the configuration fil