5 Steps to Compile Yocto Using Docker Containers

Homepage 5 Steps to Compile Yocto Using Docker Containers

These days, Docker – and containers in general – are difficult to miss. On the Docker website, a container is defined as “a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.”

Containers are used to address:

security, in particular by making it possible to run an application independently from the rest of the system;
software dependency management, by making it possible to deliver an application with its software dependencies and their configurations;
scalability, by making it possible to precisely configure the hardware resources (CPU, memory, bandwidth, etc.) allocated to each container.

How to create a Docker container for compiling Yocto

As a software developer, I’ve run into compatibility issues between my Linux distribution and the dependencies required for a compilation when using the Yocto Project. Docker is effective at solving these types of problems. In this article, I’ll therefore show you how to create a Docker container for compiling Yocto.

Install Docker

First of all, you must install Docker. There are two versions of the software. The Community Edition, which is free of charge, and the Enterprise Edition, for which you’ll have to purchase a license. We’ll be working with the first. You may even find this version in the software repositories of your preferred distribution. This is true for Debian and Ubuntu, where the following command should suffice:

sudo apt-get install docker.io

For Fedora, try:

sudo dnf install docker

If Docker is not included in the software repository of your distro or you wish to install the latest available version, you may need to go through the Docker website.

Useful commands to manage images and containers, the 2 key notions in Docker container

In Docker, there are two distinct notions: images and containers. An image is the sum of all software that is required to run an application. A container is generated by running an image.

You can therefore have several containers for the same image. In this section, you’ll find useful commands for working with images and containers.

Commands to manage images in Docker containers

First of all, you’ll want to list of all the images on your machine. To do so, you can run the following command:

docker image ls

or its alias:

docker images

You’ll see the following information:

REPOSITORY: the name of the image
TAG: the version of the image
IMAGE ID: a unique identifier for the image
CREATED: length of time since the image was created
SIZE: the size of the image

You shouldn’t have a lot of images. To pull an image, you can use the following command:

docker image pull <registry>/<image>:<tag>

The « registry » is the server that contains the image. If you omit this information, the « registry » will be defined as « docker.io ». The « tag » is the desired version of the image. If you omit this information, it will be defined as « latest ».

You can therefore pull images from the public registry « docker.io ». Existing images can be viewed on DockerHub website.

If you search for ubuntu, you should land on the following page of DockerHub website.

On the right-hand side of the page, there’s a command for pulling the image:

docker pull ubuntu

This is short for:

docker image pull docker.io/ubuntu:latest

This page includes all the tags that are available for this image.

Another way to pull an existing image is to import it via a .tar archive. Run the following:

docker image load -i <archive name>.tar

or its alias:

docker load -i <archive name>.tar

To do it that way, you will need to have previously exported the image using the command:

docker image save -o <archive name>.tar <image>:<tag>

or its alias:

docker save -o <archive name>.tar <image>:<tag>

You can also create your own image. To do so, you’ll have to write a Dockerfile (see next section) and ask Docker to generate the image:

docker build -t <image>:<tag> <path to the dockerfile>

Finally, to delete an image, you can use the following command:

docker image rm <image>:<tag>

or its alias:

docker rmi <image>:<tag>

Commands to manage your Docker container

Now that you know how to manage images, it’s time to learn how to manage containers. To create a Docker container and then run an application inside of it, use the following command:

docker run

You’ve got various options here. In our example, you may decide to use:

-t or --tty which lets you allocate a pseudo-TTY;
-i or --interactive which lets you keep the STDIN open. The first of these options gives you an interactive container;
-v or --volume which lets you create a bind mount between the host machine and the container;
--name which lets you name the container;
--rm which lets you have the container deleted automatically once the command is completed.

After using any of these options, you’ll have to enter the image that will be used to create the container and eventually the command that will be run. Your command will therefore look like this:

docker run -it -v <host directory>:<container directory> <image>:<tag> <command>

You can use the following command to list the running containers:

docker container ls

docker ps

By default, this command only shows the containers that are running. The option “-a” allows to display all the containers. You’ll see the following information:

CONTAINER ID: the container’s unique identifier
IMAGE: the image (with its tag) from which the container was created
COMMAND: the command that is running or was run
CREATED: length of time since the container was created
STATUS: the status of a container, which can be:

◦ created

◦ restarting

◦ removing

◦ running

◦ paused

◦ exited

PORTS: “exposed” ports, or those on which the container is likely to listen. (For ports, we can use the “–expose” option with the “docker run” command.)
NAMES: the name of the container (a random name is provided if you do not enter one)

It’s also possible to pause or create a container without running it… but I will not cover these scenarios in this article.

Instead, we’ll focus on the “running” status, which is displayed in the preceding command by the “Up” message followed by the amount of time since the container started running, and the “exited” status, which is displayed by the “Exited” message followed by the exit code in brackets and the amount of time since the container stopped running.

To start a container that has stopped running, use:

docker container start <container ID>| <container name>

docker start <container ID>| <container name>

While being created, the container will run the specified command. If the container is interactive, you’ll have to reattach STDOUT, STDERR with the option “-a”, and STDIN with the option “-i”.

To stop a running container, use:

docker container stop <container ID>| <container name>

docker stop <container ID>| <container name>

To delete a stopped container, use the following:

docker container rm <container ID>| <container name>

docker rm <container ID>| <container name>

Create your own image with Dockerfile

If you’re unable to find an appropriate image, then you’ll have to create your own. To do so, we’ll use a “Dockerfile”. A Dockerfile consolidates all the instructions that are necessary for creating a new image using the “docker build” command from above.

This command searches for a Dockerfile in the specified directory. All the other files and folders in the specified directory are sent to the Docker daemon to be used during the creation of the image. The directory should therefore only include those files which will be useful for creating the image (otherwise you’ll have to create a “.dockerignore” file).

To comment a line in the Dockerfile, start the line with the “#” symbol. The other lines should be composed of a command followed by arguments. The command is traditionally written in uppercase letters, but lowercase letters will also do.

The following commands will help us as we go forward:

FROM: starts the build and lets you choose an image to serve as the base
RUN: lets you run a command
ARG: lets you define the arguments that will be passed by the user to the Docker daemon which builds the image
ENV: lets you define the environment variables of the container
USER: lets you define the username to be used for subsequent commands of the Dockerfile
WORKDIR: lets you define the work directory to be used for subsequent commands of the Dockerfile
COPY: lets you copy a file from the host machine in the image
ENTRYPOINT: lets you define the image to be run when starting the container

Compiling Yocto with Docker Container, initial approach

As I write this article, the latest version of Yocto is 2.7 (Warrior). For more information on Yocto versions and release dates, check out Yocto project’s releases Wiki.

To find out which Linux distros are supported, head over to the Yocto website. Here’s the link to version 2.7.

If, unfortunately, the distribution you use is not on the list (or not in the right version), to avoid compatibility issues, we’ll create a container with Docker that lets you compile everything.

To compile Yocto, you’ll have to install some packages on a classic distribution. We’ll therefore create an image containing all these packages. Let’s start by choosing a distribution from the list:

FROM ubuntu:18.04

Next, let’s install all the desired packages on the image. The Yocto manual provides the following list:


RUN apt-get update && apt-get install -y gawk wget git-core diffstat unzip \

            texinfo gcc-multilib build-essential chrpath socat cpio python \

            python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping \

            python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm

If you only install the above, you’ll get the following error message when compiling Yocto:

Please use a locale setting which supports UTF-8 (such as LANG=en_US.UTF-8).
Python can’t change the filesystem locale after loading so we need a UTF-8 when Python starts or things won’t work.

You’ll therefore have to install the locales. The command becomes:


RUN apt-get update && apt-get install -y gawk wget git-core diffstat unzip \

            texinfo gcc-multilib build-essential chrpath socat cpio python \

            python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping \

            python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm locales

Next, we’ll create a user called “dev” to compile Yocto. Let’s use the following command:


RUN groupadd -g 1000 dev \

            && useradd -u 1000 -g dev -d /home/dev dev \

            && mkdir /home/dev \

            && chown -R dev:dev /home/dev

And generate the locales:


RUN locale-gen en_US.UTF-8

We must also define the locale we wish to use:


ENV LANG en_US.UTF-8

And change the username:


USER dev

Finally, we’ll define the work directory:


WORKDIR /home/dev

The full Dockerfile will therefore look like this:


FROM ubuntu:18.04

RUN apt-get update && apt-get install -y gawk wget git-core diffstat unzip \

            exinfo gcc-multilib build-essential chrpath socat cpio python \

            python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping \

            python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm locales

RUN groupadd -g 1000 dev \

            && useradd -u 1000 -g dev -d /home/dev dev \

            && mkdir /home/dev \

            && chown -R dev:dev /home/dev

RUN locale-gen en_US.UTF-8

ENV LANG en_US.UTF-8

USER dev

WORKDIR /home/dev

After saving this file by itself in a separate directory, we can run the command that will create our image:

docker build -t <image name>:<tag> .

We can then use this image to create a container:


docker run -i -t --name first-test :<tag>

To fetch Yocto and compile the core-image-minimal image for the qemux68 architecture, run the following commands:

git clone -b warrior git://git.yoctoproject.org/poky
cd poky
source oe-init-build-env
bitbake core-image-minimal

Once the compilation is complete, you’ll find the results of your compilation in the tmp/deploy/images/qemux86/ directory.

Bringing Improvements to your Yocto compilation

It won’t take long for you to notice the limitations of the above version.

Installing missing packages

If you look for vim or nano when editing a file, you’re not going to find them. You’ll therefore have to install one of these packages. What’s more, the free version does not come with the auto-completion feature. Instead, you’ll have to install the “bash-completion” package and configure your .bashrc. Finally, you may have tried to configure your Linux kernel with the following command:

bitbake -c menuconfig virtual/kernel

It will probably fail and generate the following message:

ERROR: linux-yocto-5.0.19+gitAUTOINC+31de88e51d_00638cdd8f-r0 do_menuconfig: No valid terminal found, unable to open devshell.
Tried the following commands:
            tmux split-window -c « {cwd} » « do_terminal »
            tmux new-window -c « {cwd} » -n « linux-yocto Configuration » « do_terminal »
            xfce4-terminal -T « linux-yocto Configuration » -e « do_terminal »
            terminology -T= »linux-yocto Configuration » -e do_terminal
            mate-terminal –disable-factory -t « linux-yocto Configuration » -x do_terminal
            konsole –separate –workdir . -p tabtitle= »linux-yocto Configuration » -e do_terminal
            gnome-terminal -t « linux-yocto Configuration » -x do_terminal
            xterm -T « linux-yocto Configuration » -e do_terminal
            rxvt -T « linux-yocto Configuration » -e do_terminal
            tmux new -c « {cwd} » -d -s devshell -n devshell « do_terminal »
            screen -D -m -t « linux-yocto Configuration » -S devshell do_terminal
ERROR: linux-yocto-5.0.19+gitAUTOINC+31de88e51d_00638cdd8f-r0 do_menuconfig:
ERROR: linux-yocto-5.0.19+gitAUTOINC+31de88e51d_00638cdd8f-r0 do_menuconfig: Function failed: do_menuconfig
ERROR: Logfile of failure stored in: /home/dev/poky/build/tmp/work/qemux86-poky-linux/linux-yocto/5.0.19+gitAUTOINC+31de88e51d_00638cdd8f-r0/temp/log.do_menuconfig.1785
ERROR: Task (/home/dev/poky/meta/recipes-kernel/linux/linux-yocto_5.0.bb:do_menuconfig) failed with exit code ‘1’

You can install “screen”, “vim”, and “bash-completion” to fix these problems. The line in our Dockerfile for installing the packages thus becomes:


RUN apt-get update && apt-get install -y gawk wget git-core diffstat unzip \

            texinfo gcc-multilib build-essential chrpath socat cpio python \

            python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping \

            python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm locales \

            vim bash-completion screen

Next, go ahead and copy the .bashrc in the container:

COPY ./bashrc /home/dev/.bashrc

And create a “bashrc” file alongside your Dockerfile that contains:

# enable programmable completion features (you don’t need to enable
# this, if it’s already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if ! shopt -oq posix; then
            if [ -f /usr/share/bash-completion/bash_completion ]; then
                        . /usr/share/bash-completion/bash_completion
            elif [ -f /etc/bash_completion ]; then
                        . /etc/bash_completion
            fi
fi

Securing your work by sharing it between the host machine and container

The result of the compilation only appears in the tree directory of the container. If you delete the container, you’ll lose all of the work done. You may therefore wish to share this data with the host machine:


docker run -i -t --name first-test -v <host directory>:/home/dev  yocto-compile

As a result, any changes you make to your container’s /home/dev directory will also take place in the designated host directory. This may cause issues regarding access rights. We only set the PUID in the Dockerfile and the PGID was already set to 1000.

If the user of the host machine uses different IDs, you’re likely to run into problems when trying to write in your host machine’s tree directory from the container. We’ll therefore pass the PUID and PGID as parameters to the Dockerfile.

By the way, if you create the image for yourself, you can use the same username and the same path between the host machine and the container for the project. This will ensure there is a valid absolute path both on your host machine and in your container, thereby making it easier for you.

If a compilation error occurs, Yocto usually provides a path to a log file where the error is stored. As this path is an absolute path, you can open it on your host machine without having to change it. The Dockerfile will therefore look like this:


FROM ubuntu:18.04



RUN apt-get update && apt-get install -y gawk wget git-core diffstat unzip \

            texinfo gcc-multilib build-essential chrpath socat cpio python \

            python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping \

            python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm locales \

            vim bash-completion screen



ARG USERNAME=dev



ARG PUID=1000



ARG PGID=1000



RUN groupadd -g ${PGID} ${USERNAME} \

            && useradd -u ${PUID} -g ${USERNAME} -d /home/${USERNAME} ${USERNAME} \

            && mkdir /home/${USERNAME} \

            && chown -R ${USERNAME}:${USERNAME} /home/${USERNAME}



RUN locale-gen en_US.UTF-8



ENV LANG en_US.UTF-8



COPY ./bashrc /home/${USERNAME}/.bashrc



USER ${USERNAME}



WORKDIR /home/${USERNAME}

And the command for generating the image as follows:


docker build -t <image name>:<tag> --build-arg USERNAME=<user name> --build-arg PUID=<puid> --build-arg PGID=<pgid>  .

Since the data has been shared with the host machine, you can destroy the container once you have exited it. The command for creating the container therefore becomes:


docker run -it --rm -v <working directory>:<working directory> <image name>:<tag>

You can subsequently compile Yocto by running:

cd <working directory>
git clone -b warrior git://git.yoctoproject.org/poky
cd poky
source oe-init-build-env
bitbake core-image-minimal

Creating scripts to automate and gain time

In this last section, we’ll create a few shell scripts to make our life easier.

Here they are:

StartBuild.sh, which will allow us to start the compilation by creating a container with the right parameters
yocto-entrypoint.sh, or the script which will let us fetch sources and start the compilation of Yocto
docker-entrypoint.sh, a simple script that serves as the entry point to the container

yocto-entrypoint.sh may need to be updated over time. We’ll use docker-entrypoint.sh (which will serve as the entry point to the container) to avoid having to generate a new image for each modification.

The main role of this script will be to call yocto-entrypoint.sh, which will be passed to the container by using the -v option of the docker run command. StartBuild.sh will execute the docker run with the right arguments.

We’ll start by creating a docker-entrypoint.sh script alongside our Dockerfile. It will contain:


#!/bin/bash

set -e

[ $# -eq 0 ] && set -- yocto

if [ "$1" = "yocto" ]; then

    shift; set -- ${YOCTO_ENTRYPOINT} "$@"

fi

exec "$@"

As you can see, if we do not pass parameters or the first parameter of the script is yocto, then we’ll call the Yocto compilation script whose path is made available by the environment variable YOCTO_ENTRYPOINT.

We’ll create the yocto-entrypoint.sh script in our work directory (a separate directory that does not include the Dockerfile):


#!/bin/bash



SUPPORTED_MACHINES="

            qemuarm

            qemuarm64

            qemumips

            qemumips64

            qemuppc

            qemux86

            qemux86-64

            beaglebone-yocto

            genericx86

            genericx86-64

            mpc8315e-rdb

            edgerouter

"



SUPPORTED_YOCTO="

            thud

            warrior

"

WORKDIR="/"



yocto_sync()

{

            if [ -d ${WORKDIR}/yocto/sources/poky ]; then

                        cd ${WORKDIR}/yocto/sources/poky

                        git fetch origin

                        git checkout origin/$1

            else

                        mkdir -p "${WORKDIR}/yocto/sources"

                        cd "${WORKDIR}/yocto/sources"

                        git clone -b $1 git://git.yoctoproject.org/poky.git

            fi

}



is_in_list()

{

            local key="$1"

            local list="$2"



            for item in $list; do

                        if [ "$item" = "$key" ]; then

                                    return 0

                        fi

            done



            return 1

}



is_machine_supported()

{

            local MACHINE="$1"



            is_in_list "$MACHINE" "$SUPPORTED_MACHINES"

}



is_yocto_supported()

{

            local YOCTO="$1"



            is_in_list "$YOCTO" "$SUPPORTED_YOCTO"

}







show_usage()

{

            echo "Usage: StartBuild.sh command [args]"

            echo "Commands:"

            echo "    sync  "

            echo "        Sync Yocto version"

            echo "        E.g. sync quemux86 warrior"

            echo

            echo "    all"

            echo "        Build Yocto"

            echo

            echo "    bash"

            echo "        Start an interactive bash shell"

            echo

            echo "    help"

            echo "        Show this text"

            echo

            exit 1

}





main()

{

            if [ $# -lt 1 ]; then

                        show_usage

            fi



            if [ ! -d "${WORKDIR}/yocto/sources" ] && [ "$1" != "sync" ]; then

                        echo "The directory 'yocto/sources' does not yet exist. Use the 'sync' command"

                        show_usage

            fi





            case "$1" in

                        all)

                                    cd ${WORKDIR}/yocto/

                                    source sources/poky/oe-init-build-env build

                                    bitbake core-image-minimal

                        ;;



                        sync)

                                    shift; set -- "$@"

                                    if [ $# -ne 2 ]; then

                                                echo "sync command accepts only 2 arguments"

                                                show_usage

                                    fi



                                    if ! is_machine_supported "$1"; then

                                                echo "$1 is not a supported machine: ${SUPPORTED_MACHINES}"

                                                show_usage

                                    fi



                                    if ! is_yocto_supported "$2"; then

                                                echo "$2 is not a supported yocto version: ${SUPPORTED_YOCTO}"

                                                show_usage

                                    fi



                                    yocto_sync $2



                                    cd "${WORKDIR}/yocto"

                                    source sources/poky/oe-init-build-env build

                                    sed -i "s/^MACHINE ??= .*$/MACHINE ??= \"$1\"/" conf/local.conf

                        ;;



                        bash)

                                    cd "${WORKDIR}/yocto"

                                    source sources/poky/oe-init-build-env build

                                    bash

                        ;;



                        help)

                                    show_usage

                        ;;



                        *)

                                    echo "Command not supported: $1"

                                    show_usage

            esac

}



main $@

In the main function, there are three primary events:

all, which lets you start the compilation of a core-image-minimal image.
sync, which parameterizes the machine and the desired Yocto version. This lets you fetch or update the sources and modify the configuration file so the right machine is used.
Bash, which lets you source the environment and access a bash. We can then immediately run the bitbake commands manually.

You may have noticed that the WORKDIR variable is currently defined as “improperly configured”. It will be configured by the StartBuild.sh script, which we will create in our work directory alongside the yocto-entrypoint.sh script:


#!/bin/bash



WORKDIR="$(pwd)"



sed -i "s|^WORKDIR=.*$|WORKDIR=\"${WORKDIR}\"|" ./yocto-entrypoint.sh



docker run -i -t --rm \

            -v ${WORKDIR}:${WORKDIR} \

            -v $(pwd)/yocto-entrypoint.sh:/yocto-entrypoint.sh \

            yocto-compile:latest \

            yocto $@

You can see that the name of the image must be “yocto-compile” and the tag must be “latest”. Otherwise you must modify the script. This script modifies the WORKDIR variable from the yocto-entrypoint.sh script. WORKDIR is the directory that will be shared between the host machine and the container.

Naturally, you’ll have to modify the Dockerfile to:

define the environment variable YOCTO_ENTRYPOINT
add the docker-entrypoint.sh script to the image and define it as the entry point.

The Dockerfile will then look like this:


FROM ubuntu:18.04



RUN apt-get update && apt-get install -y gawk wget git-core diffstat unzip \

            texinfo gcc-multilib build-essential chrpath socat cpio python \

            python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping \

            python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev xterm locales \

            vim bash-completion screen



ARG USERNAME=dev



ARG PUID=1000



ARG PGID=1000



ENV YOCTO_ENTRYPOINT ${YOCTO_ENTRYPOINT:-/yocto-entrypoint.sh}



RUN groupadd -g ${PGID} ${USERNAME} \

            && useradd -u ${PUID} -g ${USERNAME} -d /home/${USERNAME} ${USERNAME} \

            && mkdir /home/${USERNAME} \

            && chown -R ${USERNAME}:${USERNAME} /home/${USERNAME}



RUN locale-gen en_US.UTF-8



ENV LANG en_US.UTF-8



COPY ./bashrc /home/${USERNAME}/.bashrc



USER ${USERNAME}



WORKDIR /home/${USERNAME}



COPY ./docker-entrypoint.sh /



ENTRYPOINT ["/docker-entrypoint.sh"]

Next, you must grant the three new bash scripts permission to run the command:

chmod +x <script name>

In summary, if:

the username is dev
the user has a PUID of 1000
the user has a PGID of 1000
the desired name for the Docker image is “yocto-compile”
the desired tag for the Docker image is “latest”
the target machine is qemuarm
the branch of poky is warrior

You can generate the image by entering the directory containing the Dockerfile and running the following command:


docker build -t yocto-compile:latest --build-arg USERNAME=dev --build-arg PUID=1000 --build-arg PGID=1000  .

Then, in the work directory containing the StartBuild.sh and yocto-entrypoint.sh scripts, you can fetch the sources and configure the machine with:

./StartBuild.sh sync qemuarm warrior

And start the compilation with the command:

./StartBuild.sh all

Getting rid of a common problem when compiling Yocto

You may encounter the following problem while compiling Yocto:

ERROR: No space left on device or exceeds fs.inotify.max_user_watches?

According to the message, the problem may have one of two origins:

you’re out of disk space
you’ve reached the inotify limit set by the kernel (bitbake uses inotify watches on Yocto configuration files and recipes)

The first step then is to determine which of these is causing the problem. To find out if your disk is full, run the following command:

df -h

If the disk turns out to be full, you’ll have to free up some space. You can also visit Yocto Project’s Reference Manual.

For older versions of Yocto, you can use RM_OLD_IMAGE.

If your disk is not the source of the problem, then you’ll have to increase the number of authorized inotify watches. To do so, first determine your current number of watches with:

sysctl fs.inotify.max_user_watches

Then, increase this value. By default, my distribution indicates 8192. We can set it to 65536 with the following command:

sudo sysctl fs.inotify.max_user_watches=65536

I’ve just demonstrated how easy it is to compile Yocto without having to worry about software compatibility issues. You may have to modify your Dockerfile or scripts to fit your own particular project, or make a few improvements such as: