Cross Compiling For Raspbian

As promised in the previous blog post I will outline how we can leverage multiarch in order to speed up the compilation process for the Raspberry Pi.

Raspbian is mainly a recompiled Debian. The recompilation was necessary because the official Debian armhf port requires an ARMv7-A capable CPU while the Pi 1 and Pi Zero are only equipped with an ARMv6 capable CPU.

I rather prefer to automate things instead of writing lengthy instructions and therefore a few steps will be sufficient to start the cross compilation adventure on an Ubuntu (16.04 or newer, amd64 architecture required) host:

Update - 26-March-2018

Due to the great demand, I finally upgraded the setup to Raspbian stretch! You can now enjoy gcc 6.3 within the cross compilation container. The instructions below do now reflect this version change.

Update - 18-October-2019

An updated setup for Raspbian buster is available!

Container Setup

The following repository contains the container setup instructions:

git clone https://github.com/lueschem/edi-raspbian.git
cd edi-raspbian
git checkout stretch

Under the assumption that edi is already installed, we can directly generate our mulitarch cross compilation container:

sudo edi -v lxc configure raspbian-stretch-cross stretch-cross.yml

And enter it (password is ChangeMe!, you can change it using passwd):

lxc exec raspbian-stretch-cross -- login ${USER}

That’s it - we are ready to cross compile a program:

Hello World

We change into the shared workspace folder edi-workspace and within the subfolder hello we write some code:

mkdir -p ~/edi-workspace/hello
cd ~/edi-workspace/hello
cat << EOF > hello.cpp
#include <iostream>
 
int main()
{
    std::cout << "Hello Raspbian!" << std::endl;
    return 0;
}
EOF

And we compile it using the pre installed C++ cross compiler:

arm-linux-gnueabihf-g++ -v hello.cpp -o hello-raspbian

And obviously - we want to test our brand new executable:

./hello-raspbian

But wait - why were we able to run this binary? Just in case we do not trust the cross compiler we can verify that it is indeed an armhf binary: file hello-raspbian. If you are still thinking that this executable is not trustworthy then you are welcome to download it to your Raspberry Pi and execute it there.

If you really want to know why we were just able to run this armhf binary within the amd64 container then you can run the following command:

 strace ./hello-raspbian 2>&1 | grep qemu-arm-static

Maybe we are now curious where we got this shiny cross compiler from. The following commands

dpkg -S $(realpath $(which arm-linux-gnueabihf-g++))
apt-cache policy g++-6-arm-linux-gnueabihf

will reveal that the cross compiler got downloaded from a custom repository. The reason for having this custom repository is that I had to recompile some Debian binaries in order to garantee correct cross compilation with the options -march=armv6 -mfpu=vfp instead of -march=armv7-a -mfpu=vfpv3-d16 -mthumb.

Obviously, a hello world program is boring and we will now do something that will break our Raspberry Pi ssh access if we should have done something wrong with the cross toolchain:

Recompilation of the SSL Library

Like within the previous blog post we fetch the source code:

mkdir -p ~/edi-workspace/openssl-cross && cd ~/edi-workspace/openssl-cross
apt-get source libssl1.1
cd openssl-1.1.0f

And we compile it:

export DEB_BUILD_OPTIONS=nocheck; time debuild -us -uc -aarmhf

As expected, this was seven times faster than on the Raspberry Pi 3 Model B.

Now it is time for the risky part of the game. We copy this library to the Raspberry Pi using ssh and we hope that the subsequent installation will not brick our ssh access:

scp ../libssl1.1_1.1.0f-3+deb9u1_armhf.deb pi@RASPBERRY_PI_IP_ADDRESS:
ssh pi@RASPBERRY_PI_IP_ADDRESS
sudo dpkg -i libssl1.1_1.1.0f-3+deb9u1_armhf.deb

You have been warned - I really bricked my ssh access once when I tried to install binaries that were compiled without the adjusted compiler settings.

Let us see whether we have more luck this time:

sudo systemctl restart ssh
exit
ssh pi@RASPBERRY_PI_IP_ADDRESS
exit

Uff - everything went well!

The following example will show us the cross compilation of a package with slightly more dependencies:

Recompilation of man-db

First we fetch the source code of man-db (the tools that deal with man pages). This was a truly random choice of a package that contains executables. Within the container raspbian-jessie-cross we execute the following commands:

cd ~/edi-workspace
apt-get source man-db
cd man-db-2.7.6.1 

man-db is with regards to dependencies slightly more complex than the openssl library and therefore we need some build dependencies:

sudo apt-get build-dep -a armhf man-db

Now we have green lights for the man-db build:

debuild -us -uc -aarmhf

Out of curiosity we can install this package on the Raspberry Pi:

scp ../man-db_2.7.6.1-2_armhf.deb pi@RASPBERRY_PI_IP_ADDRESS:
ssh pi@RASPBERRY_PI_IP_ADDRESS
sudo dpkg -i man-db_2.7.6.1-2_armhf.deb

With the commmand man man we can verify that the cross compiled binary seems to work fine on the Raspberry Pi.

Conclusion

With the above description only a few steps are needed to prepare a cross development toolchain for Raspbian. The tricky part of the setup is hidden behind an Ansible playbook that is kicked off by the tool edi. If you are interested in the details you can take a look at the playbook that is part of the edi-raspbian git repository.

With the very same toolchain I have also successfully compiled a recent Raspberry Pi kernel.

Everything above is still new and experimental and I do not expect it to work under all circumstances. I am sure that there are more knowledgeable people around than me who can improve the above setup. Pull requests or constructive comments are highly appreciated!

Further Reading

Please read this blog post if you are interested in running pure Debian on a Raspberry Pi 2 or 3.

If you would like to add your favourite integrated development environment (IDE) to your cross compilation toolchain you can read on here.

You can also take a look at the presentation I did for an embedded GNU/Linux developer meetup.