Introduction¶

Load the Linux/GNU option. This will load a general Linux/GNU config. We’ll end up replacing the config but it gives us a good starting point. Ct-ng armv7-rpi2-linux-gnueabihf Note 8/29: this is a better starting point than my original suggestion. The config file below will change the remaining settings to accomodate for the different processor. Darwin/Mac OS X Cross-Compiler for Linux. This is a set of tools for creating a Linux-based distcc node for compiling Mac OS X software. It is comparable to XCode 2.2.1 (GCC 4.0.1 build 5250,.

This document will guide you in choosing the right Clang optionsfor cross-compiling your code to a different architecture. It assumes youalready know how to compile the code in question for the host architecture,and that you know how to choose additional include and library paths.

However, this document is not a “how to” and won’t help you setting yourbuild system or Makefiles, nor choosing the right CMake options, etc.Also, it does not cover all the possible options, nor does it containspecific examples for specific architectures. For a concrete example, theinstructions for cross-compiling LLVM itself may be of interest.

After reading this document, you should be familiar with the main issuesrelated to cross-compilation, and what main compiler options Clang providesfor performing cross-compilation.

Cross compilation issues¶

In GCC world, every host/target combination has its own set of binaries,headers, libraries, etc. So, it’s usually simple to download a packagewith all files in, unzip to a directory and point the build system tothat compiler, that will know about its location and find all it needs towhen compiling your code.

On the other hand, Clang/LLVM is natively a cross-compiler, meaning thatone set of programs can compile to all targets by setting the -targetoption. That makes it a lot easier for programmers wishing to compile todifferent platforms and architectures, and for compiler developers thatonly have to maintain one build system, and for OS distributions, thatneed only one set of main packages.

But, as is true to any cross-compiler, and given the complexity ofdifferent architectures, OS’s and options, it’s not always easy findingthe headers, libraries or binutils to generate target specific code.So you’ll need special options to help Clang understand what targetyou’re compiling to, where your tools are, etc.

Another problem is that compilers come with standard libraries only (likecompiler-rt, libcxx, libgcc, libm, etc), so you’ll have tofind and make available to the build system, every other library requiredto build your software, that is specific to your target. It’s not enough tohave your host’s libraries installed.

Finally, not all toolchains are the same, and consequently, not every Clangoption will work magically. Some options, like --sysroot (whicheffectively changes the logical root for headers and libraries), assumeall your binaries and libraries are in the same directory, which may nottrue when your cross-compiler was installed by the distribution’s packagemanagement. So, for each specific case, you may use more than oneoption, and in most cases, you’ll end up setting include paths (-I) andlibrary paths (-L) manually.

• be host/target specific or more flexible
• be in a single directory, or spread out across your system
• have different sets of libraries and headers by default
• need special options, which your build system won’t be able to figureout by itself

General Cross-Compilation Options in Clang¶

Target Triple¶

The basic option is to define the target architecture. For that, use-target<triple>. If you don’t specify the target, CPU names won’tmatch (since Clang assumes the host triple), and the compilation willgo ahead, creating code for the host platform, which will break lateron when assembling or linking.

• arch = x86_64, i386, arm, thumb, mips, etc.
• sub = for ex. on ARM: v5, v6m, v7a, v7m, etc.
• vendor = pc, apple, nvidia, ibm, etc.
• sys = none, linux, win32, darwin, cuda, etc.
• abi = eabi, gnu, android, macho, elf, etc.

The sub-architecture options are available for their own architectures,of course, so “x86v7a” doesn’t make sense. The vendor needs to bespecified only if there’s a relevant change, for instance between PCand Apple. Most of the time it can be omitted (and Unknown)will be assumed, which sets the defaults for the specified architecture.The system name is generally the OS (linux, darwin), but could be speciallike the bare-metal “none”.

When a parameter is not important, it can be omitted, or you canchoose unknown and the defaults will be used. If you choose a parameterthat Clang doesn’t know, like blerg, it’ll ignore and assumeunknown, which is not always desired, so be careful.

Finally, the ABI option is something that will pick default CPU/FPU,define the specific behaviour of your code (PCS, extensions),and also choose the correct library calls, etc.

CPU, FPU, ABI¶

Once your target is specified, it’s time to pick the hardware you’llbe compiling to. For every architecture, a default set of CPU/FPU/ABIwill be chosen, so you’ll almost always have to change it via flags.

• -mcpu=<cpu-name>, like x86-64, swift, cortex-a15
• -mfpu=<fpu-name>, like SSE3, NEON, controlling the FP unit available
• -mfloat-abi=<fabi>, like soft, hard, controlling which registersto use for floating-point

The default is normally the common denominator, so that Clang doesn’tgenerate code that breaks. But that also means you won’t get the bestcode for your specific hardware, which may mean orders of magnitudeslower than you expect.

For example, if your target is arm-none-eabi, the default CPU willbe arm7tdmi using soft float, which is extremely slow on modern cores,whereas if your triple is armv7a-none-eabi, it’ll be Cortex-A8 withNEON, but still using soft-float, which is much better, but still notgreat.

Toolchain Options¶

There are three main options to control access to your cross-compiler:--sysroot, -I, and -L. The two last ones are well known,but they’re particularly important for additional librariesand headers that are specific to your target.

There are two main ways to have a cross-compiler:

1. When you have extracted your cross-compiler from a zip file intoa directory, you have to use --sysroot=<path>. The path is theroot directory where you have unpacked your file, and Clang willlook for the directories bin, lib, include in there.

   In this case, your setup should be pretty much done (if noadditional headers or libraries are needed), as Clang will findall binaries it needs (assembler, linker, etc) in there.

2. When you have installed via a package manager (modern Linuxdistributions have cross-compiler packages available), makesure the target triple you set is also the prefix of yourcross-compiler toolchain.

   In this case, Clang will find the other binaries (assembler,linker), but not always where the target headers and librariesare. People add system-specific clues to Clang often, but asthings change, it’s more likely that it won’t find than theother way around.

   So, here, you’ll be a lot safer if you specify the include/librarydirectories manually (via -I and -L).

Target-Specific Libraries¶

All libraries that you compile as part of your build will becross-compiled to your target, and your build system will probablyfind them in the right place. But all dependencies that arenormally checked against (like libxml or libz etc) will matchagainst the host platform, not the target.

Cross Compile Linux Macos

So, if the build system is not aware that you want to cross-compileyour code, it will get every dependency wrong, and your compilationwill fail during build time, not configure time.

Also, finding the libraries for your target are not as easyas for your host machine. There aren’t many cross-libraries availableas packages to most OS’s, so you’ll have to either cross-compile themfrom source, or download the package for your target platform,extract the libraries and headers, put them in specific directoriesand add -I and -L pointing to them.

Also, some libraries have different dependencies on different targets,so configuration tools to find dependencies in the host can get thelist wrong for the target platform. This means that the configurationof your build can get things wrong when setting their own librarypaths, and you’ll have to augment it via additional flags (configure,Make, CMake, etc).

Multilibs¶

When you want to cross-compile to more than one configuration, forexample hard-float-ARM and soft-float-ARM, you’ll have to have multiplecopies of your libraries and (possibly) headers.

Some Linux distributions have support for Multilib, which handle thatfor you in an easier way, but if you’re not careful and, for instance,forget to specify -ccc-gcc-namearmv7l-linux-gnueabihf-gcc (whichuses hard-float), Clang will pick the armv7l-linux-gnueabi-ld(which uses soft-float) and linker errors will happen.

The same is true if you’re compiling for different ABIs, like gnueabiand androideabi, and might even link and run, but produce run-timeerrors, which are much harder to track down and fix.

In my previous posts, I came to the realization that the Raspberry Pi is not very fast! This results lots of chair spinning time while waiting for my projects to compile. A

After I did some brief research, I came across crosstool-ng. It enagbles anyone to create a toolchain to compile Raspberry Pi code directly on a (much faster) Macbook.

If you are unfamiliar with the process of compiling a toolchain on your own computer, let me be frank: it’s not fun.

Luckily, with this post you can get yours working in no time.

Update 8/29/16: I have updated this tutorial in several places for newer systems. I’m currently running a Macbook Pro with 10.11.6.

Note 8/29: This tutorial now is focused on the Raspberry Pi 3 B. Steps can be tweaked to account for older Pis

So without further ado lets do this thing.

Before we get started

Before we start anything I recently compiled the toolchain for RPi3. It will save you a whole bunch of time to download it here rather than go through this procedure.

These files include:

• Linux Kernel: 4.3

• hardfp: yes

• arch: armv8-a

• proc: cortex-a53

• glibc 2_22

• gcc 5.2.0

• binutils 2.25.1

• gdb 7.10

• gmp 6.0.0

• mpfr 3.1.3

• mpc 1.0.3

For those who want to continue for giggles, by all means…

Install Homebrew

You will need Hombrew to install some dependencies. If you haven’t already installed it you can run the command below:

Install crosstool-ng

Note: a few other dependencies get installed when crosstool-ng is compiled. Be prepared to wait a little while everything assembles.

Install gettext

Note: this is more of a precaution then a requirement. I believe by the end of this process that this was not necessary for getting crosstool-ng to work.

Create two case-sensitive disk images

Open up Disk utility. Click on the New Image button.

1. You need a disk at least 15GB in size. This will house all the source code and object files when all said and done.

2. The next disk can be exactly the same but only 250MB in size. (When fully compiled and compressed everything turned out to be around 107MB)

Note 8/29: for some reason there is a bug on OSX which prevents you from formating a case sensitive drive in Disk Utility. So create a non-case sensitive image and format it to a case sensitive one. The file system needs to be case sensitive.

Note* 8/29: Also, should you make a disk that is not the correct size, you can invoke the resize command to fix it!

Install GNU grep

Crosstools relies on the use of GNU grep. The grep built with OSX is not 100% exactly the same. So, let’s build it!

Note 8/29: this now can be done while installing crosstool-ng using the --with-grep option.

Edit paths.sh file

My paths.sh file was located here:

I changed the grep line from:

Cross Compiling For Linux On Mac Windows 10

To:

Note 8/29: likely can be avoided with the note above. You can also edit your .bash_profile to temporarily set which grep to use

Load the Linux/GNU option

This will load a general Linux/GNU config. We’ll end up replacing the config but it gives us a good starting point.

Note 8/29: this is a better starting point than my original suggestion. The config file below will change the remaining settings to accomodate for the different processor.

Install config file

Download the config file here.

You will have to copy it to your case sensitive disk image and rename it to .config.

Run Mac App On Linux

Modify the config file

Run the following in your working directory.

Change the following as needed. Note: This only needs to be changed if you change the names of the .dmg images.

Paths and misc options

Note: all of these are under the ** Paths ** section.

• Local tarballs directory

  I used /Volumes/xtools-build-env/src. Make sure you set yours to your setup.

• Working directory

  I used /Volumes/xtools-build-env/.build. Make sure you set yours to your setup.

• Prefix directory

  I used /Volumes/xtools/${CT_TARGET}. Make sure you set yours to your setup.

Note: the next few settings are under the ** Extracting ** section.

• Stop after extracting tarballs

  This option should be checked.

• Parallel jobs

  Note 8/29: new version already has this value set. You can leave it be.

Download and extract the packages

Run the following command:

The build command will stop after extracting all the sources.

Change source file

In ./.build/src/binutils-2.25.1/gold/gold-threads.cc you will need to change the file at line 284. Here is the before and after code blocks:

Change it to:

Update the ulimit

Ulimit controls the amount of resources allowed by a shell instance. In this case we need to increase this limit in order to prevent compilation errors.

Undo some the extract only config option

Undo one of the config settings we changed earlier. Open up:

Paths and misc options.

Note: the next few settings are under the ** Extracting ** section.

• Stop after extracting tarballs

  This option should be unchecked.

Begin the build!

Run:

Play the waiting game

Depending on how fast your setup is it may take a few hours to compile fully. If you’re impatient you can always get the binaries I just compiled here

In the end

By the time it’s done doing its thing you should have a fully capable cross platform toolchain for the Raspberry Pi! (Woot) An easy way to test it is to do the following:

(Hit ctrl-d to escape)

Copy test over to your Raspberry Pi.

Then ssh in and run the test executable

Other Notes

New notes as of 8/29 are as follows:

STOP/RESTART Crosstools now has a nifty stop and restart feature. Should a build break on a particular sub-component, you can actually fix the issue and continue the build from where it broke. It saves a ton of time. In order to take advantage of the feature you need to enable CT_DEBUG_CT_SAVE_STEPS in your .config

Then you can invoke the STOP or RESTART command:

ct-ng list-stepsct-ng build RESTART=cc_core_pass_1

Building Static Becuase OSX does not build based on static libraries we need to make sure those options are disabled. This is already done in my config file but for those who are interested here are the flags:

Thank you to Rolando for posting this in the comments!

Golang Cross Compile For Linux On Mac

Many thanks

I used several blog posts and articles over the web to get this to work. Many thanks to their previous efforts.

Cross Compile Linux Kernel On Mac Os X

Last Modified: 2020.3.7