Cross-Platform Options in Go

One of the key strengths of the Go programming language is its powerful tooling focused on developer's productivity. Fast compilation and ease of dependency management are vital for the ecosystem. But tools for writing and building multi-platform code are what make Go truly special.

Writing and maintaining cross-platform software (without distribution) is a two-stage process:

  • the code needs to be written in a way that allows changing its behaviour based on the platform it's running on
  • the code then needs to be compiled for each supported platform.

We talk about the second stage first, and then dig deeper into the first.

Terminology

Before we get too far in the discussion, it's good to agree on the terminology used in next sections.

In this work, as well as in the community, we use the following terms with the corresponding meanings:

  • Go operating system, or simply operating system, as per the environment variable GOOS – one of the target operating systems Go supports. Expressed as one of the predefined values.
  • Go architecture, or simply architecture, as per the environment variable GOARCH – one of the the target system architectures Go supports. Similarly to the operating system, it's one of the predefined values.
  • Go platform, or just platform, a combination of the two, an operating system and architecture, in this order. This term is not used in code or as a setting, but is used in discussions and documentation. When you see a use of the term "platform" in a conversation that is related to cross-platform software in Go, it means a combination of an operating system and architecture. For instance, these sentences are written on the darwin/amd64 platform.

It's worth reminding that each of GOOS and GOARCH and, therefore, the platform, can, and usually is, defined implicitly by inferring the value from the environment, if not set explicitly. When you're running go build on your Intel-based Mac without setting the environment variables, GOOS=darwin and GOARCH=amd64 are supplied to the compiler. In this case, the target platform is darwin/amd64. On a Raspberry Pi under Linux, it would be linux/arm and additional GOARM. Similar effect can be achieved on any other platform with variables explicitly set as follows GOOS=linux GOARCH=arm GOARM=6.

While we're on this topic, it's no harm in repeating that when only one of the variables specified to a value that is different from the current system value, the other is inferred from the current system. For the rest of the book, if not mentioned explicitly, the implied value of GOARCH is amd64.

Okay, now we can move forward. The next section is a quick refresher of the compilation process, and then we dive into creating packages that support several platforms.

Compiling For Multiple Platforms

Even if a project has no code that depends on a platform, the binary format has to match the requirements and expectations of a target system. In Go, there are two options for compiling code for more than one platform:

  • simply building on each of the target platforms
  • cross-compile for each of the supported platforms.

With the first option, the code for each of the supported platforms is built on that platform. In its simplest form, the build process looks no different from what we do every day – go build is go build, after all. What differs is the artefact which is specific to the platform it has been built on.

The second option offers a slightly different approach. The result is controlled by a pair of well-known environment variables that tell the compiler what it should produce. This process is also known as cross-compilation:

GOOS=freebsd GOARCH=amd64 go build -o myapp-freebsd-amd64
GOOS=linux GOARCH=arm go build -o myapp-linux-arm
GOOS=windows GOARCH=386 go build -o myapp-windows-386.exe
GOOS=darwin GOARCH=amd64 go build -o myapp-darwin-amd64

Which of the two options when to use? It depends on your environment, priorities, available resources and limitations. The native compilation option is the easiest from the tooling perspective. You simply run the same set of tools to test and build code, and get the results. A major downside is the need to run and maintain instances of all platforms, which might be expensive as it requires some (potentially, significant) additional resources (i.e. costs you time, attention and money).

On the other hand, the second option is less demanding to the build infrastructure. All code is built on the developers's machine or in a single instance of CI/CD. Platform-agnostic code is tested on the most available platform, but running tests for platform-specific code still requires native instances.

So choose wisely. The guideline is to use cross-compilation for producing release artefacts, and to use specialised, automated test environment to run tests against platform-dependent code.

It's worth noting that, in its simplest form, a cross-platform program has no differences in the implementation between supported platforms (not counting potential differences in the standard library). In such case, all what is needed is compiling a binary for each platform. That's exactly what we've just covered.

Compiling code for multiple systems is a relatively easy step. But to build something, we have to write it first. The question is how to do it, and do it in a right way.

Developing for Multiple Platforms

When working on functionality which involves special features of an operating system, a specialised implementation is required for the varying part. Then, different implementations need to be incorporated with the platform-independent business logic of the project.

Go offers build constraints to enable conditional compilation. This allows to write specialised code for a particular platform, and guarantees that the code is included and available only when compiled for that target system. For all other platforms, which are not covered by constraints, such code simply does not exist.

Build constraints can be expressed in two ways:

  • using the file name
  • using build tags.

When compiling, the two environment variables GOOS and GOARCH and/or the build flag -tags are used to control the process. Specifically, these settings tell the compiler when and what to include in the resulting binary.

As you will learn or remember soon, while the two ways under the hood work in the same way (due to the go/build package), in everyday development each of them has own pros and cons. They can be used independently in simple scenarios. More often though, they're used together as it allows more flexible behaviour, but sometimes it's just the only way to express the constraints. Let's briefly recap what each of them is, and then learn how to employ them to make the design of a package better.

NOTICE: The contents below covers topics such as using build tags and file names for conditional compilation. The text has been written when Go 1.16 has been released. Everything below applies for Go versions up to 1.16. Starting from version 1.17, things will change. There is a proposal which has been accepted. This is going to be quite a large language change, on par with introduction of Modules. I will keep this part of the unit updated as the new behaviour is available, and it's clear what happens to the existing approach. For now, we focus on the existing way of managing compilation.

The File Name

We start with the method based on the file name suffix. If a source file has a suffix that is a valid operating system, architecture or a combination of the two, then the file is compiled only for that platform.

Here are some examples, for a package mypkg:

  • mypkg_linux.go is included in the binaries for Linux and any architecture (of course, for those that are supported by Go)
  • mypkg_amd64.go is included in binaries for any operating system running on the amd64 architecture (also known as x86_64, Intel 64, but these are not valid names in Go)
  • mypkg_windows_386.go is included in the binary only when targeted at x86-based 32-bit Windows
  • mypkg_posix.go is included on any platform (remember, platform = os + arch, hence on any os and any arch) as posix is not a valid operating system nor architecture identifier for the Go toolchain.

When to include a file is controlled by the environment at the build time:

  • when running go build the values are derived from the system the build is running on, i.e. your machine or the machine the command is executed on
  • or when GOOS and/or GOARCH are explicitly set.

This also works with testing. To write a platform-specific test you simply add the familiar test suffix to the end of the corresponding file name or to any file that will include such code:

  • mypkg_linux_test.go is included only when running go test on Linux, or GOOS=linux go test on any system
  • mypkg_amd64_test.go is included only when running go test on an x86-based 64-bit system, or GOARCH=amd64 go test on any other architecture
  • mypkg_windows_386_test.go is included only when running go test on an x86-based 32-bit Windows, or GOOS=windows GOARCH=386 go test on any system
  • and so on.

For all available GOOS and GOARCH valid values and combinations please visit this page.

As you see, the approach offers an easy way for separating code for a particular system. It's enough in simple cases or when code is different between all the systems your project works on. But what if a more fine grained control is needed? Or what if code is the same for Windows and Linux, but is different for macOS?

Another use case is including/excluding code at the compilation time based not only on a platform, or even not on the platform at all, but on a custom criterion. For example, you may want to include some features in the free version of your product, and other features only in the binary shipped to the paying users. You may even have multiple paid tiers with three sets of features, and the corresponding binaries should include all features from the preceding tiers plus something else. Or you may even want to offer different implementations of the same feature based on the tier?

The answer to these and many other questions about conditional compilation is build tags.

Build Tags

Build constraints in Go are expressed in a form of build tags. As you may already know, a build tag is a line comment that begins with // +build and lists the conditions under which a file should be included when compiling and/or testing the project.

Basics

The following rules are in effect when using build constraints:

  • multiple tags may be listed on the same line
  • multiple lines with build tags may appear, one next to another
  • tags may appear in any kind of source file (not limited to Go)
  • constraints must appear close to the top of the file
  • they may be preceded only by blank lines and other comments
  • a series of build tag lines must be followed by an empty line (to distinguish from the documentation)
  • all this means that in Go files, build constraints must appear only before the package clause.

There are also certain rules for writing and grouping build tags:

  • when listed on the same line, space-separated tags are interpreted as OR
  • if an option contains a comma, then it's evaluated as AND of its separated terms
  • allowed symbols for a term are letters, digits, underscores and dots
  • a term can be negated when preceded with an exclamation mark !
  • when constraints are expressed as multiple lines, the overall result is the AND of individual lines
  • a special tag of // +build ignore can be used to exclude the file from consideration.

Here are some examples:

  • a simple build constraint
// +build linux,arm windows,386

It results to the following: (linux AND arm) OR (windows AND 386).

  • multiple build tags
// +build linux darwing
// +build amd64

It results to the following: (linux OR darwin) AND (amd64)

  • with negation
// +build linux darwin,!cgo

Which results to (linux) OR (darwin AND NOT cgo).

More Control

The use of build tags is not limited to only platforms. Other conditions can be expressed in the very same way, for even more advanced control over the compilation. In the example below, we want to offer three tiers in the service – free, silver and gold. Each subsequent tier should include the features from the preceding tier. This is how it can be achieved on the compilation level which may help protect your app against cracking and/or bypassing licensing terms:

  • define a base line, with no tags, in base.go
  • define a the first paid tier, or silver level, in silver.go
// +build silver
  • define a the second paid tier, gold, in gold.go
// +build silver
// +build gold

Notice that we have to list the both tags, silver and gold for the second tier, as we want it to include the features from the preceding tier. That's because of the way how the boolean logic is expressed using the constraints.

Passing Tags

Now, how to tell compiler about constraints?

Build constraints are inferred from three places, two of which you already know:

  • some environment variables, such as GOOS, GOARCH, CGO_ENABLED, etc
  • the suffix of the name of a Go source file
  • when explicitly passed as a special build flag of comma separated values when running build, test and other commands, as -tag tag1,tag2,tag3.

Note that the file name's suffix method is mentioned here. That's because, as said earlier, under the hood it works as a build constraint. When a file has a suffix that matches a valid supported operating system, architecture, or a combination of the two, the file is considered to have an implicit build constraint set to the values in the suffix.

Using Only Build Tags

Having reminded ourselves what a build tag is, let's continue exploring options for cross-platform development.

As mentioned earlier, conditions for platform-dependent compilation can be expressed using file name suffixes. In a basic scenario, when the implementation details are different for, say, windows and linux, it's easy to express with mypkg_windows.go and mypkg_linux.go, in addition to mypkg.go.

However, what to do when two different platforms have the exact same implementations? One way is to simply have two separate files with same contents. For example, app_darwin.go and app_freebsd.go. But this kind of duplication is not something we normally want, unless we have to.

A much better approach is to use build tags for expressing conditions for the compilation process. It helps in making things cleaner and removes the need of duplicating the code. Add a build tag to the file, and give it a meaningful name that does not conflict with the supported operating systems, say app_unix.go:

// +build darwin freebsd

Now whenever the code is built on each of the mentioned platforms, or with the help of the environment variable GOOS, code will be included in binaries for either of the operating systems.

Using Suffixes and Tags

Finally, let's consider another possible situation where a feature's implementation is different for some platforms, but is the same for others. The suffix-based method can be used in combination with build tags.

To illustrate this situation, let's assume the low-level implementation is different for Windows on the one hand, and Linux with macOS on the other, though the latter is the same for Linux and macOS. Then a reasonable approach would be to put the Windows-specific code into myapp_windows.go file, and let the suffix control inclusion of this code. For Linux and macOS, put implementation into myapp_posix.go and add the following build constraint at the top of the file:

// +build linux darwin

And that's it. The environment variables and/or build flags passed to go build or go test are now in control of what's included during the process.

Notes

As you see, Go offers convenient tools for writing, building and shipping cross-platform software. Those who wrote programs for more than one platform in the past agree that Go has made huge progress in making the process easier. Those who haven't got cross-platform experience yet would probably not be surprised, if the first experience happened to be with Go. But then they would definitely notice the difference if faced cross-platform development with other languages.

These tools are great, but that's not enough on its own to produce great software and developer experience in short and long term. To achieve a good level of efficiency we need something else, and that's what the next section is about – actual advice on cross-platform packages.