Provide Cross-Platform Tests Only When You Must

Provide platform-specific tests only when you must.

If we had lived in a perfect world, platform-agnostic tests would have been enough. Since this is not the case, under some circumstances we have to provide our code with specialised tests. There are two main reasons for having specialised tests:

the feature being tested is not available on all platforms, hence there is no way to test it uniformly in accordance with the previous guideline
or the platform-specific implementation detail is crucial, and requires an extra cautious approach.

Note that the latter of the two reasons is rather rare for most cases of casual and business software. A good illustration of this and the preceding guidelines can be found in the net package of the Go's standard library. The vast majority of the package's tests operates on platform-independent code, and only a few parts are provided with specialised tests.

When a single or a group of platforms needs a separate set of tests, follow the same rules as with regular cross-platform code:

in simple cases, use file name suffixes, e.g. something_darwin_test.go
in cases where a number of platforms share the same implementation and tests, use a meaningful suffix and build tags, e.g. something_unix_test.go and // +build darwin freebsd.

To give an example, let's add a test to the feature that is unique for the darwin platform in the demo code shown previously. As a reminder, this is the platform-specific implementation we're going to test, something/something_darwin.go:

package something

import (
	"fmt"
)

var (
	feature4 = []byte(`this feature has different implementations but yields the same result`)
)

func DoFeature3() {
	fmt.Println("this is a unique feature for the mac")
}

func doFeature4() string {
	return string(feature4)
}

In particular, we're interested in the DoFeature3 function, as it's available only on the Mac; on all other platforms it's mocked as func DoFeature3() {}. The implementations are different, so there is no way it can be expressed in a clean way. Of course, one can always write if runtime.GOOS() == "darwin" { ... }, but that's rather naïve. Why would we allow such a leak of details to a higher level of abstraction?

Instead, we create a file to hold the tests for the platform of interest, something_darwin_test.go in our case, and fill it with test code:

package something_test

import (
	"bufio"
	"os"
	"strings"
	"testing"

	"../something"
)

func TestDoFeature3(t *testing.T) {
	stdout := os.Stdout
	r, w, err := os.Pipe()
	if err != nil {
		t.Errorf("failed to create reader and writer: %s\n", err)
		t.FailNow()
	}

	os.Stdout = w
	something.DoFeature3()

	b := bufio.NewReader(r)
	out, err := b.ReadString('\n')
	if err != nil {
		t.Errorf("failed to read from reader: %s\n", err)
		t.FailNow()
	}

	if err := w.Close(); err != nil {
		t.Errorf("failed to close writer: %s\n", err)
		t.FailNow()
	}

	os.Stdout = stdout

	exp := "this is a unique feature for the mac"
	if act := strings.TrimSuffix(out, "\n"); act != exp {
		t.Errorf("expected: %q, got: %q\n", exp, act)
	}
}

Running the tests emits the following output:

on macOS

$ GOOS=darwin go test ./something/... -v
=== RUN   TestDoFeature3
--- PASS: TestDoFeature3 (0.00s)
=== RUN   TestFeature4
--- PASS: TestFeature4 (0.00s)
=== RUN   TestRunFeature4
--- PASS: TestRunFeature4 (0.00s)
PASS
ok  	_/Users/user/projects/xplatform/something	0.006s

and on Linux, the output is still the same as before

$ docker run -it --rm -v $(pwd):/root/xplatform golang:1.15 bash
% cd /root/xplatform/
% go test ./something/... -v
=== RUN   TestFeature4
--- PASS: TestFeature4 (0.00s)
=== RUN   TestRunFeature4
--- PASS: TestRunFeature4 (0.00s)
PASS
ok  	_/root/xplatform/something 0.003s

Organising cross-platform code and tests this way allows for:

better encapsulation of responsibilities
better isolation of changes
more clarity on what, when, where and how
and this all results into a healthier software and process.

This example shows a couple of important things – that it is possible to have specialised tests if needed, and that the complexity and cost of maintenance grow much faster with each new varying detail. While being able to test the specifics of a platform is important, it's also important to use this ability wisely. The best code is unwritten code; but a better code is the one that checks for the outcome rather than for a specific implementation detail. This supports the guideline – write platform-specific tests only when you must, and try to cover as much as possible at the platform-independent level. It will save you time and energy.

Golocron – Software Development With Go

Provide Cross-Platform Tests Only When You Must