GoNIX and gokrazy: two ways to run Go on bare Linux

I have been using gokrazy for a couple of years, first for the photo display and since then as the deployment target for anything that has to sit on a Raspberry Pi and not be looked at. This week I came across GoNIX, which at first glance looks like the same idea: throw away the C userland, keep the Linux kernel, write everything above it in Go.

They are not the same idea. They share a kernel and a language and a dislike of “a patchwork of C accreted over decades”, and then head in opposite directions.

What each one is

gokrazy is Michael Stapelberg’s platform for deploying Go programs as appliances. You write a Go program, list it in an instance config, and gok builds an SD card or disk image holding the kernel, the firmware, a tiny Go init and your binaries. There is no shell, no package manager, no SSH, no libc. The root filesystem is read-only and updates replace it wholesale using A/B partitions. Persistent state goes on a separate /perm partition. You look at the box through a web page that shows each process’s status and lets you restart it. The pitch is that there are four moving parts, the kernel, the Pi firmware, the Go toolchain and gokrazy itself, and the project keeps all four current for you with kernels typically landing within a day of upstream.

GoNIX is James Mills’s (prologic’s) attempt at a “Go-first UNIX”. It grew out of his earlier uLinux. PID 1 is Go and handles mounting, devices, networking, service supervision, gettys and shutdown. A single multi-call coreutils binary is symlinked under dozens of names, busybox style. DHCP, netlink, udev-style device nodes and an SSH server are all Go. But it is still a UNIX: you ssh root@ into it, you get a shell, and you run pkg install htop. The package manager and the ports tree come from uLinux and build C software against musl where there is no sensible Go alternative, so you can have a browser, a C toolchain, cron, syslog and even a labwc Wayland desktop. The stated goal is a base system you can “read end to end in an afternoon”, not a replacement for your daily-driver distro.

The actual difference

The line gokrazy draws is between your Go programs and everything else. Everything else is deleted. If it doesn’t build with CGO_ENABLED=0 it doesn’t exist, and that is treated as a feature. There is no way to interactively poke at a running system unless you deploy breakglass on purpose.

The line GoNIX draws is between the base system, which must be Go, and the ecosystem, which can be whatever serves users best. The base system is memory safe and legible; the stuff above it is ordinary packaged software. There is a shell because a UNIX has a shell.

So gokrazy is an appliance builder and GoNIX is an operating system. That makes them answer different questions:

Where they overlap: library images

The one place they genuinely compete is GoNIX’s library mode, which is new and is what made me look twice.

$ go install git.mills.io/prologic/gonix/cmd/gonix@latest
$ gonix pack .
gonix: wrote hello.iso (18.9 MB)
$ gonix run -p 8080:8080 hello.iso

That packs one Go program with a kernel built “for virtual machines and nothing else” and a minimal init called libinit into a hybrid BIOS/UEFI ISO. No shell, no coreutils, no userland. The init mounts the kernel filesystems, brings up the network, mounts a data disk at /data if there is one, runs your program, restarts it with backoff if it dies, and powers the machine off when nothing is left to run. Configuration is an env file of KEY=value lines that can arrive via QEMU, a NoCloud seed disk or the cloud metadata service.

That is very close to what gokrazy does, and the differences are telling:

• Target. gokrazy’s kernel and firmware work is aimed at Raspberry Pis with real Wi-Fi, Bluetooth and HDMI. GoNIX’s library kernel is aimed at virtio devices in a micro-VM. gokrazy will also run in QEMU and GoNIX will presumably boot on some metal, but each has a home and it isn’t the other’s.
• Lifecycle. gokrazy assumes the box is long-lived and you will push new builds to it with gok update. GoNIX library images assume the image is the version: a job VM that runs, exits, and the machine powers off. There is no documented update path because you are expected to repack and reboot.
• Observability. gokrazy gives you a web status page and remote syslog. GoNIX gives you the console and RFC 5424 syslog. Neither gives you a shell unless you ask (-debug for GoNIX, breakglass for gokrazy).

Which one I would use

For most of my work I am going to stay with gokrazy. Nearly everything I deploy is a Pi in my homelab running lots of Go services for years, and gokrazy has been doing exactly that for me. The A/B update story and the fact that someone else keeps the kernel current are the whole point, and the no-shell discipline has stopped me doing things on the box I would later regret. GoNIX would have to offer something gokrazy doesn’t for that kind of job, and so far it doesn’t.

For a Go program I want to run as a tiny VM on a cloud or under QEMU, GoNIX library images look like the quicker route. An 18 MB ISO that boots to my program in milliseconds, with cloud-init style configuration, is a nicer unit to hand to a hypervisor than a gokrazy disk image. I have not tried it yet; that is the next experiment.

For anything where I want to log in and look around, GoNIX proper is the only one of the two that is even trying. A minimal Go UNIX with a package manager is an interesting thing to have in the toolbox even if it never leaves QEMU.

The thing I keep coming back to is that gokrazy’s constraint, that nothing not written in Go may exist on the box, is what makes its guarantees cheap. GoNIX has chosen to give up that constraint above the base system in exchange for being a general-purpose OS. Both are honest about the trade, which is why it is worth knowing both.

Links

• GoNIX about, library images, source
• gokrazy, supported platforms, source
• My earlier gokrazy notes