Return to Plan 9

Plan 9 from Bell Labs has held the same charm after my last visit that took few days. This time I'll keep this operating system in an emulator where I can explore into it when I am distracted.

I downloaded 9front's iso image and retrieved papers and manuals for this system. I'm going to tell about the initial experiences that were left from running 9front on qemu, and explain why I am exploring into this system.

Installation

This system does not require an installation to a computer. It comes on a live CD and you can also use it to boot from an existing Plan 9 root server in your network.

The installation of Plan 9 was straightforward. There's a program guiding you through if you need it.

The resulting blank system comes with a package set curated by 9front cumminty that can be then configrued to your liking. After installation I created a new user through the file server connection and ran the /sys/lib/newuser -script to open up the window system.

I had to retry the installation three times before I found the right settings but I find this acceptable when much better funded systems often have similar performance.

Manuals

I like that they still extensively use the man pages and that I can find the most information about my system there. I've always liked the contents numbered by section in Unix manuals.

Many of the papers and texts you can find from Plan 9 website are worthwhile to read. I think I've read "introduction to operating system abstractions" twice through now because I had read it before.

In the previous post I referenced Scopenhauer's writing about authorship and style. The man proposes that if you find a good writer then let him tell his thing himself and don't quote him. A post I made before that is also great read and it tells what I recently learned about writing.

Suckless and 9front communities have some of the best documentation I've seen. The principle of frugality when writing software in itself is great because it means the limited bandwidth you have for documenting things is not being wasted.

File namespaces

Plan 9 has been designed to be discoverable. Every driver that exposes features through directories has their own hashed letter that can be listed by cat calling /dev/drivers.

Do you want to know what state do processes lug around? Everything's exposed in the /proc -directory. You can pick a process there and ls to see what's in a process.

At first sight per-process namespaces are confusing but it is actually quite simple as it is controlled by process forking. The windowing system forks a new namespace and process for each window you open. It's useful that this behavior depends on the context.

After you've tried a resolution with a terminal command, you may want that the system boots with that resolution. You can set this into the kernel by running 9fs 9fat. It mounts the boot directory and you can access the config in /n/9fat/plan9.ini. However if you look this from the another window, it's clear they don't see the mountpoint you created in the another window.

The 9fs itself would seem to be a script and by examining it I found out that the command ensures the dossrv is active and then tells it to mount the partition.

Hidden files

Plan 9 has no hidden files. Rob Pike told that the hiding of dotfiles was a bug in early unices. He's probably right about this. My system stores 151 dot files in my home directory giving it a resemblance of a garbage can.

(I've archived the text in case Xah Lee's link ever goes down, it's along this post in ./unix-dot.md)

Keyboard

Available keyboard maps are listed in /sys/lib/kbmap and they are sequences like below. The first column is a modifier, the second column is input scancode and the last column is the utf-8 rune they map to. To use the keybord map you copy this file into the /dev/kbmap.

1   39 'Ö
1   40 'Ä
1   51 ';
1   52 ':

Most of the system-provided keyboard mappings are sparse and none of them matched the svorak keyboard map I am using, so I created my own keyboard map. On the retrospect maybe the maps themselves could be catenated to create larger keyboard maps. It was incredibly easy to create your own kbmap and it'd have been even easier if I had known which key produces which scancode.

There was no stress on how check whether the map was working or not, I could just copy it into the device file to test it. After I was done I put the new map to lib/kbmap/fi and configured lib/profile to load it from there.

I got plenty of keymap customizations on my system and they extend all the way down to the keyboard's firmware. My ergodox has a custom keymapping behavior. When I press sym+E, it produces shift+3 and pressing sym+D produces altgr+3. I can vastly improve on this config and add my own unicode key codes to the keyboard if I ever switch to Plan 9.

On-screen keyboard

When remapping the keyboard it was helpful to have an on-screen keyboard. Instructions for this configuration are on the rio manual page. Add the configuration for rio to the lib/profile file and you get a small on-screen keyboard that you can click-on to produce character output.

Window system

The glenda user profile opens a terminal and a stats program that shows system load. When you create your own user profile it opens up gray blank. There is an initial command that you supply to rio in order to open up windows on the screen.

My immediate thought to this all was "Can I record all the windows I had open and start them up again?" Sure and the command to do so reads on the rio manual pgae! I think the name stands for "rendered I/O".

Eventually I found the wloc -command and it prints out a script that allows you to recreate the current window configuration. Btw. All the information provided by the wloc is available in rio's file server. Listing up the files in /srv shows you a rio-named file server there.

boot               dos     plumb.cheery.368
slashn             cons    factotum
rio.cheery.374     cs      hjfs.cmd   
riowctl.cheery.374 dns     mntexport  slashmnt

Soft restart

After doing configurations to the window system I wanted to reload the rio without otherwise booting the system. That "/srv" listing shows the rio is running on the process 374. I guessed that if I kill that process, it runs the rio down. Indeed "kill rio" shows how to send the kill note to a process.

After running the command it opens up into a full-screen terminal where you can write a command to run lib/profile and resume to the screen.

Boot process

The boot process of a Plan 9 kernel is not terribly complicated. It starts up with the requested config and then calls /$cputype/init. The init sets up the initial environment, name space and starts a shell.

Environment variables

When I echo $cputype it prints "amd64". There's an environment but no system calls for it. The environment is a directory. I can ls into /env and see the /env/cputype there. If I cat /env/home; echo, it does the same as echo $home and prints "/usr/cheery".

Development environment

Compilers, assemblers and linkers are labeled with a letter by architecture they compile to. The lc / reveals subdirectories such as "spim", "68000", "amd64" etc.. Compiler manual tells which systems they are:

0c spim    little-endian MIPS 3000 family
1c 68000   Motorola MC68000
2c 68020   Motorola MC68020
5c arm     little-endian ARM
6c amd64   AMD64 and compatibles (e.g., Intel EM64T)
7c arm64   ARM64 (ARMv8)
8c 386     Intel i386, i486, Pentium, etc.
kc sparc   Sun SPARC
qc power   Power PC
vc mips    big-endian MIPS 3000 family

Clear distinction between architectures means that a single file server can boot up any other machine irrespective of the archictecture each runs. The root can be compiled to support them all and then the bootloader finds the corresponding kernel from its respective directory and loads it.

Looking into $home/lib/profile reveals that it binds two directories:

bind -qa $home/bin/rc /bin
bind -qa $home/bin/$cputype /bin

The command ns | grep '/bin $' tells what is bound to the /bin. It shows information specific to my system:

bind /amd64/bin /bin
bind -a /rc/bin /bin
bind -a /usr/cheery/bin/rc /bin
bind -a /usr/cheery/bin/amd64 /bin

If I touch my home bin directory and insert a script into "bin/rc", the script appears in the "/bin". If I ls -l it shows me that it's my script and I have just inserted it into the system.

--rw-rw-r-- M 22 cheery sys 0 Oct 27 09:59 foo

As a consequence of some good discipline any Plan 9 system can compile to any other system.

Mkfiles

How does the build system know about different architectures and how does it know which compiler to pick? It happens through mkfiles. Each mkfile starts like this:

</$objtype/mkfile

The $objtype is set by /$cputype/init and as the consequence of the above line it chooses the tools being used. If I follow and check what's in the /amd64/mkfile I find:

</sys/src/mkfile.proto
CC=6c
LD=6l
O=6
AS=6a

Then a look into /sys/src/mkfile.proto reveals:

#
# common mkfile parameters shared by all architectures
#

OS=05678qv
CPUS=spim arm amd64 386 power mips
CFLAGS=-FTVw
LEX=lex
YACC=yacc
MK=/bin/mk

# recursive mk will have these set from the parent
# this is never what we want.  clear them

TARG=
OFILES=
HFILES=
YFILES=
LIB=

That it has been built like this means there's no mystery of where each variable and piece of information comes from.

Programming

I'm reading a guide that shows how to write some programs. It doesn't introduce mkfiles straight away. I type in "cat > take.c" and type in the following, then press CTRL+D.

#include <u.h>
#include <libc.h>
void main(int, char*[])
{
    print("take me to your leader!\n");
    exits(nil);
}

I know by reading the mkfiles that the compiler I'm supposed to use comes from the group "6".

The libc is a library of its own, but I don't need to import it. That is being done for me. This means I can write in

6c take.c
6l take.6

Then I got 6.out. I forgot the ./ but to my surprise it runs the correct program.

If you read through man mk you'll find out how to write the mkfile. In this case it's just:

</$objtype/mkfile
take: take.$O
    $LD $LDFLAGS -o $target $prereq

%.$O: %.c
    $CC $CFLAGS $stem.c

I like how it is using $target, $prereq and $stem instead of some magic symbols. The produced binary is a bit large, 31kB, but when I check into the /amd64/lib/lib.a, it's 523kB which means that the loader is not just dumping everything together.

The produced binary is in the old a.out format and it's documented in the manual. Likewise there are tools to read the format, for instance with nm you can list the symbols in each file.

How come none of this has been ported over to Linux?

You may conclude that everything here is unique to Plan 9. However people already know about it and everything has been already copied over.

  1. Linux can change the root namespace on process-basis and bind directories.
  2. You can containerize applications such that they entirely have their own namespaces.
  3. It's got architecture-specific files in their own directory, eg. /lib/x86_64-linux-gnu/ and a few other directories you find.
  4. They got not only one, but two keyboard maps you can configure yourself.
  5. It got several manual formats and all of them are still in use.
  6. There is a /proc directory that lets you touch processes.
  7. Device drivers can create their own files.
  8. Everything's been madly documented several times over.

It's been like this for 4 decades. Rob Pike kept the "cat -v considered harmful" presentation in 1983. The mechanism by which operating system rots is well-known by now.

  1. Somebody adds functionality and cuts corners to make it easy to use.
  2. The additional functionality bloats the documentation.
  3. When the tool becomes difficult to learn, people skip learning it.
  4. Alternatives are created and documented and they remain along the original tool.
  5. The layers of documentation and software pile up and the cycle repeats.

Lowest operating system layers aren't easy to learn or use and the replacements are written over them. The exact same functionality is wrapped and/or repeated across all software.

Here's a toilet analogy: Imagine that somebody starts putting all their trash through a porcelain. It works for a while but it eventually is clogged. They build a bigger toilet on the aside and when it clogs up they start going to the trashcan and build a rail system to move it outside. Everything eventually dumps like it should, but you got a miniature rail system within your house and somebody just hired best engineers in the world to renovate the house so that they can upgrade it to a maglev.

Parallels to programming languages

When the file namespace can change per-process it starts looking like a variable namespace in an interpreter. Files are values and programs are procedures. The shell is there for composing programs together with pipes.

The concept in itself quite powerful as can be demonstrated, but it does not make a nice user experience for novices. Insert a bad script into lib/profile or give a wrong command to a cron service and you'll get an error message much later when you're not ready to look into it.

There's a relatively easy way to fix this. You introduce types and typecheck shell commands in their predicted context before they're left in place. Untyped computer systems themselves can be delightfully simple. Plan 9's filesystem read/writes would seem to have vastly varying behavior depending on where they're being used, but it will easily trip down misconfigured software.

Types as user interfaces

Types can be treated like micro-protocols or as an user interface description. Shell scripts and programs could be type-checked with the namespaces that they manipulate. Protocols could be written just like they are in Plan 9 now, but they would have additional commands to disclose which protocol they produce.

For an example, the kbmap could tell that it's reading/writing lists of modifier/scancode/utf rune pairs. Then you'd have a program that interprets the type and constructs an user interface from it. Likewise programs and scripts would be annotated with types.

This is not a simple thing to achieve though. I didn't even thought about it before I had:

  1. Seen 9P protocol outside of context of Plan 9 and was intrigued by its simplicity again.
  2. I saw how Purescript developers are representing their types in Javascript, essentially giving Javascript the role of untyped lambda calculus.
  3. I understood how linear types can represent protocols as-it.
  4. I played with associating types to regular expressions encoding/decoding between values of those types and strings.
  5. I examined other attempts to type file formats such as typedefs and Dhall.

For now I'm studying papers around Plan 9 and learning the wisdoms it has to offer.

That's why

Plan 9 presents a whole bunch of well-working operating system abstractions made by masters of their craft. You can find flaws in it if you search for them but it's distilled, intact and working set of software designed to form a system without noise.

9front is maintainable enough that a small group can keep it going. Cat-v and suckless communities have remained repelling and inhospitable toward the mainstream.

Good foundations are important if you're planning to build something nice. I'm going to read deep into the manuals.

Similar posts