The system · L4

Files

A task that holds a Sys can read the kernel's state as files: its tasks, its channels, the time (sysk 028). Every tree of files in sysk will speak one small protocol, Plan 9's 9P cut down to ten operations. The kernel serves its own tree, as Plan 9's kernel serves /proc; the disk's server, a task, speaks the same bytes over a pair of streams (Files on the disk, sysk 029). In the shell, ls /sys/task and cat /sys/time.

The kernel's tree

Directories are green, files blue. Each node is a number, its qid path, which says which task or channel it is about. Every file is one s-expression of at most 512 bytes, rendered when it is read, in one atomic block: a snapshot, never torn. Pick a node.

The names come from the manifest. A task is named by its function, numbered (relay-2) when several share one, and a job slot is slot-N; a channel by its name. tools/wire.rkt generates task-name and channel-name, and who sends and receives on each channel. A pool channel's ends change with each job, so the file says jobs. A task's ns is its namespace, where it finds each tree, each mount with the task that serves it (sysk 053; below); empty for a task the manifest gives none.

A task's status also says how long it has run and what it waits for (sysk 060). ticks counts the timer ticks that came while it ran its own code; waits is a channel's name, clock, device, job (a slot with no program, or a job's slots still running), alt when it waits on more than one source, or none when it is running or ready. The task that reads the file is running, so its own status always says none. time adds idle, the ticks that came while every task waited, which no task is charged for (Who ran). Each field is one word, so a host can lay out every task's status as a table, and top watches them (The shell).

Beside status and ns, each task has fault (sysk 105): (fault (count 0)) until it breaks a rule, then its last fault, (fault (count 1) (cause illegal-instruction) (at render 312) (tick 4121)), the cause by name, the function and how far into it, and the tick. A task done by a fault has (state faulted) in its status. Node 80 + i, the sixteen numbers between the channels and the namespaces (A fault kept).

A request's path

The person types cat /sys/time. Step through it.

Messages

A request and its reply are each 576 bytes: a 64-byte header and a 512-byte payload, integers little-endian. Fixed sizes keep every message one channel message later, and every bound a constant the proofs use. Reserved bytes must be zero, so later versions can use them: sysk 033 took bytes 32 to 43 of a read for reads that wait.

A path is relative: names separated by /, each 1 to 32 bytes, at most 16, never . or ... The whole path is checked before any name is looked up, so whether it is a path never depends on what the tree holds. Without .. a walk can only go down, so a server lent a subtree can never be walked out of. The status says how a request went: 0 ok, 1 no such name, 2 not permitted, 3 bad fid, 4 bad path, 5 no space, 6 io, 7 busy, 9 bad request, 15 exists, 16 not empty; the payload says it again, as (error no-such-name). Every server checks the header the same way, with one function both share, fs-begin: reserved bytes, the op, the count and the offset.

Reads that wait

A client that wants to know when a file changes does not ask again and again. It asks once: a read whose mode is mode-wait, with since, the version of the file it last saw, and wait, how many milliseconds it will wait (sysk 033). The server answers as soon as the file's version is not since, which is at once if it already is not, or when the wait runs out. Either way the answer is a read's answer, the bytes at the offset and the file's version, so the client tells the two apart by the version alone: the same as since means nothing changed. There is no new status, and no event to miss: a client that was slow sees the latest state, not a queue of what happened (research-formats.md S4).

The file server holds a job's waiting read, one at a time, and goes on serving the shell; after each of the shell's requests, and when its clock's alarm comes, it looks at the read again. It holds no task: the job waits for its reply as it always does, and the server, which never waits on a job but through alt, has one more source, its clock. The shell's own reads, and the kernel's tree, answer at once: the person is waiting on the shell, and the kernel answers every request in one atomic step. A version moves when a file's change is stored, at the clunk that ends its write session (Files on the disk); a directory's does not move yet. follow is built on it: follow /log & prints the file, then each line added, until five seconds pass with no change, or as long as its argument says: follow /log --quiet 30000 & (sysk 036; The shell).

Fids

A client names nodes by fids, small numbers the server keeps for it, as file descriptors are. The kernel keeps 8 for each task, in a table of (* ntasks fid-max) words: each entry is its node times 2, plus 1 once open, and 0 for a free fid. sys-serve passes the current task, so a task can only ever name its own fids: the table row is chosen by who asks, not by anything in the request.

fids, task by task: (node × 2) + open
shell     fid 0: 2 × 4 + 1   ; /time, open     fid 1..7: 0
prober-a  fid 0: 2 × 1       ; the root        fid 1: 2 × 4 + 1   fid 2: 2 × 2 + 1
prober-b  fid 0: 2 × 1                          fid 1: 0   ; its own: names nothing

Whatever the request

files-serve takes the request's bytes as they come, from any task holding a Sys. Its contract is the whole guarantee: whatever the 576 bytes say, it ends with the kernel's invariant holding, the same task current, and a 576-byte reply. Reading a fid's entry, walking a path, rendering a file: each is bounded by constants, and each step is proven. The test system tests/files sends what no client should, and reads the statuses back:

Its peer, prober-b, holds a Sys too. While prober-a has fid 1 open, prober-b's fid 1 names nothing, before and after its own attach; prober-a's still reads afterwards. A task that makes itself a Sys is rejected (N003): only kernel/files.sysp makes one, given a Wire, for the tasks the manifest hands (sys), or whose profile mounts the kernel (sysk 062).

The namespace

Which tree a path is in, a task learns from the manifest (sysk 053), from its profile since sysk 062 (The manifest). The profile lists its mounts, ("PATH" SERVER [TREE]): each PATH, / or /NAME, is the tree of SERVER, a task that serves a tree, or the kernel's, kernel. tools/wire.rkt makes the two streams between the holder and each server, named after the server and the holder, fsd-ask-shell and fsd-reply-shell; a server takes the requests of all its holders (sysk 063, The file server). A mount may name a directory of a server's tree, ("/" fsd "/pub"): the server roots the holder there, and the holder's table does not carry it (sysk 064, A holder's root). A path is under the longest mount that starts it, a whole name at a time: /system is on the disk, /sys/time the kernel's. The rest of the path goes to that tree as its own path, from its root. The task is handed the table as text, where its (profile NAME) stands, each server named by the alt source of room on its requests; the kernel shows it, with the servers' names, in /sys/task/NAME/ns. A server that serves jobs too is followed by its service's pool channels, (/net 7 (jobs 5 6)): a job lent something in its tree binds them (sysk 058, Files for jobs). Pick a path.

  (task shell (heap 8192) (rx commands) (tx answers) (profile observer))
  (profile observer (jobs out) (catalogue) (clock)
    ("/" fsd) ("/dev" rng) ("/net" netd) ("/prg" kernel prg) ("/sys" kernel))

A path that does not start at / starts at the shell's dot, the directory cd chose (sysk 056), as a Plan 9 process's state is its namespace and its dot. The shell makes every path whole before it looks for its mount: from / or dot, a name at a time, . none, .. back one (and / at /), as Plan 9's cleanname. So no server ever sees . or .., and a walk still only goes down; dot is a shorter way to name, not a way to reach more. ls alone lists dot, pwd says it, and completion offers names in it.

The shell keeps the table in its Ns, beside the ends it asks each tree through, and every path it is given goes through tree-of and rest-of, which read it. ls / lists the disk's root, then each mount at the root: dev/ net/ prg/ sys/, from the table, and completing /n offers /net/ the same way. Only the disk's tree makes files. tools/wire.rkt refuses a profile with nothing at /, a path mounted twice, a path deeper than one name, a server that is no task serving a tree, or a tree the kernel does not serve (The manifest).

Why each holds what it holds: /sys/wire

The manifest gives each task its channels, devices and trees, and the pool's jobs what their programs need; tools/wire.rkt writes the facts of it, each with its manifest line (sysk 088), and sysq derives from them what each one holds and why, and checks the manifest's rules (sysq 004). The build runs sysq's tools/why.rkt on the facts and tools/wirefiles.rkt makes its files into tables the kernel serves, as /prg's are (sysk 092): /sys/wire/task/NAME for each task, /sys/wire/job/PROGRAM for each program in the pool, /sys/wire/rules and /sys/wire/holdings. A file is a record a line: a holding, with the name of the rule that derives it and the chain of manifest lines it rests on; a rule that holds, with who meets a must; or, in holdings, each named rule of sysq's holdings.sysq with its head, body and doc, so a holding's (rule NAME) resolves with cat /sys/wire/holdings (sysq 016).

/ % ls /sys/wire
task/     job/      rules
/ % cat /sys/wire/job/wc
(holding (job wc shell) (has tx out) (rule "holdings.sysq:28")
  (height 2) (chain (step (by "shell holds profile observer")
  (line 50)) (step (by "shell starts jobs, (jobs out)") (line 51))
  (step (by "the pool runs wc") (line 61)) ...))
...

The files never change while the system runs, so they are read where they lie, at any offset, as long as they are. They come from the same facts as the wiring the image was built with, so they say what the kernel gives each task and job, not what a task has done with it. Tests and other systems built without sysq's run get an empty /sys/wire. Directory 6, its task and job 7 and 8, and file w 512 + w, at most 256 files.

The programs: /prg

The kernel serves a second tree, prg (sysk 057): a file for each program of the catalogue, named by it. A server may serve several trees, as a Plan 9 server does, and an attach names the one it wants in its payload; the shell's profile says which, ("/prg" kernel prg). A server with one tree refuses a name. Each file is two sections, the first two of a program file as docs/directions.md designs it: (spec ...), what the job check judges the program by, its channels and its arguments, generated from the same contract the check is; and (doc ...), its doc comment from the source, a string for each sentence. A person reads it on the machine, so it names no feature: tools/wire.rkt refuses a doc comment, a program's or a part's command's, that says sysk 057 or the like (sysk 121). The numbers are the suite's bookkeeping, for its specs and commits.

/ % ls /prg
wc        follow    ping      tell      sysc/     sysn/
/ % cat /prg/wc
(spec wc (channels (ask tx stream 576 service)
  (answer rx stream 576 service) (out tx 1 64)) (arguments
  (lines flag)))
(doc "A catalogue program: the bytes and lines of the
  file it was lent, on one line of its output, ...")

A channel is (NAME tx|rx COUNT|stream BYTES [service]), service marking a service's client end, bound to the service of whatever the job is lent (sysk 058); an argument (NAME flag) or (NAME number LO HI DEFAULT); and (clock) last, for a program that keeps time, as ping does. The text never changes while the system runs, so the kernel reads it where it lies, at any offset: a program's file may be longer than one read, as follow's is, unlike a file the kernel renders. Today the programs are compiled into the image and /prg describes them; when programs are kept on the disk, in a store of entries named by their hash, the same mount names the file server's tree instead, ("/prg" fsd prg), and the files gain a third section, the code.

A part's functions: /prg/PART

A library part's pure functions are commands too (sysc 014): sysc/sha-256 /notes/letter is the file's SHA-256, and sysn/sum /notes/header its Internet checksum (sysn 014); sysn/frame FILE decodes a frame (sysn 015), and a conversation's last in /net once jobs run side by side. A part offers them by name in PART/commands.sysk, (commands sha-256), and a manifest whose catalogue names the part, (programs ... sysc), gets the part's runner: one catalogue program, generated by tools/wire.rkt (tools/commands.rkt), whose first argument is the command's index, run as a job, so the job check judges it as it judges every program. In /prg the runner is a directory, sysc/, with a file for each command, of type function (8): its spec, made from the function's types, and the function's doc comment. Commands are numbered across the catalogue, node 384 + f; the runner's directory is its program's node, 128 + p.

/ % ls /prg/sysc
sha-256
/ % cat /prg/sysc/sha-256
(spec sha-256 (kind function) (arguments (file read))
  (value (bytes 32)) (from sysc/sha) (needs (read file)))
(doc "The SHA-256 of msg, FIPS 180-4, into out.")
/ % sysc/sha-256 /shopping
(sha-256
  "eaedc9b7135a26651e1b0cc6a8a5f5d2e82ecd0e9b4745b37ff3b58af058e09d")

The shapes come from the types, a first set: a state (mut T), which the runner makes from its layer's scratch words (sha-new of sha-scratch words, given back by sha-buf); the bytes (ref (buf u8)), from the file the shell lends read-only, so the need is (read file), derived; and a result (mut (buf u8)) of the length the function requires, said in hex as sha256sum writes it. A file comes a payload at a time, through the three found by the command's name, sha-256-begin, sha-256-add, sha-256-end. A function that is private, has effects, or takes anything else is refused at build, by name. Text and number arguments wait: a job's arguments are words of the job table. A program may not take a part's name (the register, kit/parts.js) unless it is that part's runner, so a program and a part's directory never meet in /prg; sysk's own programs stay at the top.

A result may be a value instead of bytes (sysk 090, for sysn's frame and dns). When NAME-end returns i64, it has written a value's text into its result from 0, with sysl/text, and returns where the text ends; the runner says it as a value, in pieces when longer than a line, so a host shows it by its head's view. And a parser that needs a whole message, since a header cannot be decoded a payload at a time, is one function, NAME ((msg (ref (buf u8))) (out (mut (buf u8)))) i64, requiring (<= (len msg) N): the runner reads the file whole into a block of N bytes and says (NAME (error too-long N)) for a longer one. The types say which shape a command is; in /prg its spec says (value (sexp M)), and (file read (whole N)). The runner trusts no text: unless sx-one reads it as one value, parentheses inside strings counting for nothing, the command says (NAME (error not-a-value)).

In the shell

In the shell's namespace (above), /sys and below is the kernel's tree, /dev and below the entropy driver's (sysk 046, Devices as files), /net and below netd's (sysk 052, The network), and everything else the disk's, whose root is /. (ls PATH) lists a directory, a record for each name, (entry (name "shell") (type dir) ...), which a host shows as names, a directory's with a / after it (sysk 066, Views), and (ls) lists dot (sysk 056); (cat PATH) prints a file, wrapped to the screen's 63 bytes between fields, so (ended no) stays whole, and inside a string between words (sysk 057). Through sugar, without the parentheses, and a host (abridged):

/ % ls /sys/task
uart/      terminal/  sugar/     shell/
slot-0/    slot-1/    slot-2/    slot-3/
/ % cat /sys/task/shell/status
(task (name shell) (id 3) (state live) (alarm none) (ticks 20)
  (waits none))
/ % cat /sys/task/sugar/status
(task (name sugar) (id 2) (state live) (alarm none) (ticks 1)
  (waits lines))
/ % cat /sys/chan/screen
(channel (name screen) (bytes 64) (depth 2) (queued 0)
  (ended no) (from terminal) (to uart))
/ % cat /sys/time
(time (ns 950235000) (ticks 948) (idle 929))
/ % cat /sys/task/../time
not a path: /sys/task/../time

The client side is lib/fs.sysp. A request is bytes, whatever carries it, so the library builds them: one function per operation (fs-attach-req, fs-walk-req, fs-open-req, fs-read-req, fs-write-req, fs-create-req, and fs-fid-req for remove, stat and clunk). fs-ask-sys asks the kernel's tree through the gate; a server over streams is asked by the client that holds the streams, as the shell asks the disk's. tests/files/prober.sysp builds its requests the same way. A function both modes reach, and that calls a gate, is compiled twice by sysp (sysp 024): once for tasks, once for the kernel. A server's side has one function too (sysk 059): fs-stamped, the rights of a lend that a lease's stamp allows, of the protocol's three, for each loan by its number (sysk 089), by which fsd caps what it lent a job (Within its need).

The disk's file server writes, creates and removes, with copy-on-write blocks: Files on the disk. A job can be lent part of the disk's tree, read through the same operations: Files for jobs (sysk 031). Not yet: files that wait for events (sysk 032).