From fc4c563ca14ddec3de3aeef5a861f9707eaae389 Mon Sep 17 00:00:00 2001 From: salmonik Date: Fri, 21 Aug 2026 17:18:09 +0200 Subject: docs: ADR 0006 (executable binaries) + implementation plan + glossary terms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ADR 0006: PE32+ loader in rsnix, versioned exports-table ABI constructed per invocation, task-per-binary, single dd-able system image with /bin/cli and /bin/gui, shell-owned grammar with injected line streams, per-terminal input discipline as prerequisite phase, frozen built-in core, 0.6.0 at first exec. Plan: five phases (image → input discipline → loader/exec → streams + migration → GUI binaries), each with a QEMU verification gate. Glossary: Binary, Loader, Exports table, System image; kind = launch surface. --- CONTEXT.md | 35 ++++++ docs/adr/0006-executable-binaries-and-bin-tree.md | 115 ++++++++++++++++++ docs/plan-executable-binaries.md | 140 ++++++++++++++++++++++ 3 files changed, 290 insertions(+) create mode 100644 docs/adr/0006-executable-binaries-and-bin-tree.md create mode 100644 docs/plan-executable-binaries.md diff --git a/CONTEXT.md b/CONTEXT.md index c27f059..01f5c69 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -116,6 +116,41 @@ message queue. A GUI program is an app; there is no separate "process" concept — the app is its task plus its window. The home terminal is app 0 (window 0). +## Binary + +A PE32+ file holding executable code that the kernel can load and run +without being recompiled into the kernel image. A binary runs against +the kernel's exported interface, never by linking kernel state directly. +Binaries come in two kinds, separated by **launch surface** rather than +implementation: **CLI binaries** live in /bin/cli and run inside a +terminal session (invoked by a shell, inheriting its input/output); +**GUI binaries** live in /bin/gui and are launchable with no terminal at +all — they own their windows — which makes enumerating launchable apps +as simple as listing /bin/gui. Every binary runs as its own task — a CLI binary's task is +waited on by the calling terminal, a GUI binary's task owns its window. + +## System image + +The single dd-able FAT filesystem image the build produces: bootloader, +kernel, /bin, /etc and seed content on one volume. Imaging it onto a disk +yields a complete installed system. Binaries live on the system image +under /bin/cli and /bin/gui; the shell finds them by scanning mounted +volumes — no volume is privileged. + +## Loader + +The kernel facility that takes a binary from disk, places it in memory, +and starts it. Loading changes no address-space setup — binaries run in +the one shared identity-mapped space. + +## Exports table + +The interface between a binary and the kernel: a versioned table of +functions the kernel hands to a binary at entry (output, filesystem, +memory, …). The caller constructs it per invocation, so the same binary's +output can be aimed at a terminal or into the shell's filter pipeline. +All kernel state stays behind the table — a binary never owns a copy of it. + ## Terminal A program that presents a command line in a window: a text terminal diff --git a/docs/adr/0006-executable-binaries-and-bin-tree.md b/docs/adr/0006-executable-binaries-and-bin-tree.md new file mode 100644 index 0000000..17109c4 --- /dev/null +++ b/docs/adr/0006-executable-binaries-and-bin-tree.md @@ -0,0 +1,115 @@ +# ADR 0006 — Executable binaries: PE loader, exports-table ABI, the /bin tree, and the system image + +## Status + +Accepted. + +## Context + +Nothing can be executed inside raisernics that was not compiled into the +kernel image: every command is a built-in function inside the `shell` +crate, dispatched by a match statement. The kernel has no program loader, +no exec concept, and (per design) no address-space separation — all tasks +share one identity-mapped space. We want commands to become real, +loadable binaries so the system can gain software without recompiling the +kernel, so GUI apps and CLI tools are structurally the same kind of thing, +and so a launcher can enumerate launchable apps by listing a directory. + +## Decision + +- **Format: PE32+.** Binaries are built with the existing + `x86_64-unknown-uefi` toolchain into standalone PE images. The loader + allocates pages, copies segments, applies `.reloc` base relocations at + load time, and jumps to the entry point. No page-table changes — + loading is alloc-copy-relocate-jump in the one shared address space. +- **Loader lives in `rsnix`** as a kernel facility (alongside `fat`, + `ahci`). +- **ABI: versioned exports table passed at entry** — not syscalls, not + well-known addresses. The kernel hands the binary a table of function + pointers (filesystem, memory, window/event, streams) with a version + field; both sides share the type definitions in-workspace, so the + contract is compile-time checked and old on-disk binaries fail loudly on + a version mismatch. +- **The caller constructs the table per invocation.** Output injection: + `put_line` points either at the terminal or at the shell's filter + pipeline head. Input injection: `get_line` reads from wherever the + shell resolved the grammar's input side (keyboard, file, TIL-truncated + source). Redirection is dependency injection, not global state. +- **Task per binary.** Every binary runs as its own task. CLI binaries + are waited on by the calling terminal's shell (completion via message); + GUI binaries are fire-and-forget. A faulting binary kills only its own + task, honouring ADR 0003's graceful-death guarantee. +- **System image.** The build produces one dd-able FAT32 image containing + bootloader, kernel, `\bin\cli\*`, `\bin\gui\*`, `\etc\*`, and seed + content. Imaging it onto a disk yields a complete installed system. + The volume mounts through the ordinary `/disk//` mechanism with + full read-write; overwriting a command on the image persists across + reboots. +- **Resolution rule.** For name N: run `\bin\\N` from the first + mounted volume that has it; else the built-in N; else "unknown + command". No volume is privileged; no boot-volume identity exists. +- **Kind = launch surface.** `/bin/cli` holds programs that run inside a + terminal session; `/bin/gui` holds programs launchable with no terminal + (they own their windows). A future launcher enumerates `/bin/gui`. +- **The shell owns 100% of the grammar (ADR 0002).** Binaries never see + USE/TIL/filters/sinks — they receive clean argv and injected line + streams. Any binary gets stdin/stdout for free; which commands accept + input keywords stays a policy choice. +- **^C and stop flow through the stream functions.** `put_line -> false` + means stop (head kill or pager ^C); `get_line` returns Eof (^D) or + Abort (^C) distinctly. The `-p` pager blocks inside the stream call in + the command task's context. No signals, no abort flags. +- **Per-terminal input discipline is a prerequisite phase.** Keyboard + routing becomes total: IRQ ring → WM pump → focused window's queue, + always. Each terminal's shell owns canonical mode from its own queue; + the home terminal becomes an ordinary focusable window and the + `FOCUSED == CONSOLE` sentinel retires. Binaries consume assembled + lines through a bounded per-invocation channel fed by the owning shell. +- **Frozen built-in core**: `help`, `clear`, `echo`, `poweroff`/`exit`, + `halt`, `rescan` stay built-ins permanently — needed before disks exist + or existing to fix them. Everything else migrates (`cat` last; it + exercises every grammar corner). +- **Version 0.6.0 lands when exec first works** (loader + ABI + one + binary end-to-end). + +## Alternatives considered + +- **Flat raw binary at a fixed address** — trivial loader, but fixed + addresses forbid concurrent resident binaries (fatal under preemptive + tasks). +- **ELF** — familiar but fights the toolchain for zero benefit here. +- **Syscall gate (`int` + numbers)** — more OS-flavored, but stdout would + become "the current command's sink" global state — the exact bug class + the ACTIVE-terminal incident came from — and buys nothing without + address spaces. +- **Static linking of rsnix into binaries** — forbidden for stateful + modules: each binary would get private copies of mount tables and node + pools. +- **Synchronous execution in the caller's task** — simplest, but a + faulting binary halts task 0 and the machine. +- **ramfs-seeded /bin** — rejected in favour of the system image: rw + through FAT write-back persists across reboots, and the image doubles + as the distribution artifact. +- **Boot-volume identity / root-path unification** (`\etc` visible at + `/etc`) — deferred; needs VFS bind/alias machinery. +- **A third "TUI" binary kind for rsdosedit** — rejected; `edit` is + classified CLI (it lives in a terminal session) and migrates only after + the CLI ABI grows raw-mode surface access. Until then it stays a + built-in. + +## Consequences + +- Positive: raisernics gains an exec story without abandoning the + single-address-space philosophy; commands become hot-swappable at + runtime (`cat x WRITING /bin/cli/ls`) and persistent across reboots; + CLI and GUI programs share one spawn path, loader, and ABI. +- Positive: the grammar becomes enforceable by construction — binaries + cannot break ADR 0002 because they never see it. +- Negative: build orchestration grows a stage (build `/bin` crates → + stage artifacts → build image); cargo does not chain these natively. +- Negative: streaming output crosses a task boundary; `-t` flush and ^C + latency gain one scheduling round trip. +- Negative: a compute-bound binary that never calls the streams cannot be + ^C'd (policy, same blind spot `stress` has today). +- Deferred: raw-mode CLI ABI extension (for `edit`), FAT shadowing of + `/bin`, boot-volume identity, launcher. diff --git a/docs/plan-executable-binaries.md b/docs/plan-executable-binaries.md new file mode 100644 index 0000000..cd6f1d0 --- /dev/null +++ b/docs/plan-executable-binaries.md @@ -0,0 +1,140 @@ +# Implementation plan — executable binaries and the /bin tree + +Companion to ADR 0006. Phases are strictly ordered where dependencies +say so; each phase ends with a QEMU verification gate. Version bumps per +the glossary rules. + +--- + +## Phase 0 — System image (build only, no kernel changes) + +The build produces one dd-able FAT32 image instead of the current +two-source setup (vvfat `esp/` directory + separate `/tmp/disk.img`). + +- [ ] Extend `scripts/mkdisk.sh` (or add `scripts/mksystem.sh`) to create + `system.img`: `mkfs.fat -C`, then `mcopy` in + `\bzImage`, `\EFI\` (as rsbl requires), `\etc\*` (motd moves out of + the `fs.rs` code literal? — optional here), seed content. +- [ ] Stage `\bin\cli\` and `\bin\gui\` directories (empty for now). +- [ ] Makefile: build kernel → stage image; `run`/`run-gui`/`debug` + attach **only** `system.img` on ide.0 (delete the + `-drive file=fat:rw:$(ESP_DIR)` vvfat lines). +- [ ] Verify OVMF boots the single image end-to-end. + +**Gate:** image boots; `ls /disk//` shows `\bin`, `\etc`, seed files; +writes under `/disk//` survive poweroff (existing FAT write-back). + +--- + +## Phase 1 — Per-terminal input discipline (prerequisite) + +Keyboard routing becomes total; canonical mode becomes per-terminal shell +property. This is the original "per-instance shell" step, completed. + +- [ ] WM pump routes keys unconditionally: IRQ ring → pump → focused + window's queue. No task ever polls the global ring directly. +- [ ] Home terminal (window 0) becomes an ordinary focusable window: + clicking it focuses it; retire the `FOCUSED == CONSOLE` sentinel + and the `focused() == 0` special cases from commit `09a44f1`. + `focus_console()` becomes `focus(0)`. +- [ ] Each terminal's shell assembles lines canonically from its own + queue (`tty::read_line_from_queue` path); delete the global-ring + path through `tty::poll_key`. +- [ ] Raw-mode consumers that remain built-ins (`edit`, pager) read keys + via the same focused-window queue, not the ring. +- [ ] Two terminals + home terminal: input always lands in the focused + one; unfocused terminals keep rendering output from running + commands. + +**Gate:** three terminals open; typing switches cleanly with focus; no +keystroke ever appears in two terminals; `make test-disk` regression +passes (it drives the shell via injected keys). + +--- + +## Phase 2 — Loader, Exports table, first exec (**bump 0.6.0**) + +- [ ] New crate `rsapi` (or `rsnix::abi`): the versioned `Exports` struct, + entry-point signature, ABI event/argv types shared by kernel and + binaries. Header carries `version: u32`. +- [ ] `rsnix::load`: PE32+ parser (headers, sections, `.reloc` + application), alloc-copy-relocate, entry lookup. ~300–500 lines. +- [ ] Spawn path: load → new task (own stack, guard page) → jump to entry + with `(&Exports, argc, argv)`; completion message to spawner; + pages freed on exit. +- [ ] Resolution: name → scan mounted volumes' `\bin\\` → + else built-in → else unknown. +- [ ] argv block: shell copies parsed positional args into the binary's + memory; `argc/argv` C-style. +- [ ] First binary: `hello.efi` in `/bin/cli/` — calls one export + (`put_line("hello")`) and returns 0. +- [ ] Crash containment check: a `boom.efi` that faults kills only its + task; shell survives; `jobs` shows the corpse. + +**Gate:** `hello` runs from the image, prints via exports, exits; +`boom` crashes gracefully; built-in fallback still works when the file +is absent; **version bump to 0.6.0**. + +--- + +## Phase 3 — Streams, pipeline integration, line-tool migration + +- [ ] Per-invocation line channel: bounded queue between owning shell and + command task; shell pumps its key queue → canonical assembly → + channel while a command runs (shell main loop gains a "pumping" mode). +- [ ] Exports streams: `get_line() -> {Line, Eof, Abort}`, + `put_line(line) -> bool` (false = stop). Shell-side implementations + feed the filter pipeline / sink / tty exactly as built-ins do today. +- [ ] Grammar translation shell-side: USE/TIL resolve into the injected + source; ` TIL ` sugar strips paths from argv; + filters/sinks wrap `put_line`. Binaries never see keywords. +- [ ] ^C semantics: shell sees ^C while pumping → marks channel aborted → + command's next stream call returns Abort/stop → completion → shell + flushes `-t` windows. `-p` pager blocks inside `put_line`. +- [ ] Migrate simple tools to `/bin/cli/`: `echo` stays frozen; migrate + `ls`, `mem`, `pci`, `date`, `info`, `mkdir`, `touch`, `rm`, + `hexdump`, `klog`, … Each keeps its grammar behaviour verbatim. +- [ ] Migrate `cat` last (input consumer of every corner: heredoc, USE, + TIL sugar, multi-path). + +**Gate:** full `scripts/test-disk.sh` grammar suite passes unchanged with +external binaries; `-h/-t/-g/-p` chains work across the task boundary; +^C flushes a tail window on an infinite stream. + +--- + +## Phase 4 — GUI binaries + +- [ ] Window/event exports: `create_window(title,w,h) -> id`, + `window_buffer(id) -> (ptr,w,h)`, `flush(id)`, `destroy(id)`, + `recv_event(&mut Ev) -> bool` (Ev mirrors Key/AppMouse/Focus/Close). +- [ ] Fire-and-forget spawn: shell returns immediately after spawning; + exit = destroy window + task done. +- [ ] Migrate paint → `/bin/gui/paint`; `draw` resolves through the + normal rule (no special case). +- [ ] F1-spawned terminals keep working alongside GUI binaries. + +**Gate:** `draw` launches the migrated binary; pen draws; F4/close button +closes it; focus cycle and title-bar drag unaffected. + +--- + +## Frozen built-in core (permanent) + +`help`, `clear`, `echo`, `poweroff`/`exit`, `halt`, `rescan`. + +Rationale: needed before disks exist, or exist to fix them. `edit` also +stays built-in until the deferred raw-mode CLI ABI exists. + +## Explicitly deferred (non-goals for this rework) + +- Raw-mode surface access in the CLI ABI (needed before `edit` migrates). +- FAT shadowing of `/bin` beyond the system image. +- Boot-volume identity / root-path unification (`\etc` at `/etc`). +- Launcher (will simply enumerate `/bin/gui` once Phase 4 lands). +- Fixing compute-bound-binary ^C blindness (accepted policy). + +## Version story + +- Phases 0–1 land as 0.5.x increments if separate. +- **0.6.0 at Phase 2** — exec exists (glossary class `b`: big feature). -- cgit v1.2.3