diff options
| author | salmonik <salmonik@tuta.io> | 2026-08-21 17:18:09 +0200 |
|---|---|---|
| committer | salmonik <salmonik@tuta.io> | 2026-08-21 17:18:09 +0200 |
| commit | fc4c563ca14ddec3de3aeef5a861f9707eaae389 (patch) | |
| tree | e88483e79ca3ca6d980cdbf984cf9278ccef1d27 | |
| parent | wm: fix usize underflow in move_window at screen edges (diff) | |
| download | rs-dos-fc4c563ca14ddec3de3aeef5a861f9707eaae389.tar.gz rs-dos-fc4c563ca14ddec3de3aeef5a861f9707eaae389.zip | |
docs: ADR 0006 (executable binaries) + implementation plan + glossary terms
ADR 0006: PE32+ loader in rsnix, versioned exports-table ABI constructed
per invocation, task-per-binary, single dd-able system image with
/bin/cli and /bin/gui, shell-owned grammar with injected line streams,
per-terminal input discipline as prerequisite phase, frozen built-in
core, 0.6.0 at first exec.
Plan: five phases (image → input discipline → loader/exec → streams +
migration → GUI binaries), each with a QEMU verification gate.
Glossary: Binary, Loader, Exports table, System image; kind = launch
surface.
| -rw-r--r-- | CONTEXT.md | 35 | ||||
| -rw-r--r-- | docs/adr/0006-executable-binaries-and-bin-tree.md | 115 | ||||
| -rw-r--r-- | docs/plan-executable-binaries.md | 140 |
3 files changed, 290 insertions, 0 deletions
| @@ -116,6 +116,41 @@ message queue. A GUI program is an app; there is no separate "process" | |||
| 116 | concept — the app is its task plus its window. The home terminal is app 0 | 116 | concept — the app is its task plus its window. The home terminal is app 0 |
| 117 | (window 0). | 117 | (window 0). |
| 118 | 118 | ||
| 119 | ## Binary | ||
| 120 | |||
| 121 | A PE32+ file holding executable code that the kernel can load and run | ||
| 122 | without being recompiled into the kernel image. A binary runs against | ||
| 123 | the kernel's exported interface, never by linking kernel state directly. | ||
| 124 | Binaries come in two kinds, separated by **launch surface** rather than | ||
| 125 | implementation: **CLI binaries** live in /bin/cli and run inside a | ||
| 126 | terminal session (invoked by a shell, inheriting its input/output); | ||
| 127 | **GUI binaries** live in /bin/gui and are launchable with no terminal at | ||
| 128 | all — they own their windows — which makes enumerating launchable apps | ||
| 129 | as simple as listing /bin/gui. Every binary runs as its own task — a CLI binary's task is | ||
| 130 | waited on by the calling terminal, a GUI binary's task owns its window. | ||
| 131 | |||
| 132 | ## System image | ||
| 133 | |||
| 134 | The single dd-able FAT filesystem image the build produces: bootloader, | ||
| 135 | kernel, /bin, /etc and seed content on one volume. Imaging it onto a disk | ||
| 136 | yields a complete installed system. Binaries live on the system image | ||
| 137 | under /bin/cli and /bin/gui; the shell finds them by scanning mounted | ||
| 138 | volumes — no volume is privileged. | ||
| 139 | |||
| 140 | ## Loader | ||
| 141 | |||
| 142 | The kernel facility that takes a binary from disk, places it in memory, | ||
| 143 | and starts it. Loading changes no address-space setup — binaries run in | ||
| 144 | the one shared identity-mapped space. | ||
| 145 | |||
| 146 | ## Exports table | ||
| 147 | |||
| 148 | The interface between a binary and the kernel: a versioned table of | ||
| 149 | functions the kernel hands to a binary at entry (output, filesystem, | ||
| 150 | memory, …). The caller constructs it per invocation, so the same binary's | ||
| 151 | output can be aimed at a terminal or into the shell's filter pipeline. | ||
| 152 | All kernel state stays behind the table — a binary never owns a copy of it. | ||
| 153 | |||
| 119 | ## Terminal | 154 | ## Terminal |
| 120 | 155 | ||
| 121 | A program that presents a command line in a window: a text terminal | 156 | A program that presents a command line in a window: a text terminal |
diff --git a/docs/adr/0006-executable-binaries-and-bin-tree.md b/docs/adr/0006-executable-binaries-and-bin-tree.md new file mode 100644 index 0000000..17109c4 --- /dev/null +++ b/docs/adr/0006-executable-binaries-and-bin-tree.md | |||
| @@ -0,0 +1,115 @@ | |||
| 1 | # ADR 0006 — Executable binaries: PE loader, exports-table ABI, the /bin tree, and the system image | ||
| 2 | |||
| 3 | ## Status | ||
| 4 | |||
| 5 | Accepted. | ||
| 6 | |||
| 7 | ## Context | ||
| 8 | |||
| 9 | Nothing can be executed inside raisernics that was not compiled into the | ||
| 10 | kernel image: every command is a built-in function inside the `shell` | ||
| 11 | crate, dispatched by a match statement. The kernel has no program loader, | ||
| 12 | no exec concept, and (per design) no address-space separation — all tasks | ||
| 13 | share one identity-mapped space. We want commands to become real, | ||
| 14 | loadable binaries so the system can gain software without recompiling the | ||
| 15 | kernel, so GUI apps and CLI tools are structurally the same kind of thing, | ||
| 16 | and so a launcher can enumerate launchable apps by listing a directory. | ||
| 17 | |||
| 18 | ## Decision | ||
| 19 | |||
| 20 | - **Format: PE32+.** Binaries are built with the existing | ||
| 21 | `x86_64-unknown-uefi` toolchain into standalone PE images. The loader | ||
| 22 | allocates pages, copies segments, applies `.reloc` base relocations at | ||
| 23 | load time, and jumps to the entry point. No page-table changes — | ||
| 24 | loading is alloc-copy-relocate-jump in the one shared address space. | ||
| 25 | - **Loader lives in `rsnix`** as a kernel facility (alongside `fat`, | ||
| 26 | `ahci`). | ||
| 27 | - **ABI: versioned exports table passed at entry** — not syscalls, not | ||
| 28 | well-known addresses. The kernel hands the binary a table of function | ||
| 29 | pointers (filesystem, memory, window/event, streams) with a version | ||
| 30 | field; both sides share the type definitions in-workspace, so the | ||
| 31 | contract is compile-time checked and old on-disk binaries fail loudly on | ||
| 32 | a version mismatch. | ||
| 33 | - **The caller constructs the table per invocation.** Output injection: | ||
| 34 | `put_line` points either at the terminal or at the shell's filter | ||
| 35 | pipeline head. Input injection: `get_line` reads from wherever the | ||
| 36 | shell resolved the grammar's input side (keyboard, file, TIL-truncated | ||
| 37 | source). Redirection is dependency injection, not global state. | ||
| 38 | - **Task per binary.** Every binary runs as its own task. CLI binaries | ||
| 39 | are waited on by the calling terminal's shell (completion via message); | ||
| 40 | GUI binaries are fire-and-forget. A faulting binary kills only its own | ||
| 41 | task, honouring ADR 0003's graceful-death guarantee. | ||
| 42 | - **System image.** The build produces one dd-able FAT32 image containing | ||
| 43 | bootloader, kernel, `\bin\cli\*`, `\bin\gui\*`, `\etc\*`, and seed | ||
| 44 | content. Imaging it onto a disk yields a complete installed system. | ||
| 45 | The volume mounts through the ordinary `/disk/<UID>/` mechanism with | ||
| 46 | full read-write; overwriting a command on the image persists across | ||
| 47 | reboots. | ||
| 48 | - **Resolution rule.** For name N: run `\bin\<kind>\N` from the first | ||
| 49 | mounted volume that has it; else the built-in N; else "unknown | ||
| 50 | command". No volume is privileged; no boot-volume identity exists. | ||
| 51 | - **Kind = launch surface.** `/bin/cli` holds programs that run inside a | ||
| 52 | terminal session; `/bin/gui` holds programs launchable with no terminal | ||
| 53 | (they own their windows). A future launcher enumerates `/bin/gui`. | ||
| 54 | - **The shell owns 100% of the grammar (ADR 0002).** Binaries never see | ||
| 55 | USE/TIL/filters/sinks — they receive clean argv and injected line | ||
| 56 | streams. Any binary gets stdin/stdout for free; which commands accept | ||
| 57 | input keywords stays a policy choice. | ||
| 58 | - **^C and stop flow through the stream functions.** `put_line -> false` | ||
| 59 | means stop (head kill or pager ^C); `get_line` returns Eof (^D) or | ||
| 60 | Abort (^C) distinctly. The `-p` pager blocks inside the stream call in | ||
| 61 | the command task's context. No signals, no abort flags. | ||
| 62 | - **Per-terminal input discipline is a prerequisite phase.** Keyboard | ||
| 63 | routing becomes total: IRQ ring → WM pump → focused window's queue, | ||
| 64 | always. Each terminal's shell owns canonical mode from its own queue; | ||
| 65 | the home terminal becomes an ordinary focusable window and the | ||
| 66 | `FOCUSED == CONSOLE` sentinel retires. Binaries consume assembled | ||
| 67 | lines through a bounded per-invocation channel fed by the owning shell. | ||
| 68 | - **Frozen built-in core**: `help`, `clear`, `echo`, `poweroff`/`exit`, | ||
| 69 | `halt`, `rescan` stay built-ins permanently — needed before disks exist | ||
| 70 | or existing to fix them. Everything else migrates (`cat` last; it | ||
| 71 | exercises every grammar corner). | ||
| 72 | - **Version 0.6.0 lands when exec first works** (loader + ABI + one | ||
| 73 | binary end-to-end). | ||
| 74 | |||
| 75 | ## Alternatives considered | ||
| 76 | |||
| 77 | - **Flat raw binary at a fixed address** — trivial loader, but fixed | ||
| 78 | addresses forbid concurrent resident binaries (fatal under preemptive | ||
| 79 | tasks). | ||
| 80 | - **ELF** — familiar but fights the toolchain for zero benefit here. | ||
| 81 | - **Syscall gate (`int` + numbers)** — more OS-flavored, but stdout would | ||
| 82 | become "the current command's sink" global state — the exact bug class | ||
| 83 | the ACTIVE-terminal incident came from — and buys nothing without | ||
| 84 | address spaces. | ||
| 85 | - **Static linking of rsnix into binaries** — forbidden for stateful | ||
| 86 | modules: each binary would get private copies of mount tables and node | ||
| 87 | pools. | ||
| 88 | - **Synchronous execution in the caller's task** — simplest, but a | ||
| 89 | faulting binary halts task 0 and the machine. | ||
| 90 | - **ramfs-seeded /bin** — rejected in favour of the system image: rw | ||
| 91 | through FAT write-back persists across reboots, and the image doubles | ||
| 92 | as the distribution artifact. | ||
| 93 | - **Boot-volume identity / root-path unification** (`\etc` visible at | ||
| 94 | `/etc`) — deferred; needs VFS bind/alias machinery. | ||
| 95 | - **A third "TUI" binary kind for rsdosedit** — rejected; `edit` is | ||
| 96 | classified CLI (it lives in a terminal session) and migrates only after | ||
| 97 | the CLI ABI grows raw-mode surface access. Until then it stays a | ||
| 98 | built-in. | ||
| 99 | |||
| 100 | ## Consequences | ||
| 101 | |||
| 102 | - Positive: raisernics gains an exec story without abandoning the | ||
| 103 | single-address-space philosophy; commands become hot-swappable at | ||
| 104 | runtime (`cat x WRITING /bin/cli/ls`) and persistent across reboots; | ||
| 105 | CLI and GUI programs share one spawn path, loader, and ABI. | ||
| 106 | - Positive: the grammar becomes enforceable by construction — binaries | ||
| 107 | cannot break ADR 0002 because they never see it. | ||
| 108 | - Negative: build orchestration grows a stage (build `/bin` crates → | ||
| 109 | stage artifacts → build image); cargo does not chain these natively. | ||
| 110 | - Negative: streaming output crosses a task boundary; `-t` flush and ^C | ||
| 111 | latency gain one scheduling round trip. | ||
| 112 | - Negative: a compute-bound binary that never calls the streams cannot be | ||
| 113 | ^C'd (policy, same blind spot `stress` has today). | ||
| 114 | - Deferred: raw-mode CLI ABI extension (for `edit`), FAT shadowing of | ||
| 115 | `/bin`, boot-volume identity, launcher. | ||
diff --git a/docs/plan-executable-binaries.md b/docs/plan-executable-binaries.md new file mode 100644 index 0000000..cd6f1d0 --- /dev/null +++ b/docs/plan-executable-binaries.md | |||
| @@ -0,0 +1,140 @@ | |||
| 1 | # Implementation plan — executable binaries and the /bin tree | ||
| 2 | |||
| 3 | Companion to ADR 0006. Phases are strictly ordered where dependencies | ||
| 4 | say so; each phase ends with a QEMU verification gate. Version bumps per | ||
| 5 | the glossary rules. | ||
| 6 | |||
| 7 | --- | ||
| 8 | |||
| 9 | ## Phase 0 — System image (build only, no kernel changes) | ||
| 10 | |||
| 11 | The build produces one dd-able FAT32 image instead of the current | ||
| 12 | two-source setup (vvfat `esp/` directory + separate `/tmp/disk.img`). | ||
| 13 | |||
| 14 | - [ ] Extend `scripts/mkdisk.sh` (or add `scripts/mksystem.sh`) to create | ||
| 15 | `system.img`: `mkfs.fat -C`, then `mcopy` in | ||
| 16 | `\bzImage`, `\EFI\` (as rsbl requires), `\etc\*` (motd moves out of | ||
| 17 | the `fs.rs` code literal? — optional here), seed content. | ||
| 18 | - [ ] Stage `\bin\cli\` and `\bin\gui\` directories (empty for now). | ||
| 19 | - [ ] Makefile: build kernel → stage image; `run`/`run-gui`/`debug` | ||
| 20 | attach **only** `system.img` on ide.0 (delete the | ||
| 21 | `-drive file=fat:rw:$(ESP_DIR)` vvfat lines). | ||
| 22 | - [ ] Verify OVMF boots the single image end-to-end. | ||
| 23 | |||
| 24 | **Gate:** image boots; `ls /disk/<UID>/` shows `\bin`, `\etc`, seed files; | ||
| 25 | writes under `/disk/<UID>/` survive poweroff (existing FAT write-back). | ||
| 26 | |||
| 27 | --- | ||
| 28 | |||
| 29 | ## Phase 1 — Per-terminal input discipline (prerequisite) | ||
| 30 | |||
| 31 | Keyboard routing becomes total; canonical mode becomes per-terminal shell | ||
| 32 | property. This is the original "per-instance shell" step, completed. | ||
| 33 | |||
| 34 | - [ ] WM pump routes keys unconditionally: IRQ ring → pump → focused | ||
| 35 | window's queue. No task ever polls the global ring directly. | ||
| 36 | - [ ] Home terminal (window 0) becomes an ordinary focusable window: | ||
| 37 | clicking it focuses it; retire the `FOCUSED == CONSOLE` sentinel | ||
| 38 | and the `focused() == 0` special cases from commit `09a44f1`. | ||
| 39 | `focus_console()` becomes `focus(0)`. | ||
| 40 | - [ ] Each terminal's shell assembles lines canonically from its own | ||
| 41 | queue (`tty::read_line_from_queue` path); delete the global-ring | ||
| 42 | path through `tty::poll_key`. | ||
| 43 | - [ ] Raw-mode consumers that remain built-ins (`edit`, pager) read keys | ||
| 44 | via the same focused-window queue, not the ring. | ||
| 45 | - [ ] Two terminals + home terminal: input always lands in the focused | ||
| 46 | one; unfocused terminals keep rendering output from running | ||
| 47 | commands. | ||
| 48 | |||
| 49 | **Gate:** three terminals open; typing switches cleanly with focus; no | ||
| 50 | keystroke ever appears in two terminals; `make test-disk` regression | ||
| 51 | passes (it drives the shell via injected keys). | ||
| 52 | |||
| 53 | --- | ||
| 54 | |||
| 55 | ## Phase 2 — Loader, Exports table, first exec (**bump 0.6.0**) | ||
| 56 | |||
| 57 | - [ ] New crate `rsapi` (or `rsnix::abi`): the versioned `Exports` struct, | ||
| 58 | entry-point signature, ABI event/argv types shared by kernel and | ||
| 59 | binaries. Header carries `version: u32`. | ||
| 60 | - [ ] `rsnix::load`: PE32+ parser (headers, sections, `.reloc` | ||
| 61 | application), alloc-copy-relocate, entry lookup. ~300–500 lines. | ||
| 62 | - [ ] Spawn path: load → new task (own stack, guard page) → jump to entry | ||
| 63 | with `(&Exports, argc, argv)`; completion message to spawner; | ||
| 64 | pages freed on exit. | ||
| 65 | - [ ] Resolution: name → scan mounted volumes' `\bin\<kind>\<name>` → | ||
| 66 | else built-in → else unknown. | ||
| 67 | - [ ] argv block: shell copies parsed positional args into the binary's | ||
| 68 | memory; `argc/argv` C-style. | ||
| 69 | - [ ] First binary: `hello.efi` in `/bin/cli/` — calls one export | ||
| 70 | (`put_line("hello")`) and returns 0. | ||
| 71 | - [ ] Crash containment check: a `boom.efi` that faults kills only its | ||
| 72 | task; shell survives; `jobs` shows the corpse. | ||
| 73 | |||
| 74 | **Gate:** `hello` runs from the image, prints via exports, exits; | ||
| 75 | `boom` crashes gracefully; built-in fallback still works when the file | ||
| 76 | is absent; **version bump to 0.6.0**. | ||
| 77 | |||
| 78 | --- | ||
| 79 | |||
| 80 | ## Phase 3 — Streams, pipeline integration, line-tool migration | ||
| 81 | |||
| 82 | - [ ] Per-invocation line channel: bounded queue between owning shell and | ||
| 83 | command task; shell pumps its key queue → canonical assembly → | ||
| 84 | channel while a command runs (shell main loop gains a "pumping" mode). | ||
| 85 | - [ ] Exports streams: `get_line() -> {Line, Eof, Abort}`, | ||
| 86 | `put_line(line) -> bool` (false = stop). Shell-side implementations | ||
| 87 | feed the filter pipeline / sink / tty exactly as built-ins do today. | ||
| 88 | - [ ] Grammar translation shell-side: USE/TIL resolve into the injected | ||
| 89 | source; `<path> TIL <marker>` sugar strips paths from argv; | ||
| 90 | filters/sinks wrap `put_line`. Binaries never see keywords. | ||
| 91 | - [ ] ^C semantics: shell sees ^C while pumping → marks channel aborted → | ||
| 92 | command's next stream call returns Abort/stop → completion → shell | ||
| 93 | flushes `-t` windows. `-p` pager blocks inside `put_line`. | ||
| 94 | - [ ] Migrate simple tools to `/bin/cli/`: `echo` stays frozen; migrate | ||
| 95 | `ls`, `mem`, `pci`, `date`, `info`, `mkdir`, `touch`, `rm`, | ||
| 96 | `hexdump`, `klog`, … Each keeps its grammar behaviour verbatim. | ||
| 97 | - [ ] Migrate `cat` last (input consumer of every corner: heredoc, USE, | ||
| 98 | TIL sugar, multi-path). | ||
| 99 | |||
| 100 | **Gate:** full `scripts/test-disk.sh` grammar suite passes unchanged with | ||
| 101 | external binaries; `-h/-t/-g/-p` chains work across the task boundary; | ||
| 102 | ^C flushes a tail window on an infinite stream. | ||
| 103 | |||
| 104 | --- | ||
| 105 | |||
| 106 | ## Phase 4 — GUI binaries | ||
| 107 | |||
| 108 | - [ ] Window/event exports: `create_window(title,w,h) -> id`, | ||
| 109 | `window_buffer(id) -> (ptr,w,h)`, `flush(id)`, `destroy(id)`, | ||
| 110 | `recv_event(&mut Ev) -> bool` (Ev mirrors Key/AppMouse/Focus/Close). | ||
| 111 | - [ ] Fire-and-forget spawn: shell returns immediately after spawning; | ||
| 112 | exit = destroy window + task done. | ||
| 113 | - [ ] Migrate paint → `/bin/gui/paint`; `draw` resolves through the | ||
| 114 | normal rule (no special case). | ||
| 115 | - [ ] F1-spawned terminals keep working alongside GUI binaries. | ||
| 116 | |||
| 117 | **Gate:** `draw` launches the migrated binary; pen draws; F4/close button | ||
| 118 | closes it; focus cycle and title-bar drag unaffected. | ||
| 119 | |||
| 120 | --- | ||
| 121 | |||
| 122 | ## Frozen built-in core (permanent) | ||
| 123 | |||
| 124 | `help`, `clear`, `echo`, `poweroff`/`exit`, `halt`, `rescan`. | ||
| 125 | |||
| 126 | Rationale: needed before disks exist, or exist to fix them. `edit` also | ||
| 127 | stays built-in until the deferred raw-mode CLI ABI exists. | ||
| 128 | |||
| 129 | ## Explicitly deferred (non-goals for this rework) | ||
| 130 | |||
| 131 | - Raw-mode surface access in the CLI ABI (needed before `edit` migrates). | ||
| 132 | - FAT shadowing of `/bin` beyond the system image. | ||
| 133 | - Boot-volume identity / root-path unification (`\etc` at `/etc`). | ||
| 134 | - Launcher (will simply enumerate `/bin/gui` once Phase 4 lands). | ||
| 135 | - Fixing compute-bound-binary ^C blindness (accepted policy). | ||
| 136 | |||
| 137 | ## Version story | ||
| 138 | |||
| 139 | - Phases 0–1 land as 0.5.x increments if separate. | ||
| 140 | - **0.6.0 at Phase 2** — exec exists (glossary class `b`: big feature). | ||
