# CONTEXT.md — raisernics Kernel Glossary ## raisernics Custom x86_64 kernel for the RNG ecosystem. Toy project — self-contained, no strict userspace separation, TempleOS-inspired design philosophy. Built in Rust, compiled for the `x86_64-unknown-uefi` target (stable). ## Boot interface The kernel is a **PE32+ EFI application** loaded via UEFI `LoadImage`/`StartImage`. The existing bootloader (rsbl) loads us from `\bzImage` and passes the UEFI system table. No bootloader changes needed. ## Output strategy Two-phase console model: - **Phase 1 (boot services alive):** UEFI `con_out->OutputString()` — works with whatever UEFI firmware routes (serial + framebuffer). - **Phase 2 (native):** Raw serial UART 16550 driver on COM1 (I/O port `0x3F8`). Activated after `ExitBootServices` when we own the hardware. ## Dependency philosophy Aligns with RNG philosophy: minimal dependencies. Freestanding Rust, no libc, no external crates at the kernel/core level. UEFI protocol structs are hand-defined (not from `uefi-rs` crate) to stay lightweight. ## Paging strategy Long mode mandates paging (`CR0.PG = 1`), so we accept that tax but keep it simple. Identity-mapped 4-level tables, no higher-half, no demand paging, no guard pages except task stacks — each task's stack gets one unmapped guard page below it, and a page-fault handler turns an overflow into a task panic. `paging.rs` provides identity map/unmap/query helpers. At boot the kernel runs on UEFI's tables; `adopt_tables()` rebuilds that map into the kernel's own tables and switches CR3 — needed because UEFI maps its page-table pages read-only, so the firmware tree can't be modified in place. Firmware large pages (2 MB / 1 GB) are split into 4 KB tables on demand; page-table pages come from the physical page allocator. ## Physical memory Before `ExitBootServices`, the kernel reads the UEFI memory map to know which physical regions are free. The heap is a first-fit **free-list allocator** with splitting and coalescing: blocks carry a 16-byte header (size + free flag), free blocks are linked in address order, and an 8-byte back-pointer before each payload lets `free` find its block even when alignment left a gap. When the free list is exhausted, the heap **grows** by taking pages from the page allocator (proportional to the request), keeping the two allocators cooperating. A **bitmap page allocator** manages physical pages: one bit per 4 KB page spanning the conventional regions, so MMIO holes and firmware-reserved gaps are never handed out, and the heap region itself is reserved to keep the allocators from double-allocating the same RAM. Contiguous runs with power-of-two alignment are supported (`alloc_pages_aligned`). ## Input strategy **PS/2 keyboard, interrupt-driven.** IRQ1 pushes scancodes into a ring buffer (`interrupts::KEYBUF`); the shell polls it via `read_scancode()`. Port I/O to `0x60`/`0x64`. Scancode set 1 (US QWERTY layout in the initial build). ## Display strategy Two-phase model matching output: - **Phase 1:** UEFI `con_out->OutputString()` (framebuffer + serial handled by firmware). - **Phase 2:** GOP (Graphics Output Protocol) framebuffer obtained before `ExitBootServices`, driven by a built-in 8×16 bitmap font. Text rendered by drawing glyphs directly into the framebuffer. The framebuffer address stays valid after `ExitBootServices` — the GPU's linear frame buffer is system memory that we own once boot services are gone. ## Window A rectangular display surface owned by exactly one task and composited onto the 800×600 canvas by the window manager. The global console (vt) becomes the first window, owned by the shell; further windows are spawned per task (a terminal, a drawing canvas, …). Windows may overlap, so they carry a z-order. Every window is a pixel surface: a terminal window is a text grid rendered into its own pixels, a drawing canvas draws pixels directly. The ruling constraint is "stupid simple". ## Chrome Window-manager-owned UI drawn on the canvas but not a window: the background/desktop fill, per-window title bars, and (later) a launcher/menu. Chrome is not focusable and does not take input as a window. ## Title bar Per-window chrome strip at the top of a window showing the window's title and its minimize and close buttons. Drawn by the window manager, not the app. ## Minimized A window state in which the window is not rendered (hidden); the app keeps running. The inverse of the normal visible state. ## Close The action that destroys a window and ends its app. ## App A task that owns exactly one window and receives input events through its message queue. A GUI program is an app; there is no separate "process" concept — the app is its task plus its window. The home terminal is app 0 (window 0). ## Binary A PE32+ file holding executable code that the kernel can load and run without being recompiled into the kernel image. A binary runs against the kernel's exported interface, never by linking kernel state directly. Binaries come in two kinds, separated by **launch surface** rather than implementation: **CLI binaries** live in /bin/cli and run inside a terminal session (invoked by a shell, inheriting its input/output); **GUI binaries** live in /bin/gui and are launchable with no terminal at all — they own their windows — which makes enumerating launchable apps as simple as listing /bin/gui. Every binary runs as its own task — a CLI binary's task is waited on by the calling terminal, a GUI binary's task owns its window. ## System image The single dd-able FAT filesystem image the build produces: bootloader, kernel, /bin, /etc and seed content on one volume. Imaging it onto a disk yields a complete installed system. Binaries live on the system image under /bin/cli and /bin/gui; the shell finds them by scanning mounted volumes — no volume is privileged. For user-facing paths, /bin is a transparent VFS alias: `ls /bin/cli` resolves to the first mounted volume whose root has \bin, so nobody needs /disk//bin/... (ADR 0006). ## Loader The kernel facility that takes a binary from disk, places it in memory, and starts it. Loading changes no address-space setup — binaries run in the one shared identity-mapped space. ## Exports table The interface between a binary and the kernel: a versioned table of functions the kernel hands to a binary at entry (output, filesystem, memory, …). The caller constructs it per invocation, so the same binary's output can be aimed at a terminal or into the shell's filter pipeline. All kernel state stays behind the table — a binary never owns a copy of it. ## Terminal A program that presents a command line in a window: a text terminal emulator plus a shell command loop, rendered into the window's pixel buffer. The home terminal is the persistent full-screen chrome-less desktop; additional terminals are ordinary windows (F1). ## Focus The single window (if any) that receives keyboard and mouse input. Focus is set by clicking a window, which also raises it; input is delivered to the focused app's message queue. With no focused window, a click only sets focus. ## Keybinding A global keyboard shortcut handled by the window manager before input is routed to the focused window. It triggers a WM action (spawn a terminal, focus/restore the next window, close, …) and is never delivered to an app. ## Keyboard layout Scancode set 1, US QWERTY layout. Hardcoded scancode→ASCII table. Other layouts can be added later by swapping the translation table. ## Task A unit of concurrent execution in the kernel: its own kernel stack (with a guard page below it), scheduler bookkeeping, and no address-space separation — all tasks share the one identity-mapped space and the kernel's facilities (TempleOS-style, not a process). Tasks are preemptively round-robin-scheduled on the 100 Hz PIT tick; the shell is task 0 and an idle `hlt` task is the always-ready fallback. A task may sleep, yield, and log to serial/klog; disk access is serialized by a shared disk controller (a re-entrant lock), and each terminal is its own program instance. A panicking or stack-overflowing task dies gracefully (logged, marked done, slot reaped) without taking the kernel down. ## Development loop Standalone project — no integration with the RNG initramfs build system. Testing via QEMU with OVMF. The kernel PE binary is placed on a FAT image as `\bzImage` (matching rsbl's expected path) and also as `\EFI\BOOT\BOOTX64.EFI` (what the firmware boots directly). Two commands, compile + boot: ```sh ./scripts/build.sh # cargo build --workspace --target x86_64-unknown-uefi --release ./scripts/run.sh # stage /tmp/system.img if stale, then boot QEMU ``` (`./scripts/run.sh` alone does both; `--headless`, `--debug`, `--monitor`, `--fresh` as per its `--help`. `make` / `make run` are thin aliases.) ## Versioning Workspace version is `a.b.c` (single version for all crates): - **`a`** — very big overhaul (the kind of change that is 1.0.0 → 2.0.0 in other software). Reserved for fundamental rewrites; a future GUI era could qualify. - **`b`** — big feature (multitasking, the shell grammar, klog…). - **`c`** — fix or small feature. Bump the workspace version in the root `Cargo.toml` when a change of that class lands — do not leave the version to drift. Current: `0.5.4` — the GUI era landed with 0.5.0 (graphics layer + cursor compositor); 0.5.1 fixed the canvas at 800×600; 0.5.2 added fault diagnostics (screen + serial dumps for every CPU exception) and real-hardware hardening (framebuffer-size clamping, fb-region exclusion from the allocators, authoritative fb geometry logging); 0.5.3 changed the scroll's full-frame push to per-word volatile stores like the boot-time `clear()` instead of one 1.92 MB rep-movsb (a REP MOVS to an uncached real-GPU framebuffer can stall forever — a stall, not an exception — hence no crash dump); 0.5.4 allocates the shadow backbuffer from the fb-aware page allocator (it can no longer land on the framebuffer — the heap free list spans the fb hole, so a 1.9 MB shadow could alias the display and turn every scroll into an MMIO→MMIO memcpy) and converts the scroll's shadow memmove to per-word stores as a belt-and-suspenders measure; 0.5.5 makes every boot stage visible on real hardware (no serial console): `gop::boot_marker` prints the boot progress straight to the framebuffer (the last marker before a hang identifies the death site) and pre-shell panics now draw a white-on-red `*** PANIC ***` band instead of a silent blank screen, and the shadow fallback allocation is panic-proof (try_reserve; a heap OOM degrades to no-shadow mode instead of halting); 0.5.6 makes every fault dump carry the kernel image base + a stack backtrace (rbp-chain plus a raw scan of the stack above the interrupted rsp, filtered to the in-memory .text range parsed from the PE headers — so a real-hardware crash can be resolved against the PDB offline); 0.5.7 (current) adds a full register dump to every fault line (rax..rdi, rbx, r8..r15) and an allocator corruption guard: `dealloc` validates the back-pointer at `[ptr-8]` (in-heap block, sane size, in-use) and, when the header is clobbered, prints `ptr`/`block`/heap range + the memory around the block + the live stack chain, writes the crash band + crash.log, and halts — instead of the opaque #GP inside `__rust_dealloc` that the OptiPlex showed. A future major GUI overhaul (graphics windows beyond the text surface) would be the `a` bump (1.0.0). ## Project layout Workspace of four crates, all built for the `x86_64-unknown-uefi` target: - `rsnix` — the kernel library (module breakdown below) - `shell` — command loop and built-ins - `rsdos` — EFI entry point (`efi_main`) and boot orchestration - `rsdosedit` — kernel-native terminal editor `rsnix` module breakdown: - `main.rs` — entry point, panic handler, top-level init - `uefi.rs` — UEFI protocol type definitions - `serial.rs` — UART 16550 driver - `paging.rs` — identity page table helpers - `memory.rs` — UEFI memory map, free-list heap allocator (grows from page allocator), bitmap page allocator - `paging.rs` — identity page-table helpers (map/unmap/query, large-page split) - `power.rs` — `ResetSystem`-based shutdown, hlt loop fallback - `rtc.rs` — CMOS/RTC clock (ports 0x70/0x71), BCD/12h normalization - `pci.rs` — legacy PCI config space (0xCF8/0xCFC), full enumeration; `config_write` - 64-bit BAR reader for the AHCI driver - `ahci.rs` — SATA AHCI host driver (polling, DMA, no interrupts): scans PCI for class 1/subclass 6/prog-if 1, maps ABAR, starts port 0 (FRE/ST/DMAE), runs IDENTIFY + READ/WRITE DMA EXT through a 4 KiB DMA buffer - `fat.rs` — FAT32: BPB parse, whole-FAT cache in RAM with write-back to both FAT copies, cluster chains, directory scan with checksum-validated LFN reassembly, LFN + 8.3 writing, FSInfo free-count maintenance, mkdir/rm/read/write - `fs.rs` — VFS: dirs, ramfs files, char devices (`/dev/mem`, `rtc`, `pci`, `version`, `meminfo`, `null`, `zero`, `console`, `serial`; `/tmp` ramfs; `/etc/motd`); `/disk` mount node + 24-slot ephemeral node pool so shell commands work on disk paths unchanged - `gop.rs` — framebuffer rasteriser (glyph blits, pixel shifts); the text surface itself lives in `vt` - `vt.rs` — terminal emulator: cell grid, cursor, ANSI CSI parser (H/J/K, cursor moves, SGR 16-colour, `?25l/h`), dirty-cell blitting, UTF-8 decoding (box-drawing `│`/`═`, arrows `←↑→↓`, em dash `—` render procedurally instead of `?`). The inverse cursor is mirrored into the rendered grid so scrolling / `clear` can never strand stale inverse cells on the framebuffer. - `tty.rs` — line discipline: canonical `read_line` (echo, line buffer, ^C/^D) and raw `read_byte` (terminal byte protocol: arrows → `ESC [ A–D`, ^X → `0x18`, …) - `strbuf.rs` — stack fixed-capacity ASCII string (moved from shell) - `font.rs` — 8×16 bitmap font data - `ps2.rs` — PS/2 keyboard driver (polling); generic `Ctrl(char)` and `CtrlArrow` events - `shell.rs` — command loop and built-in commands - `rsdosedit/` — kernel-native port of the RNG `rsedit` terminal editor (no_std; talks to `tty` raw mode + `vt` ANSI; saves via the VFS) ## Milestone status (v0.1) ✅ **Booting** — kernel loads via rsbl, prints banner via UEFI console (verified on real hardware) ✅ **ExitBootServices** — memory map captured via static `.bss` buffer (see `docs/adr/0001-static-buffer-for-memory-map.md`) ✅ **Serial** — UART 16550 at COM1, 115200 8N1 ✅ **Framebuffer** — GOP located, 8×16 bitmap font renders text ✅ **Keyboard** — PS/2 polling, scancode set 1, US QWERTY ✅ **Shell** — commands: `help`, `echo`, `clear`, `hello`, `info`, `mem`, `halt`, `poweroff` ✅ **Heap allocator** — first-fit free-list with splitting + coalescing, `alloc`/`free` (boot-time self-test verifies alignment, reuse, and full coalescing) ✅ **Physical pages** — bitmap allocator over conventional regions, contiguous runs, heap reservation, page-0 trap guard (boot-time self-test included) ✅ **Paging helpers** — identity map/unmap/query, 2 MB/1 GB large-page split, own tables adopted at boot (boot-time self-test included) ✅ **Heap growth** — heap expands into the page allocator when exhausted ✅ **Shell** — `mmap` dumps the UEFI map, `stress` exercises alloc/free + growth ✅ **Poweroff** — `poweroff`/`exit` shut down via UEFI `ResetSystem` (ACPI S5; `halt` keeps the plain hlt loop) ✅ **RTC** — CMOS clock read at boot and via `date` (BCD/12h handled) ✅ **PCI** — port-based config access, full bus enumeration via `pci` ✅ **VFS** — “everything is a file”: dirs, ramfs, char devices (`ls`/`cat`/`hexdump`/`mkdir`/`touch`/`rm`, `echo >`/`>>`; devices regenerate on read, `/dev/mem` is raw physical RAM) ✅ **^C / ^D** — Ctrl+C cancels the line or aborts a running stream (`cat`/`hexdump` check between chunks); Ctrl+D is EOF (no poweroff — removed this session; see session notes below) ✅ **tty + vt (terminal) — the 1:1 UNIX console split**: `tty` owns the line discipline (canonical echo/buffer/^C/^D for the shell; raw mode translates keys to the classic terminal byte protocol), `vt` owns the screen (cell grid + ANSI CSI parser + 16-colour SGR + dirty-cell blits into the GOP framebuffer). `gop_println!` etc. now render through `vt`. ✅ **rsdosedit** — the RNG `rsedit` editor ported to the kernel (no_std, no libc): `edit ` opens a full-screen editor with undo, syntax highlighting, search (^W), save (^S), exit (^X, twice to discard), word nav (Ctrl+arrows), smart indent. Files live in the ramfs. Termios → `tty` raw mode, stdout → `vt`, `std::fs` → VFS, SIGWINCH → fixed geometry. Verified in QEMU: edit/type/navigate/insert/save/exit, exact bytes via `cat`/`hexdump`, syntax colours via framebuffer dump. ✅ **AHCI + FAT32 (persistent disk)** — a SATA AHCI driver (polling DMA, no interrupts) plus a FAT32 filesystem, mounted under `/disk//`; every shell command works on disk paths: `ls`/`cat`/`hexdump`/ `echo >`/`>>`/`mkdir`/`touch`/`rm`/`edit`. The driver scans **all** implemented ports (CAP.NP/PI) and starts every port with a device; `fat::init` mounts **every** valid FAT32 volume it finds, each under `/disk//` where UID = `