# CONTEXT.md — raisernics Kernel Glossary

## raisernics

Custom x86_64 kernel for the RNG ecosystem.  Toy project — self-contained,
no strict userspace separation, TempleOS-inspired design philosophy.
Built in Rust, compiled for the `x86_64-unknown-uefi` target (stable).

## Boot interface

The kernel is a **PE32+ EFI application** loaded via UEFI
`LoadImage`/`StartImage`.
The existing bootloader (rsbl) loads us from `\bzImage` and passes the UEFI
system table.  No bootloader changes needed.

## Output strategy

Two-phase console model:

- **Phase 1 (boot services alive):** UEFI `con_out->OutputString()` — works
  with whatever UEFI firmware routes (serial + framebuffer).
- **Phase 2 (native):** Raw serial UART 16550 driver on COM1 (I/O port
  `0x3F8`).  Activated after `ExitBootServices` when we own the hardware.

## Dependency philosophy

Aligns with RNG philosophy: minimal dependencies.  Freestanding Rust,
no libc, no external crates at the kernel/core level.  UEFI protocol
structs are hand-defined (not from `uefi-rs` crate) to stay lightweight.

## Paging strategy

Long mode mandates paging (`CR0.PG = 1`), so we accept that tax but keep it
simple.  Identity-mapped 4-level tables, no higher-half, no demand paging,
no guard pages except task stacks — each task's stack gets one unmapped
guard page below it, and a page-fault handler turns an overflow into a
task panic.  `paging.rs` provides identity map/unmap/query helpers.  At
boot the kernel runs on UEFI's tables; `adopt_tables()` rebuilds that map
into the kernel's own tables and switches CR3 — needed because UEFI maps its
page-table pages read-only, so the firmware tree can't be modified in place.
Firmware large pages (2 MB / 1 GB) are split into 4 KB tables on demand;
page-table pages come from the physical page allocator.

## Physical memory

Before `ExitBootServices`, the kernel reads the UEFI memory map to know
which physical regions are free.  The heap is a first-fit **free-list
allocator** with splitting and coalescing: blocks carry a 16-byte header
(size + free flag), free blocks are linked in address order, and an 8-byte
back-pointer before each payload lets `free` find its block even when
alignment left a gap.  When the free list is exhausted, the heap **grows**
by taking pages from the page allocator (proportional to the request),
keeping the two allocators cooperating.  A **bitmap page allocator** manages
physical pages: one bit per 4 KB page spanning the conventional regions, so
MMIO holes and firmware-reserved gaps are never handed out, and the heap
region itself is reserved to keep the allocators from double-allocating the
same RAM.  Contiguous runs with power-of-two alignment are supported
(`alloc_pages_aligned`).

## Input strategy

**PS/2 keyboard, interrupt-driven.**  IRQ1 pushes scancodes into a ring
buffer (`interrupts::KEYBUF`); the shell polls it via `read_scancode()`.
Port I/O to `0x60`/`0x64`.  Scancode set 1 (US QWERTY layout in the
initial build).

## Display strategy

Two-phase model matching output:

- **Phase 1:** UEFI `con_out->OutputString()` (framebuffer + serial handled
  by firmware).
- **Phase 2:** GOP (Graphics Output Protocol) framebuffer obtained before
  `ExitBootServices`, driven by a built-in 8×16 bitmap font.  Text rendered
  by drawing glyphs directly into the framebuffer.

The framebuffer address stays valid after `ExitBootServices` — the GPU's
linear frame buffer is system memory that we own once boot services are gone.

## Keyboard layout

Scancode set 1, US QWERTY layout.  Hardcoded scancode→ASCII table.
Other layouts can be added later by swapping the translation table.

## Task

A unit of concurrent execution in the kernel: its own kernel stack
(with a guard page below it), scheduler bookkeeping, and no
address-space separation — all tasks share the one identity-mapped
space and the kernel's facilities (TempleOS-style, not a process).
Tasks are preemptively round-robin-scheduled on the 100 Hz PIT tick;
the shell is task 0 and an idle `hlt` task is the always-ready
fallback.  A task may sleep, yield, and log to serial/klog; only the
shell touches the disk, filesystem, and terminal.  A panicking or
stack-overflowing task dies gracefully (logged, marked done, slot
reaped) without taking the kernel down.

## Development loop

Standalone project — no integration with the RNG initramfs build system.
Testing via QEMU with OVMF.  The kernel PE binary is placed on a FAT
image as `\bzImage` (matching rsbl's expected path).

## Versioning

Workspace version is `a.b.c` (single version for all crates):

- **`a`** — very big overhaul (the kind of change that is 1.0.0 → 2.0.0
  in other software).  Reserved for fundamental rewrites; a future GUI
  era could qualify.
- **`b`** — big feature (multitasking, the shell grammar, klog…).
- **`c`** — fix or small feature.

Bump the workspace version in the root `Cargo.toml` when a change of
that class lands — do not leave the version to drift.  Current:
`0.4.0` — the GUI input foundation (PS/2 mouse + scheduler message
queues) landed as a `b` bump; the GUI era itself is reserved for the
`a` bump (1.0.0).

## Project layout

Workspace of four crates, all built for the `x86_64-unknown-uefi` target:

- `rsnix` — the kernel library (module breakdown below)
- `shell` — command loop and built-ins
- `rsdos` — EFI entry point (`efi_main`) and boot orchestration
- `rsdosedit` — kernel-native terminal editor

`rsnix` module breakdown:

- `main.rs` — entry point, panic handler, top-level init
- `uefi.rs` — UEFI protocol type definitions
- `serial.rs` — UART 16550 driver
- `paging.rs` — identity page table helpers
- `memory.rs` — UEFI memory map, free-list heap allocator (grows from page
  allocator), bitmap page allocator
- `paging.rs` — identity page-table helpers (map/unmap/query, large-page split)
- `power.rs` — `ResetSystem`-based shutdown, hlt loop fallback
- `rtc.rs` — CMOS/RTC clock (ports 0x70/0x71), BCD/12h normalization
- `pci.rs` — legacy PCI config space (0xCF8/0xCFC), full
  enumeration; `config_write`
  - 64-bit BAR reader for the AHCI driver
- `ahci.rs` — SATA AHCI host driver (polling, DMA, no interrupts): scans PCI for
  class 1/subclass 6/prog-if 1, maps ABAR, starts port 0 (FRE/ST/DMAE), runs
  IDENTIFY + READ/WRITE DMA EXT through a 4 KiB DMA buffer
- `fat.rs` — FAT32: BPB parse, whole-FAT cache in RAM with
  write-back to both FAT
  copies, cluster chains, directory scan with checksum-validated LFN
  reassembly, LFN + 8.3 writing, FSInfo free-count maintenance,
  mkdir/rm/read/write
- `fs.rs` — VFS: dirs, ramfs files, char devices (`/dev/mem`, `rtc`, `pci`,
  `version`, `meminfo`, `null`, `zero`, `console`, `serial`; `/tmp` ramfs;
  `/etc/motd`); `/disk` mount node + 24-slot ephemeral node pool so shell
  commands work on disk paths unchanged
- `gop.rs` — framebuffer rasteriser (glyph blits, pixel shifts); the text
  surface itself lives in `vt`
- `vt.rs` — terminal emulator: cell grid, cursor, ANSI CSI parser (H/J/K,
  cursor moves, SGR 16-colour, `?25l/h`), dirty-cell blitting, UTF-8
  decoding (box-drawing `│`/`═`, arrows `←↑→↓`, em dash `—` render
  procedurally instead of `?`).  The inverse cursor is mirrored into the
  rendered grid so scrolling / `clear` can never strand stale inverse
  cells on the framebuffer.
- `tty.rs` — line discipline: canonical `read_line` (echo, line buffer,
  ^C/^D) and raw `read_byte` (terminal byte protocol: arrows → `ESC [ A–D`,
  ^X → `0x18`, …)
- `strbuf.rs` — stack fixed-capacity ASCII string (moved from shell)
- `font.rs` — 8×16 bitmap font data
- `ps2.rs` — PS/2 keyboard driver (polling); generic `Ctrl(char)` and
  `CtrlArrow` events
- `shell.rs` — command loop and built-in commands
- `rsdosedit/` — kernel-native port of the RNG `rsedit` terminal editor
  (no_std; talks to `tty` raw mode + `vt` ANSI; saves via the VFS)

## Milestone status (v0.1)

✅ **Booting** — kernel loads via rsbl, prints banner via UEFI
  console (verified on real hardware)
✅ **ExitBootServices** — memory map captured via static `.bss` buffer
  (see `docs/adr/0001-static-buffer-for-memory-map.md`)
✅ **Serial** — UART 16550 at COM1, 115200 8N1
✅ **Framebuffer** — GOP located, 8×16 bitmap font renders text
✅ **Keyboard** — PS/2 polling, scancode set 1, US QWERTY
✅ **Shell** — commands: `help`, `echo`, `clear`, `hello`, `info`,
  `mem`, `halt`, `poweroff`
✅ **Heap allocator** — first-fit free-list with splitting +
  coalescing, `alloc`/`free`
  (boot-time self-test verifies alignment, reuse, and full coalescing)
✅ **Physical pages** — bitmap allocator over conventional regions, contiguous
  runs, heap reservation, page-0 trap guard (boot-time self-test included)
✅ **Paging helpers** — identity map/unmap/query, 2 MB/1 GB large-page split,
  own tables adopted at boot (boot-time self-test included)
✅ **Heap growth** — heap expands into the page allocator when exhausted
✅ **Shell** — `mmap` dumps the UEFI map, `stress` exercises alloc/free + growth
✅ **Poweroff** — `poweroff`/`exit` shut down via UEFI `ResetSystem`
  (ACPI S5; `halt` keeps the plain hlt loop)
✅ **RTC** — CMOS clock read at boot and via `date` (BCD/12h handled)
✅ **PCI** — port-based config access, full bus enumeration via `pci`
✅ **VFS** — “everything is a file”: dirs, ramfs, char devices
  (`ls`/`cat`/`hexdump`/`mkdir`/`touch`/`rm`, `echo >`/`>>`; devices
  regenerate on read, `/dev/mem` is raw physical RAM)
✅ **^C / ^D** — Ctrl+C cancels the line or aborts a running stream
  (`cat`/`hexdump` check between chunks); Ctrl+D is EOF (no poweroff —
  removed this session; see session notes below)
✅ **tty + vt (terminal) — the 1:1 UNIX console split**: `tty` owns the
  line discipline (canonical echo/buffer/^C/^D for the shell; raw mode
  translates keys to the classic terminal byte protocol), `vt` owns the
  screen (cell grid + ANSI CSI parser + 16-colour SGR + dirty-cell blits
  into the GOP framebuffer).  `gop_println!` etc. now render through `vt`.
✅ **rsdosedit** — the RNG `rsedit` editor ported to the kernel (no_std,
  no libc): `edit <path>` opens a full-screen editor with undo, syntax
  highlighting, search (^W), save (^S), exit (^X, twice to discard), word
  nav (Ctrl+arrows), smart indent.  Files live in the ramfs.  Termios →
  `tty` raw mode, stdout → `vt`, `std::fs` → VFS, SIGWINCH → fixed
  geometry.  Verified in QEMU: edit/type/navigate/insert/save/exit, exact
  bytes via `cat`/`hexdump`, syntax colours via framebuffer dump.
✅ **AHCI + FAT32 (persistent disk)** — a SATA AHCI driver (polling DMA,
  no interrupts) plus a FAT32 filesystem, mounted under `/disk/<UID>/`;
  every shell command works on disk paths: `ls`/`cat`/`hexdump`/
  `echo >`/`>>`/`mkdir`/`touch`/`rm`/`edit`.  The driver scans **all**
  implemented ports (CAP.NP/PI) and starts every port with a device;
  `fat::init` mounts **every** valid FAT32 volume it finds, each under
  `/disk/<UID>/` where UID = `<LABEL>-<8-hex-serial>` (e.g.
  `RSDOS-5BD1E55F`) — sanitized uppercase label + volume serial, unique
  per format, stable across reboots, human-recognizable and
  machine-parseable (`^[A-Z0-9_]+-[0-9A-F]{8}$`); cloned volumes get a
  `-N` suffix.  So the kernel finds the disks no matter which AHCI port
  the firmware/QEMU put them on (real hardware included), and every
  disk is visible at once (verified: `RSDOS-…` + `DATA2-…` mounted
  side by side, writes to both, fsck-clean).  Whole-FAT cache per
  volume with write-back to both FAT copies, checksum-validated LFN
  read/write (multi-entry names >13 chars stored last-first on disk —
  both the writer and reader were reversed and cancelled out; fixed),
  multi-cluster files, space-padded `.`/`..` dot entries (`..` →
  cluster 0 under the root), FSInfo free-count maintenance
  (fsck-clean after a session, verified with `fsck.fat -n`).
  **Files survive reboot** — verified by poweroff → `mdir`/`mtype` on
  the image and by a fresh boot seeing the prior session's files.

✅ **Shell grammar (ADR 0002)** — one canonical grammar for every
  command: `cmd args [USE src] [TIL marker] [filters…] [output kw dst]`.
  Keyword redirection (`WRITING`/`WRITE` overwrite, `TO` append,
  `USE`/`USING` input source, `TIL`/`UNTIL` heredoc/terminator;
  uppercase-only, aliases valid, fully reserved), C-style quoting
  (`"…"` groups, `\x` escapes — quoted text is never an operator),
  post-filter stages applied left to right (`-h N` head kills the
  upstream, `-t N` tail holds back and flushes at EOF/^C, `-g PAT`
  smart-case literal substring, `-p` screenful pager; all repeatable),
  `>`/`<`/`<<`/`>>` removed (`>` is plain text), file sinks replace the
  screen, `cat`/`hexdump` consume input when path-less (stdin heredoc,
  `cat USE a TO b` copy, `USE f TIL m` truncation).  ^D is EOF
  everywhere and never powers off.  `help` rewritten; `test-disk.sh`
  and `sendkeys.py` updated to the keyword syntax.

  Verified in QEMU (26 grammar checks on a fresh *and* a reused disk,
  disk round trip, pager pacing): head kills at N lines, ^C flushes a
  `-t` window, smart-case grep, chained/repeated filters, `WRITING`+
  `TO`, `USE..TO` copy, `USE..TIL`, heredoc → file, quoting, `-p`.
  Two bugs found and fixed during verification: (1) `WRITING` over an
  existing file wrote after stale NUL padding — `fs::size()` returns
  the node-cached size, which is stale after `set_len(0)`; the sink
  now writes at literal offset 0 after truncation; (2) `USE f TIL m`
  leaked everything after the marker — the final partial-line emit
  ran even after the marker break; fixed with a `stopped` flag.

  QEMU setup: `-drive id=disk,if=none,format=raw,file=disk.img` +
  `-device ide-hd,drive=disk,bus=ide.0` puts the data disk on AHCI port
  0; the ESP gets `bus=ide.1`.  (The implicit `-drive file=fat:rw:esp`
  otherwise grabs port 0.)  Make targets (`run`, `run-gui`, `debug`)
  attach `$(DISK_IMG)` (default `/tmp/disk.img`) and auto-create it when
  missing.  OVMF writes its `NvVars` varstore file onto the first FAT
  volume — harmless.

  The whole disk workflow is scripted (no manual mkfs/mcopy):

- `make disk` / `./scripts/mkdisk.sh` — create the FAT32 image with
    `mkfs.fat -C` (the flag that actually creates the file; plain
    `mkfs.fat` needs an existing file), label `RSDOS`, seed from
    `seed/` via mtools; `--fresh` re-formats, `--size 128M` sizes it
- `make inspect` — `mdir -i` listing of the image (mtools)
- `make test-disk` / `./scripts/test-disk.sh` — full automated round
    trip: build → deploy → ensure image → boot QEMU headless → drive
    the shell via `sendkey` through the monitor (`ls /disk`, `mkdir`,
    `echo WRITING`, `cat`, `poweroff`) → fsck-clean check → `mtype` verifies
    the written file survived poweroff
- `./scripts/sendkeys.py` — types lines into the guest via the QEMU
    monitor (`--sock`, US layout, named keys for punctuation; bare
    `-`/`.`/`/` are not valid sendkey names)

✅ **`klog` kernel log ring** — a RAM ring of the last 128 serial log
  lines (128 chars each, ~16 KB), fed by `serial_println!` (one choke
  point — every boot/low-level message captured for free).  Sequence
  numbers per line + wrap/drop accounting; a dump header reports how
  many lines were lost to wrap.  Each line carries a `[ssss.mmm]`
  timestamp — monotonic ms since boot from `interrupts::uptime_ms()`:
  the 100 Hz PIT tick (10 ms) plus interpolation from the PIT's live
  1.19 MHz down-counter read twice around the tick load (a jump = a
  tick fired between reads) for true sub-ms resolution; `TICKS` widened
  to u64 so uptime never wraps.  **The CMOS RTC cannot provide
  milliseconds** — it is a 1 Hz BCD device (seconds only) — so the
  millis come from the PIT.  Early-boot lines are `[0000.000]`
  (PIT not yet configured; `PIT_READY` gates interpolation), and
  timestamps are clamped monotonic in the ring.

  Exposed as the `/dev/klog` char
  device; the shell `klog` command is a thin alias for `cat
  /dev/klog` (same filters/redirection), so `klog -t 5`, `klog -g
  fat`, and `klog WRITING /disk/…/boot.log` (persists the boot log)
  all work.  The ring never allocates (const-constructible static);
  only the dump allocates, at shell time.  `klog` rejects positional
  args (usage) and input keywords ("does not accept input").

  **Design consequence: shell output must not feed the ring.**  The
  plan's "hook `serial_println!`" only works if that macro is
  kernel-logging-only — but the shell printed prompts/errors/results
  through it, which would have flooded the boot log with echo/cat
  output (a `cat /dev/mem` would wrap the ring).  The shell now
  writes user-facing output (banner, prompt, `term_print`/
  `term_println`, pipeline sink, halt/poweroff) directly to the
  serial port via `serial::write_str`; tty echo already did.  Only
  kernel messages (rsnix modules + `rsdos/main.rs`) go through
  `serial_println!` now.  Verified in QEMU: 28 boot lines captured
  (seq 0..27, first `=== rsnix boot ===`, last `boot: entering
  shell`), `echo hello` output absent from the ring, `klog WRITING`
  persists a boot log (mtype-verified), wrap behavior verified with a
  temporary 140-line boot loop (128 kept / 40 dropped, seq 40..167,
  oldest live line correct — loop reverted after the test).

✅ **Runtime device detection (hotplug rescan)** — the boot discovery
  pipeline is now re-runnable and diffed against live hardware:

- `ahci`: stores ABAR/port bitmap; `poll_dirty()` is a cheap per-port
    `PxSSTS.DET == 3` probe (safe on every prompt), `rescan()` starts
    newly-present ports (full `start_port` bring-up) and tombstones
    slots of removed ones.  **Slots never shift** — indices stay stable
    so VFS disk handles and `ACTIVE` selection keep working; `select()`
    rejects tombstones.
- `fat`: `Volume` gains a `mounted` flag; `rescan()` unmounts volumes
    whose disk left and mounts every disk without a mounted volume
    (which also re-tries mounts that failed at boot, e.g. a non-FAT
    disk replaced by a FAT one).  UID dedup ignores tombstones, so a
    re-plugged disk keeps its `/disk/<UID>` name.
- `fs`: the init mount loop is factored into `rescan_disks()`, which
    lazily creates `/disk`, ensures a mount node per mounted volume and
    tombstones nodes of unmounted ones — `cat /disk/<gone>/…` fails
    cleanly with "no such file".
- shell: `rescan` command plus an automatic deep rescan before each
    prompt whenever `poll_dirty()` sees a change; a one-line summary
    reports added/removed counts.  All rescan events flow through
    `serial_println!` so they land in klog (`ahci: port N up`, `fat:
    mounted …`) — the boot log doubles as a device log.

  **QEMU testability reality (verified)**: PCIe ADD works end-to-end
  (`-device pcie-root-port,id=rp1` at boot, monitor `device_add
  e1000e,bus=rp1` → the kernel's `pci` re-scan sees it; q35 root bus
  itself rejects hotplug, root ports are required).  Disk hotplug is
  **not** QEMU-modelable: the ich9-ahci's IDE buses refuse
  `device_add ide-hd` ("Bus 'ide.N' does not support hotplugging") and
  `drive_add if=ide` ("Can't hot-add drive to type 1"); PCIe
  `device_del` is a guest-cooperative protocol (the device stays in
  config space until the guest powers off the slot), so removal is
  also untestable in QEMU.  The disk arrival/removal path (PxSSTS
  based) is verified on real hardware instead.

✅ **Preemptive kernel tasks (ADR 0003)** — round-robin multitasking
  on the 100 Hz PIT tick, one shared identity-mapped address space:

- `sched`: 16-slot static task table (no ISR-time allocation); task 0
    = shell (boot stack), task 1 = idle `hlt` loop, tasks 2+ spawned
    via `sched::spawn(entry, name)` with 3 pages each (1 unmapped
    guard + 8 KiB stack); preemptive RR with `int $0x60` software
    yield; `sleep_ms`/`yield_now`; lazy reaping of Done slots.
- `gdt`: kernel-owned GDT mirroring UEFI's selector layout (no
    far-jump; CS/DS/SS keep working) plus a TSS with IST1 (page
    fault) and IST2 (double fault) stacks — a stack overflow faults
    inside the guard page, the CPU pushes the fault frame onto the
    IST stack, and the task dies gracefully instead of triple-
    faulting the machine (the ADR's guard-page promise, honored).
- `interrupts`: `isr_dispatch` returns the next frame pointer in
    rax; asm does `mov rsp, rax` → pop → iretq.  **QEMU quirk that
    bit us**: QEMU's 64-bit interrupt entry always pushes
    SS/RSP/RFLAGS/CS/RIP (40 bytes) and iretq always pops all five
    (its same-privilege branch is never taken for 64-bit iretq), so
    interrupt frames are 176 bytes and the synthetic task frames
    must carry rsp=stack-top + ss — a 160-byte frame made the first
    iretq pop rsp/ss from the unmapped page beyond the frame (task
    started with rsp=0, or the iretq itself page-faulted).  Real
    hardware pushes only 24 bytes at same privilege, so the extra
    fields are inert there; the synthetic frames set them
    explicitly either way.
- Panic handling: a task panic (or guard-page overflow) is caught,
    logged as `sched: task N "name" panicked: …`, the task is marked
    Done, and the scheduler switches away — the shell keeps running.
    Shell/idle/ISR/pre-scheduler panics still halt the kernel.
- Shell: `bg <builtin>` (spinner/ticker/counter demo + deliberate
    `boom` panic + `overflow` recursion), `jobs` (state/ticks/
    panic message), `sleep <ms>` (blocks only the shell), `kill
    <id>`.  Only the shell touches disk/fs/tty/vt; background tasks
    are headless (sleep + serial/klog only).
- Boot order note: `sched::init` must run after the page allocator
    and adopted page tables (the idle's stack + guard need both) —
    it was originally before them, silently dropping the idle task.

  Verified in QEMU (9-step script): demo interleaving at expected
  cadences, sleep keeps background tasks logging, jobs states,
  graceful `boom` panic with corpse in jobs, kill stops a task, and
  `bg overflow` dies on the guard page with the machine intact; full
  regression (rescan/til/klog/grammar/pager) re-passes.

✅ **PS/2 mouse** — auxiliary-channel driver on IRQ12, stream mode
  (3-byte packets), the GUI's second input device:

- `interrupts`: the keyboard byte ring is generalised to `ByteRing`;
    IRQ12 (vector 44) gets its own ring and stub.  Both PS/2 handlers
    route each data-port byte by the controller status register's aux
    bit (bit 5), so keyboard and mouse never steal each other's bytes
    even when both are pending at once.  PIC unmask now covers IRQ2
    (cascade) and IRQ12 (slave bit 4).
- `ps2`: `mouse_init()` enables the aux port, sets config bits
    IRQ12+aux-clock (preserving the keyboard bit), then `set defaults`
    (100 Hz / 4 counts/mm / 1:1) + `enable reporting` via the 0xD4
    channel — each waits for the device ACK with a bounded timeout so
    a machine without a mouse boots normally (log: `ps2: mouse not
    present`).  `try_poll_mouse()` reassembles packets in main
    context from the raw byte ring; the packet-start marker (bit 3)
    resyncs the stream after any lost byte.  Events are **screen-
    relative** (`dx` + = right, `dy` + = down — the PS/2 Y-up wire
    convention is negated), buttons decoded; 9-bit sign extension via
    the packet sign bits.
- `fs`: `/dev/mouse` char device — `cat /dev/mouse` drains up to 32
    pending events (`dx=+N dy=+N buttons=L-R`), reports
    `mouse: no PS/2 mouse` / `mouse: no events` otherwise.
- shell: `mouse` command streams live events (10 ms poll via
    `sleep_ms`) until ^C.

  Verified in QEMU (`make test-mouse`, headless, via the monitor's
  `mouse_move`/`mouse_button` HMP commands): boot line `ps2: mouse
  ready`, `/dev/mouse` drains a pre-consumer move (`8,4`), the live
  stream decodes `mouse_move 20 7` → `dx=+20 dy=+7 buttons=---`,
  `mouse_button 1` → `buttons=L--`, a held-button move
  (`-5,-3` while L down) → `dx=-5 dy=-3 buttons=L--`, release,
  and ^C stops the stream.  `make test-disk` regression re-passes.

✅ **Scheduler message queues (the GUI-era primitive from ADR 0003)** —
  every task gets a static 16-slot ring of `TaskMsg` (4 KB in .bss,
  no ISR-time allocation):

- `TaskMsg` = `Key(KeyEvent)` | `Mouse(MouseEvent)` | `Timer` |
    `User(u64)` — input uses the same decoded types the PS/2 drivers
    produce, so window input routing is literally "post what the
    console would have gotten"; `User(u64)` is the generic escape
    hatch.  `KeyEvent`/`MouseEvent` derive Copy for ring storage.
- `sched::send(id, msg)` — posts from any context (task, shell, ISR)
    under the scheduler lock; wakes a Blocked task straight to Ready;
    errors cleanly on a gone task or a full queue (input is lossy
    under extreme load by design).  `recv()` pops without blocking;
    `recv_blocking()` marks the task Blocked and yields.  **Lost-wakeup
    safety net**: the tick scan also wakes Blocked tasks whose queue is
    non-empty, so a task that blocked *after* its message arrived (or
    whose poster never woke it) is picked up within one 10 ms tick.
- shell: `msgrecv` spawns a task that blocks on recv and logs every
    message to serial (`msgrecv[2]: Key(F3)` …); `send <id> <type>
    [args]` posts `user <n>` / `timer` / `mouse <dx> <dy> <btn>`
    (1=L 2=R 4=M) / `key <name>` (enter esc tab space bksp arrows
    home end pgup pgdn del ctrl-c f1..f12, or a single char); `jobs`
    now shows `q=` (pending message count).
- **Real-input routing, proven**: `mouse <id>` (angle brackets) streams
    decoded IRQ12 events into that task's queue instead of the console
    — the exact loop the window manager will run.  `msgrecv` + `mouse 2`
  - `mouse_move 20 7` showed `msgrecv[2]: Mouse(+20,+7 …)` in QEMU.

  Verified in QEMU (`make test-msg`): all six message types delivered
  in order (user, timer, key enter/ctrl-c/f3, synthesized mouse),
  real routed mouse moves (including a held-button move), the 17th
  send against a 16-cap queue reports `queue full` (tested against
  the shell's never-consumed queue), and `jobs` shows q=0 + task back
  to `blocked` after draining.  `make test-disk` regression re-passes.

## Known issues / perf notes

- **Scroll cost** — scrolling still shifts the whole framebuffer (a
  ~3-8 MB pixel copy per row), but two mechanisms make it fast on real
  hardware:
  1. a RAM backbuffer (`gop::SHADOW`): every raster write goes to the
     shadow too; scrolling is a cached shadow memcpy + one write-only
     push to the display — the display is never *read* (uncached reads
     were the killer, since real-hardware GOP framebuffers are
     uncacheable/write-combining).
  2. deferred scroll batching (`vt::PENDING_SCROLLS`): the logical grid
     scrolls instantly in RAM while the framebuffer shift+push is
     deferred; N pending scrolls cost one push.  Bursts of streaming
     output (hexdump/cat) push every ~16 rows (or after ~40 ms of
     pause), and the shell syncs before each prompt.  Any CSI sequence
     (full-screen apps like the editor) or interactive input pushes
     immediately.
  Still on the someday list: a pan-based ring so scrolling never touches
  the whole framebuffer at all.
- **Editor redraw** re-blits changed cells per keystroke (fine at 1280×800);
  the "optimize drawing someday" item.
- **AHCI quirks learned (QEMU ground truth)**: the command header layout is
  FIS length in DW0 bits 4:0 and PRDTL in bits 31:16 (not the classic
  CFL-at-24 bit-fields); the H2D FIS sector count lives at bytes 12-13 with
  the device register at byte 7 (byte 10 is LBA[47:40] — no separate
  device byte); QEMU rejects H2D FISes with flags bits 6:4 set, so the
  write direction is signalled purely by the header W bit (1<<6).
  Real hardware follows the same FIS layout; the header layout matches
  Linux libahci.
- **ramfs only for `/`**: the root filesystem is still the ramfs (lost on
  reboot); persistent files live under `/disk` (AHCI/FAT32).

## Pending decisions (agreed, not yet implemented)

- **Disk recognition scope — keep simple for now**: no pre-EBS UEFI
  block-device census, no GPT/MBR partition parsing, no NVMe driver.
  (Revisit later: Linux-style recognition = PCI scan → ahci/nvme →
  partitions → superblock autodetect → by-label/by-uuid naming; our
  `/disk/<LABEL>-<SERIAL>` already mirrors the by-label/by-uuid idea.)

## Target audience

Learning exercise and toy.  Not a Linux replacement.  Not POSIX-compatible.

---

## Session notes — shell experience redesign (implemented)

Working from `shell_experience`.  Plan as stated:

- Commands accept **post-filters** as trailing flags: `-h N` (head),
  `-t N` (tail), `-g PAT` (grep).  Flag placement (at the end of the
  command) is respected.
- Redirection uses **keywords**, not `> << < >>`:
  `WRITING`/`WRITE` → overwrite, `UNTIL`/`TIL` → heredoc,
  `USE`/`USING` → input, `TO` → append.
- **No pipes.**  `head`/`tail`/`grep` needs are covered by the flags;
  a pager flag is a maybe.

Resolved this session:

- **Post-filter order = written order, left to right.**  Each flag is a
  filter stage applied to the command's output as a line stream:
  `cat file -h 50 -t 10 -g pat` = "lines containing `pat` among the last
  10 lines of the first 50 lines".  (The `shell_experience` prose for
  that example was loose; the left-to-right rule is canonical.)
- **`-t N` holds back.**  The tail stage buffers a rolling window of the
  last N lines and emits nothing until its upstream ends — EOF, or the
  user's ^C, which flushes the window to the screen.  `cat /dev/mem -g
  100 -t 10` therefore shows nothing while it runs and prints the last 10
  matches when ^C ends it.
- **`-h N` truncates.**  Once N lines have passed through, the head stage
  kills the upstream command — `cat /dev/mem -h 5` peeks 5 lines and
  stops instead of flooding.
- **`-g PAT` = smart-case literal substring.**  Match is case-insensitive
  unless PAT contains an uppercase letter, in which case it is
  case-sensitive (editor-style smart case).  No regex, no wildcards.
- **Post-filters repeat.**  `-h`, `-t` and `-g` may each appear more than
  once; every instance is a separate stage in the left-to-right chain
  (so `-h 10 -h 5` = first 5, `-t 10 -t 5` = last 5, `-g foo -g bar` =
  lines containing both).  After `-g`, the next token is the pattern
  verbatim (`-g -h` = pattern "-h").
- **Redirection keywords pair strictly.**  At most one input keyword and
  one output keyword per command line (both may appear together); the
  input keyword must precede the output keyword.  `USING f TO g` is
  accepted, `TO g USING f` is a syntax error, and so is any second
  keyword of the same kind (`TO a WRITE b`).
- **Trailing-region grammar (canonical order):**
  `cmd args [USE src] [TIL marker] [filters…] [output kw dst]` —
  input keywords first, then filter flags, then the output keyword.
  Rationale: "we feed the command input, parse it with filters, put
  that into output".  `USE` must precede `TIL`; filters come after the
  input keywords and before the output keyword.
- **`TIL` is a terminator, not a source.**  Unlike `USE` (which names an
  input source), `TIL <marker>` ends the input at a marker, so it may
  combine with `USE` — two "input" keywords are allowed, in the fixed
  order `USE` then `TIL`.
- **`TIL` truncates the source at the marker.**  The marker line is not
  part of the input and everything from the marker on is discarded —
  `USE f TIL m` reads only the text of `f` before `m`.  Marker match is
  an exact full-line match.
- **^D is EOF everywhere; ^D/^C no longer power off.**  Inside a `TIL`
  input block, ^D ends the input early exactly like a file's EOF.
  Shutting down via ^D or ^C is removed: at the prompt, ^D on an empty
  line is a no-op (fresh prompt); ^C keeps its line-cancel /
  stream-abort role; `poweroff`/`exit` remain the only shutdown verbs.
- **Input consumers: `cat` and `hexdump` only, and only when path-less.**
  With no path (or with an input keyword), they read the input stream
  (`cat USE a TO b` copies a→b; `cat TIL END` is the classic heredoc).
  All other commands reject input keywords with a syntax error — a
  command declares whether it takes input.  `USE`/`TIL` are general
  building blocks; more consumers will come later.
- **`cat <path> TIL <marker>` is sugar for `cat USE <path> TIL
  <marker>`** (refined this session): the paths are the input source,
  truncated at the first marker line across the combined stream —
  `cat a b TIL m` reads `a` then `b` and stops at the first `m` line.
  The single-source reading is unambiguous, so path args + `TIL` is
  valid; path args + `USE` stays a syntax error (two input sources).
- **Keywords: uppercase-only, aliases valid.**  `WRITE`==`WRITING`,
  `TIL`==`UNTIL`, `USE`==`USING`; lowercase forms are ordinary text, so
  `echo use the force` can never redirect.  Keywords are reserved
  operators — a keyword is always an operator, never an argument
  (collision escape: quoting, a future building block).
- **`-p` = pause flag (pager), more-style.**  A stage that pauses at
  every screenful of output and waits for a keypress; engages only when
  output exceeds one screenful, no args, no scrollback/search.  As a
  stage it chains with the others — `-h N -p -g pat -p` lets the user
  see intermediate steps of the filtering.
- **`> >> < <<` are gone.**  Keywords are the *only* redirection syntax;
  `>` is now ordinary text (`echo a > b` prints `a > b`).  Collateral:
  `scripts/test-disk.sh` and the `help` text must be rewritten to the
  keyword syntax; the milestone bullets above describing `echo >`/`>>`
  record the old behavior (superseded by these notes).
- **Quoting (C): `"..."` groups + literalizes; `\x` escapes one char.**
  Quoted/escaped text is always literal — never an operator, never
  split.  So `echo foo "WRITING" bar` prints `foo WRITING bar`; to
  print literal quote characters: `echo foo \"WRITING\" bar`.  Also
  enables spaces in filenames and multi-word `-g` patterns
  (`-g "a b"`).  Quote chars are delimiters, not content.

Assumed edge semantics (flag if wrong):

- Filtered output goes to both terminal (GOP) and serial, identically.
  A file sink (WRITING/TO) replaces terminal output entirely — screen
  is silent, as `echo >` behaved before.
- `-p` pauses terminal output only; with a file sink it is a no-op.
- `-h N` / `-t N` require N ≥ 1; otherwise syntax error.
- `cat p USE s` (path args *and* an explicit source keyword) is a
  syntax error — the input source is ambiguous.  `cat USE s`, path-less
  `cat TIL m`, and `cat <path> TIL m` (sugar for `USE <path> TIL m`)
  are the valid input forms.
- ADR 0002 (`docs/adr/0002-keyword-redirection-and-post-filters.md`)
  captures the keyword-redirection + post-filter decision; the ADR is
  the authoritative statement, these notes are the working detail.

Open questions (to resolve this session):

1. ~~Order of application of `-h`/`-t`/`-g`~~ — **resolved**: left to
   right.
2. ~~`-h`/`-t` streaming semantics~~ — **resolved**: `-t` hold-back +
   ^C flush; `-h` truncates the upstream.
3. ~~`-h`/`-t` semantics on infinite streams (e.g. `cat /dev/mem`)~~ —
   **resolved**: folded into 2.
4. ~~`-g` pattern semantics (literal substring vs regex; case
   sensitivity)~~ — **resolved**: smart-case literal substring;
   repeatable stages.
5. ~~Keyword details~~ — **resolved**: uppercase-only, both aliases
   valid, keywords fully reserved; quoting (also resolved this
   session) is the escape mechanism.
6. ~~Pager flag: in or out~~ — **resolved**: `-p` pause flag, in.
7. ~~Which commands consume input~~ — **resolved**: `cat` + `hexdump`,
   path-less only; other commands reject input keywords.
8. ~~Fate of the existing `>` / `>>` redirection~~ — **resolved**: full
   removal; `>` is plain text; scripts/help to be rewritten.

## Implementation (this session — done, verified in QEMU)

Everything above is now implemented in `shell/src/lib.rs`:

- **Tokenizer** — `"…"` groups text (delimiter, not content), `\x`
  escapes one char, a token containing any quoted/escaped char is
  flagged literal and can never be an operator or flag.
- **Parser** — canonical grammar for every command; keywords
  uppercase-only with aliases, fully reserved (quoted forms are
  literal data); strict ordering enforced (USE before TIL, input
  before filters, output keyword last) with syntax errors otherwise;
  `-h`/`-t` need N ≥ 1, `-g` takes the next token verbatim; unknown
  flags before the trailing region are ordinary arguments.
- **Filter pipeline** — stages over a line stream, left to right,
  each repeatable: `Head` kills the upstream (producers check the
  `line()` return), `Tail` holds a rolling window and flushes through
  the remaining stages at end-of-stream (EOF, ^C, or head kill),
  `Grep` is smart-case literal substring, `Pause` is a terminal-only
  screenful pager (^C aborts; dropped entirely with a file sink).
- **Sinks** — terminal (GOP + serial, identical) or file; a file sink
  replaces the screen, opens lazily on first output, `WRITING`
  truncates and writes from offset 0, `TO` appends at current size.
  Diagnostics (errors, usage) always go to the terminal, never into
  the pipeline or a file.
- **Input consumers** — `cat`/`hexdump`: no keyword = keyboard until
  ^D, `USE f` = file, `TIL m` = keyboard heredoc, `USE f TIL m` =
  file truncated at the exact marker line (marker and everything
  after discarded), and `cat <path> TIL m` / `hexdump <path> TIL m` =
  the paths as source, truncated at the first marker across the
  combined stream.  Other commands reject input keywords.
- **Ordering enforcement** — `USE`/`TIL` must precede the filters and
  the output keyword; filters must precede the output keyword;
  distinct messages for each violation ("must come before the
  filters" / "must come before the output keyword").
- **^D** — EOF everywhere, at the prompt it is a no-op (fresh
  prompt); ^C keeps line-cancel / stream-abort; `poweroff`/`exit`
  are the only shutdown verbs.
- **Collateral** — `help` text rewritten; `scripts/test-disk.sh` and
  the `sendkeys.py` docstring updated to keyword syntax.

Bugs found by QEMU verification and fixed:

1. `WRITING` over an existing file left stale NUL padding —
   `fs::size()` returns the node-cached size, stale after
   `set_len(0)`; the overwrite path now writes at literal offset 0
   (the pre-redesign echo wrote at 0 too).
2. `cat USE f TIL m` leaked everything after the marker — the
   final-partial-line emit ran even after the marker break; fixed
   with a `stopped` flag in `cat_file_until`/`hexdump_file_until`.
3. `cat file TO out TIL m` blamed the filters — the `trailing` flag
   was set by both filters and the output keyword, so the "must come
   before the filters" error fired for the output keyword too; the
   parser now distinguishes them (`TO dst TIL m` → "TIL must come
   before the output keyword").  Also: filters after the output
   keyword (`TO dst -h 2`) were silently accepted — now a syntax
   error.
4. `cat file TIL m` was rejected as ambiguous — path args + `TIL` is
   now sugar for `USE <path> TIL <marker>` (single source, truncated
   at the marker); path args + `USE` remains a syntax error.

Verified: 26 grammar checks pass on a fresh disk *and* on a reused
one (rewriting existing files), `make test-disk` round trip is
fsck-clean, the `-p` pager pauses at screenfuls and resumes on
keypress, `-h` stops `/dev/mem` at exactly N lines, and ^C flushes a
`-t` window on an infinite stream.

Follow-up verification (this session): 10 checks pass in QEMU for the
`cat <path> TIL <marker>` sugar — truncation at the marker, combined
multi-path stream truncated at the first marker, filters after the
sugar (`-h 1`), the heredoc round trip, all four ordering-error
messages, and the `USE`-with-paths ambiguity message; fsck clean.