1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
|
# CONTEXT.md — raisernics Kernel Glossary
## raisernics
Custom x86_64 kernel for the RNG ecosystem. Toy project — self-contained,
no strict userspace separation, TempleOS-inspired design philosophy.
Built in Rust, compiled for the `x86_64-unknown-uefi` target (nightly).
## Boot interface
The kernel is a **PE32+ EFI application** loaded via UEFI `LoadImage`/`StartImage`.
The existing bootloader (rsbl) loads us from `\bzImage` and passes the UEFI
system table. No bootloader changes needed.
## Output strategy
Two-phase console model:
- **Phase 1 (boot services alive):** UEFI `con_out->OutputString()` — works
with whatever UEFI firmware routes (serial + framebuffer).
- **Phase 2 (native):** Raw serial UART 16550 driver on COM1 (I/O port
`0x3F8`). Activated after `ExitBootServices` when we own the hardware.
## Dependency philosophy
Aligns with RNG philosophy: minimal dependencies. Freestanding Rust,
no libc, no external crates at the kernel/core level. UEFI protocol
structs are hand-defined (not from `uefi-rs` crate) to stay lightweight.
## Paging strategy
Long mode mandates paging (`CR0.PG = 1`), so we accept that tax but keep it
simple. Identity-mapped 4-level tables, no higher-half, no demand paging,
no guard pages. `paging.rs` provides identity map/unmap/query helpers. At
boot the kernel runs on UEFI's tables; `adopt_tables()` rebuilds that map
into the kernel's own tables and switches CR3 — needed because UEFI maps its
page-table pages read-only, so the firmware tree can't be modified in place.
Firmware large pages (2 MB / 1 GB) are split into 4 KB tables on demand;
page-table pages come from the physical page allocator.
## Physical memory
Before `ExitBootServices`, the kernel reads the UEFI memory map to know
which physical regions are free. The heap is a first-fit **free-list
allocator** with splitting and coalescing: blocks carry a 16-byte header
(size + free flag), free blocks are linked in address order, and an 8-byte
back-pointer before each payload lets `free` find its block even when
alignment left a gap. When the free list is exhausted, the heap **grows**
by taking pages from the page allocator (proportional to the request),
keeping the two allocators cooperating. A **bitmap page allocator** manages
physical pages: one bit per 4 KB page spanning the conventional regions, so
MMIO holes and firmware-reserved gaps are never handed out, and the heap
region itself is reserved to keep the allocators from double-allocating the
same RAM. Contiguous runs with power-of-two alignment are supported
(`alloc_pages_aligned`).
## Input strategy
**PS/2 keyboard polling.** Port I/O to `0x60`/`0x64`. Scancode set 1
(US QWERTY layout in the initial build). No interrupts — polling loop
inside the shell.
## Display strategy
Two-phase model matching output:
- **Phase 1:** UEFI `con_out->OutputString()` (framebuffer + serial handled
by firmware).
- **Phase 2:** GOP (Graphics Output Protocol) framebuffer obtained before
`ExitBootServices`, driven by a built-in 8×16 bitmap font. Text rendered
by drawing glyphs directly into the framebuffer.
The framebuffer address stays valid after `ExitBootServices` — the GPU's
linear frame buffer is system memory that we own once boot services are gone.
## Keyboard layout
Scancode set 1, US QWERTY layout. Hardcoded scancode→ASCII table.
Other layouts can be added later by swapping the translation table.
## Development loop
Standalone project — no integration with the RNG initramfs build system.
Testing via QEMU with OVMF. The kernel PE binary is placed on a FAT
image as `\bzImage` (matching rsbl's expected path).
## Project layout
Single Rust crate (`x86_64-unknown-uefi` target). Module breakdown:
- `main.rs` — entry point, panic handler, top-level init
- `uefi.rs` — UEFI protocol type definitions
- `serial.rs` — UART 16550 driver
- `paging.rs` — identity page table helpers
- `memory.rs` — UEFI memory map, free-list heap allocator (grows from page
allocator), bitmap page allocator
- `paging.rs` — identity page-table helpers (map/unmap/query, large-page split)
- `power.rs` — `ResetSystem`-based shutdown, hlt loop fallback
- `rtc.rs` — CMOS/RTC clock (ports 0x70/0x71), BCD/12h normalization
- `pci.rs` — legacy PCI config space (0xCF8/0xCFC), full enumeration; `config_write`
- 64-bit BAR reader for the AHCI driver
- `ahci.rs` — SATA AHCI host driver (polling, DMA, no interrupts): scans PCI for
class 1/subclass 6/prog-if 1, maps ABAR, starts port 0 (FRE/ST/DMAE), runs
IDENTIFY + READ/WRITE DMA EXT through a 4 KiB DMA buffer
- `fat.rs` — FAT32: BPB parse, whole-FAT cache in RAM with write-back to both FAT
copies, cluster chains, directory scan with checksum-validated LFN
reassembly, LFN + 8.3 writing, FSInfo free-count maintenance, mkdir/rm/read/write
- `fs.rs` — VFS: dirs, ramfs files, char devices (`/dev/mem`, `rtc`, `pci`,
`version`, `meminfo`, `null`, `zero`, `console`, `serial`; `/tmp` ramfs;
`/etc/motd`); `/disk` mount node + 24-slot ephemeral node pool so shell
commands work on disk paths unchanged
- `gop.rs` — framebuffer rasteriser (glyph blits, pixel shifts); the text
surface itself lives in `vt`
- `vt.rs` — terminal emulator: cell grid, cursor, ANSI CSI parser (H/J/K,
cursor moves, SGR 16-colour, `?25l/h`), dirty-cell blitting, UTF-8
decoding (box-drawing `│`/`═`, arrows `←↑→↓`, em dash `—` render
procedurally instead of `?`). The inverse cursor is mirrored into the
rendered grid so scrolling / `clear` can never strand stale inverse
cells on the framebuffer.
- `tty.rs` — line discipline: canonical `read_line` (echo, line buffer,
^C/^D) and raw `read_byte` (terminal byte protocol: arrows → `ESC [ A–D`,
^X → `0x18`, …)
- `strbuf.rs` — stack fixed-capacity ASCII string (moved from shell)
- `font.rs` — 8×16 bitmap font data
- `ps2.rs` — PS/2 keyboard driver (polling); generic `Ctrl(char)` and
`CtrlArrow` events
- `shell.rs` — command loop and built-in commands
- `rsdosedit/` — kernel-native port of the RNG `rsedit` terminal editor
(no_std; talks to `tty` raw mode + `vt` ANSI; saves via the VFS)
## Milestone status (v0.1)
✅ **Booting** — kernel loads via rsbl, prints banner via UEFI console (verified on real hardware)
✅ **ExitBootServices** — memory map captured via static `.bss` buffer
(see `docs/adr/0001-static-buffer-for-memory-map.md`)
✅ **Serial** — UART 16550 at COM1, 115200 8N1
✅ **Framebuffer** — GOP located, 8×16 bitmap font renders text
✅ **Keyboard** — PS/2 polling, scancode set 1, US QWERTY
✅ **Shell** — commands: `help`, `echo`, `clear`, `hello`, `info`, `mem`, `halt`, `poweroff`
✅ **Heap allocator** — first-fit free-list with splitting + coalescing, `alloc`/`free`
(boot-time self-test verifies alignment, reuse, and full coalescing)
✅ **Physical pages** — bitmap allocator over conventional regions, contiguous
runs, heap reservation, page-0 trap guard (boot-time self-test included)
✅ **Paging helpers** — identity map/unmap/query, 2 MB/1 GB large-page split,
own tables adopted at boot (boot-time self-test included)
✅ **Heap growth** — heap expands into the page allocator when exhausted
✅ **Shell** — `mmap` dumps the UEFI map, `stress` exercises alloc/free + growth
✅ **Poweroff** — `poweroff`/`exit` shut down via UEFI `ResetSystem`
(ACPI S5; `halt` keeps the plain hlt loop)
✅ **RTC** — CMOS clock read at boot and via `date` (BCD/12h handled)
✅ **PCI** — port-based config access, full bus enumeration via `pci`
✅ **VFS** — “everything is a file”: dirs, ramfs, char devices
(`ls`/`cat`/`hexdump`/`mkdir`/`touch`/`rm`, `echo >`/`>>`; devices
regenerate on read, `/dev/mem` is raw physical RAM)
✅ **^C / ^D** — Ctrl+C cancels the line or aborts a running stream
(`cat`/`hexdump` check between chunks), Ctrl+D on an empty line
powers off (EOF)
✅ **tty + vt (terminal) — the 1:1 UNIX console split**: `tty` owns the
line discipline (canonical echo/buffer/^C/^D for the shell; raw mode
translates keys to the classic terminal byte protocol), `vt` owns the
screen (cell grid + ANSI CSI parser + 16-colour SGR + dirty-cell blits
into the GOP framebuffer). `gop_println!` etc. now render through `vt`.
✅ **rsdosedit** — the RNG `rsedit` editor ported to the kernel (no_std,
no libc): `edit <path>` opens a full-screen editor with undo, syntax
highlighting, search (^W), save (^S), exit (^X, twice to discard), word
nav (Ctrl+arrows), smart indent. Files live in the ramfs. Termios →
`tty` raw mode, stdout → `vt`, `std::fs` → VFS, SIGWINCH → fixed
geometry. Verified in QEMU: edit/type/navigate/insert/save/exit, exact
bytes via `cat`/`hexdump`, syntax colours via framebuffer dump.
✅ **AHCI + FAT32 (persistent disk)** — a SATA AHCI driver (polling DMA,
no interrupts) plus a FAT32 filesystem, mounted as `/disk`; every shell
command works on disk paths: `ls`/`cat`/`hexdump`/`echo >`/`>>`/
`mkdir`/`touch`/`rm`/`edit`. The driver scans **all** implemented
ports (CAP.NP/PI) and starts every port with a device; `fat::init`
tries each disk in port order and mounts the first valid FAT32
volume — so the kernel finds the disk no matter which AHCI port the
firmware/QEMU put it on (real hardware included). Whole-FAT cache with write-back to both
FAT copies, checksum-validated LFN read/write, multi-cluster files,
space-padded `.`/`..` dot entries (`..` → cluster 0 under the root),
FSInfo free-count maintenance (fsck-clean after a session, verified
with `fsck.fat -n`). **Files survive reboot** — verified by
poweroff → `mdir`/`mtype` on the image and by a fresh boot seeing the
prior session's files.
QEMU setup: `-drive id=disk,if=none,format=raw,file=disk.img` +
`-device ide-hd,drive=disk,bus=ide.0` puts the data disk on AHCI port
0; the ESP gets `bus=ide.1`. (The implicit `-drive file=fat:rw:esp`
otherwise grabs port 0.) Make targets (`run`, `run-gui`, `debug`)
attach `$(DISK_IMG)` (default `/tmp/disk.img`). OVMF writes its
`NvVars` varstore file onto the first FAT volume — harmless.
## Known issues / perf notes
- **Scroll cost** — scrolling still shifts the whole framebuffer (a
~3-8 MB pixel copy per row), but two mechanisms make it fast on real
hardware:
1. a RAM backbuffer (`gop::SHADOW`): every raster write goes to the
shadow too; scrolling is a cached shadow memcpy + one write-only
push to the display — the display is never *read* (uncached reads
were the killer, since real-hardware GOP framebuffers are
uncacheable/write-combining).
2. deferred scroll batching (`vt::PENDING_SCROLLS`): the logical grid
scrolls instantly in RAM while the framebuffer shift+push is
deferred; N pending scrolls cost one push. Bursts of streaming
output (hexdump/cat) push every ~16 rows (or after ~40 ms of
pause), and the shell syncs before each prompt. Any CSI sequence
(full-screen apps like the editor) or interactive input pushes
immediately.
Still on the someday list: a pan-based ring so scrolling never touches
the whole framebuffer at all.
- **Editor redraw** re-blits changed cells per keystroke (fine at 1280×800);
the "optimize drawing someday" item.
- **AHCI quirks learned (QEMU ground truth)**: the command header layout is
FIS length in DW0 bits 4:0 and PRDTL in bits 31:16 (not the classic
CFL-at-24 bit-fields); the H2D FIS sector count lives at bytes 12-13 with
the device register at byte 7 (byte 10 is LBA[47:40] — no separate
device byte); QEMU rejects H2D FISes with flags bits 6:4 set, so the
write direction is signalled purely by the header W bit (1<<6).
Real hardware follows the same FIS layout; the header layout matches
Linux libahci.
- **ramfs only for `/`**: the root filesystem is still the ramfs (lost on
reboot); persistent files live under `/disk` (AHCI/FAT32).
## Target audience
Learning exercise and toy. Not a Linux replacement. Not POSIX-compatible.
|
