usb: real-hardware rework — hubs, both EHCI controllers, keyboard, xHCI hardening

EHCI (OptiPlex 7010 / Q77 and similar Intel PCH):
- enumerate *both* EHCI controllers (00:1a.0 + 00:1d.0), not just the first
- hub support: read the hub class descriptor, power/reset each port via hub
  class requests, read the child speed, then drive the child with split
  transactions carrying the hub's address and port (TT).  On 7-series PCH the
  root port presents the integrated Rate Matching Hub (8087:0024, class 9)
  and the mouse/keyboard sit behind it; without this the driver stopped at
  "no HID mouse/keyboard endpoint in config".
- HID mouse *and* keyboard, boot protocol + idle, per-device periodic QH
  chained into the frame list, per-device report buffers and counters
- periodic re-arm follows the standard shape (qTD re-activated, Next-qTD
  re-linked); the HC owns the status/toggle overlay
- DMA from the page allocator inside [16 MiB, 4 GiB): EHCI programs 32-bit
  addresses and the page bitmap spans all RAM
- 2 ms recovery after SET_ADDRESS

xHCI:
- scratchpad buffers (HCSPARAMS2) and MaxSlotsEn > 1, with per-slot EP0 and
  interrupt rings plus per-slot device contexts
- DMA from the page allocator instead of the heap allocator
- boot protocol + idle, mouse + keyboard, first-report diagnostics
- transfer events are routed to every device before matching a waiter, so a
  second device's enumeration cannot swallow an armed device's report

shared:
- rsnix/src/hid.rs: HID usage -> scancode-set-1 tables, one scancode FIFO
  feeding ps2::feed_scancode, per-device boot-keyboard report differ
- WM input pump polls PS/2 -> UHCI -> EHCI -> xHCI
- memory::alloc_pages_range(count, min, max) for bounded DMA allocations

QEMU coverage: make test-usb (EHCI + xHCI, mouse + keyboard).  QEMU's EHCI
split path delivers only the first report for FS devices, so repeated-report
delivery and the RMH path remain hardware-test items.

1 files changed, 620 insertions, 482 deletions

diff --git a/rsnix/src/usb.rs b/rsnix/src/usb.rs
index 3ed3a0c..5a4136f 100644
--- a/rsnix/src/usb.rs
+++ b/rsnix/src/usb.rs
@@ -1,33 +1,22 @@
 // ---------------------------------------------------------------------------
-// USB host controller driver — xHCI (USB 2.0/3.x) + HID mouse support.
+// USB host controller driver — xHCI (USB 2.0/3.x), HID mouse + keyboard.
 //
-// **Polled** driver: it runs the xHCI event ring from the cursor task loop
-// instead of wiring MSI/INTx into the PIC, matching how the kernel already
-// polls the PS/2 mouse (see `cursor`).
+// **Polled** driver: the event ring is drained from the cursor task loop (and
+// the WM input pump for keyboards) instead of wiring MSI/INTx into the PIC.
 //
-// Minimal, mouse-only scope:
-//   • take the controller out of reset, set up the command + event rings
-//   • reset ports, run the slot/address/configure firmware sequence
-//   • find the HID mouse boot interface, open its interrupt-IN endpoint,
-//     and translate the standard report into `ps2::MouseEvent` (reusing
-//     the cursor task's consumption path).
+// Scope:
+//   • controller reset, command + event rings, scratchpad buffers (§5.4.5)
+//   • one slot per HID device (up to MAX_XDEV), address/configure sequence
+//   • boot-protocol HID mouse and keyboard interfaces
+//   • reports decoded into ps2::MouseEvent / ps2::feed_scancode
 //
-// DMA / MMIO: the kernel is identity-mapped (physical == virtual), so bus
-// addresses are plain kernel pointers and every heap allocation is DMA-able.
-// Rings/contexts are allocated once at boot and handed to the controller
-// permanently.
+// DMA: rings/contexts come from the page allocator (physical == kernel
+// address via the identity map), never the heap, and are 64-byte aligned as
+// the xHCI requires.  The controller is 64-bit capable, so no 4 GiB bound.
 //
-// Register/TRB/context layouts follow the Linux xHCI driver definitions
-// (drivers/usb/host/xhci*.h) and the xHCI 1.2 spec.  All exact field
-// positions were cross-checked against the Linux sources:
-//   * TRB type field is bits 10-15 (TRB_TYPE(p) = p << 10)
-//   * transfer/command completion code is bits 24-31 of event dword2
-//   * slot id = bits 24-31, ep id = bits 16-20 of event dword3
-//   * endpoint context: type at bits 3-5, interval at 16-23,
-//     max packet size at bits 16-31 of dword1, TR dequeue is a 64-bit
-//     field at bytes 8-15 with DCS in bit 0
-//   * slot context: route/LAST_CTX in dword0, root-hub-port at 16-23 of
-//     dword1, speed at bits 20-23
+// Register/TRB/context layouts follow the Linux xHCI driver definitions and
+// the xHCI 1.2 spec (TRB type bits 10-15, completion code bits 24-31 of event
+// dword2, slot id bits 24-31 / ep id bits 16-20 of event dword3).
 // ---------------------------------------------------------------------------
 #![allow(non_snake_case)] // macro-style helpers mirror the Linux xHCI headers
 use core::sync::atomic::{AtomicBool, AtomicU32, Ordering};
@@ -36,7 +25,8 @@ use crate::ps2::MouseEvent;
 
 // ---- capability registers (offset from mmio base) ----
 const CAP_CAPLENGTH: usize = 0x00;
-const HCSPARAMS1: usize = 0x04; // MaxSlots[7:0] MaxIntrs[15:8] MaxPorts[23:16]
+const HCSPARAMS1: usize = 0x04; // MaxSlots[7:0] MaxIntrs[15:8] MaxPorts[31:24]
+const HCSPARAMS2: usize = 0x08; // scratchpad buffers, ERST max
 const HCCPARAMS1: usize = 0x10; // bit0=64-bit addressing; bit2=64-byte contexts
 const DBOFF: usize = 0x14;
 const RTSOFF: usize = 0x18;
@@ -47,7 +37,8 @@ const OP_USBSTS: usize = 0x04;
 const OP_CRCR: usize = 0x18; // 64-bit
 const OP_DCBAAP: usize = 0x30; // 64-bit
 const OP_CONFIG: usize = 0x38;
-const OP_PORTSC: usize = 0x400; // PORTSC port 1 (port set stride 0x10)
+const OP_PORTSC: usize = 0x400;
+const PORT_STRIDE: usize = 0x10;
 
 // ---- runtime registers (relative to mmio + RTSOFF), interrupter 0 ----
 const RT_IMAN: usize = 0x20;
@@ -67,7 +58,7 @@ const CMD_RING_CYCLE: u32 = 1 << 0;
 const TRB_CYCLE: u32 = 1 << 0;
 const TRB_IOC: u32 = 1 << 5;
 const TRB_IDT: u32 = 1 << 6;
-const TRB_TYPE_BITMASK: u32 = 0xfc00; // bits 10..15
+const TRB_TYPE_BITMASK: u32 = 0xfc00;
 fn TYPE(t: u32) -> u32 {
     t << 10
 }
@@ -148,7 +139,6 @@ fn DEV_SPEED(p: u32) -> u32 {
 }
 
 // ---- USB meta protocol ----
-const USB_DIR_IN: u8 = 0x80;
 const USB_REQ_GET_DESCRIPTOR: u8 = 0x06;
 const USB_REQ_SET_CONFIGURATION: u8 = 0x09;
 const DT_DEVICE: u8 = 0x01;
@@ -156,6 +146,19 @@ const DT_CONFIG: u8 = 0x02;
 const DT_INTERFACE: u8 = 0x04;
 const DT_ENDPOINT: u8 = 0x05;
 const HID_CLASS: u8 = 0x03;
+const HID_SUBCLASS_BOOT: u8 = 0x01;
+const HID_PROTO_KEYBOARD: u8 = 1;
+const HID_PROTO_MOUSE: u8 = 2;
+const REQ_SET_PROTOCOL: u8 = 0x0B;
+const REQ_SET_IDLE: u8 = 0x0A;
+
+const MAX_XDEV: usize = 4;
+
+#[derive(Clone, Copy, PartialEq)]
+enum Kind {
+    Mouse,
+    Keyboard,
+}
 
 // ---------------------------------------------------------------------------
 // State (cooperative single-task kernel → unsafe statics)
@@ -166,12 +169,14 @@ static mut DBB: usize = 0; // doorbell base
 static mut RTB: usize = 0; // runtime base
 static mut CTRL_CS: usize = 32; // context size
 static mut MAXPORTS: u32 = 0;
+static mut N_SLOTS: u32 = 1; // MaxSlotsEn we program
 
+#[derive(Clone, Copy)]
 struct Ring {
     raw: *mut u32,
-    count: usize, // TRBs
-    idx: usize,   // producer (command/transfer) or consumer (event) index
-    cyc: u32,     // software cycle
+    count: usize,
+    idx: usize,
+    cyc: u32,
 }
 const fn ri_empty(count: usize) -> Ring {
     Ring {
@@ -182,29 +187,60 @@ const fn ri_empty(count: usize) -> Ring {
     }
 }
 static mut CMD: Ring = ri_empty(64);
-static mut EP0: Ring = ri_empty(8);
-static mut EPIN: Ring = ri_empty(8);
 static mut EV: Ring = ri_empty(32);
 
 static mut DCBAA: *mut u64 = core::ptr::null_mut();
-static mut DEVCTX: *mut u8 = core::ptr::null_mut();
 static mut INCTX: *mut u8 = core::ptr::null_mut();
 static mut ERST: *mut u8 = core::ptr::null_mut();
+static mut SPB_ARRAY: *mut u64 = core::ptr::null_mut();
+static mut NUM_SPB: u32 = 0;
+/// Report buffers (16 bytes per device) in one DMA page.
+static mut REPORTS: *mut u8 = core::ptr::null_mut();
 
-static SLOT: AtomicU32 = AtomicU32::new(0);
-static HID_EPID: AtomicU32 = AtomicU32::new(0); // device-context index of the IN endpoint
-static mut HID_BUF: [u8; 8] = [0; 8];
-static mut HID_LEN: usize = 4;
-static mut MPS0: u32 = 8;
-
-static mut GOT_REPORT: bool = false;
-static mut QUEUED: bool = false;
-static mut RBTN: u8 = 0;
-static mut RDX: i8 = 0;
-static mut RDY: i8 = 0;
+/// One enumerated HID device (slot + interrupt-IN endpoint).
+#[derive(Clone, Copy)]
+struct XDev {
+    slot: u32,
+    dci: u32, // endpoint context index of the IN endpoint
+    ep0: Ring,
+    epin: Ring,
+    devctx: *mut u8,
+    kind: Kind,
+    len: u32,
+    kbd: crate::hid::Keyboard,
+    armed: bool,
+    ready: bool,
+    code: u32,
+    reports: u32,
+    failures: u32,
+}
+impl XDev {
+    const EMPTY: XDev = XDev {
+        slot: 0,
+        dci: 0,
+        ep0: ri_empty(8),
+        epin: ri_empty(8),
+        devctx: core::ptr::null_mut(),
+        kind: Kind::Mouse,
+        len: 0,
+        kbd: crate::hid::Keyboard::new(),
+        armed: false,
+        ready: false,
+        code: 0,
+        reports: 0,
+        failures: 0,
+    };
+}
+static mut XDEVS: [XDev; MAX_XDEV] = [XDev::EMPTY; MAX_XDEV];
+static NXDEV: AtomicU32 = AtomicU32::new(0);
 
 static INITOK: AtomicBool = AtomicBool::new(false);
-static MOUSEPRESENT: AtomicBool = AtomicBool::new(false);
+static DISCOVERED_TRIED: AtomicU32 = AtomicU32::new(0);
+static LAST_SCAN: AtomicU32 = AtomicU32::new(0);
+
+unsafe fn dev_mut(i: usize) -> &'static mut XDev {
+    unsafe { &mut *core::ptr::addr_of_mut!(XDEVS[i]) }
+}
 
 fn cs() -> usize {
     unsafe { CTRL_CS }
@@ -245,14 +281,17 @@ fn rt(off: usize) -> usize {
     unsafe { RTB + off }
 }
 
-fn dma_alloc(bytes: usize, align: usize) -> *mut u8 {
-    unsafe {
-        let layout = alloc::alloc::Layout::from_size_align(bytes.max(1), align).unwrap();
-        let p = alloc::alloc::alloc(layout);
-        if !p.is_null() {
-            core::ptr::write_bytes(p, 0, bytes.max(1));
+/// DMA page(s) from the page allocator (physical == CPU address).  Always
+/// 4 KiB aligned, which satisfies every xHCI ring/context alignment rule.
+fn dma_alloc(bytes: usize, _align: usize) -> *mut u8 {
+    let pages = bytes.div_ceil(4096).max(1);
+    match crate::memory::alloc_pages_above(pages, 0x100_0000) {
+        Some(p) => {
+            let ptr = p as *mut u8;
+            unsafe { core::ptr::write_bytes(ptr, 0, pages * 4096) };
+            ptr
         }
-        p
+        None => core::ptr::null_mut(),
     }
 }
 
@@ -273,10 +312,8 @@ unsafe fn trb_push(r: &mut Ring, d0: u32, d1: u32, d2: u32, d3: u32) {
     }
 }
 
-// ---------------------------------------------------------------------------
-// Event ring: drain produced events, calling handle(d0,d1,d2,d3) per event,
-// then advance ERDP to the consumption point so the controller reuses it.
-// ---------------------------------------------------------------------------
+/// Drain produced events, calling `handle(d0,d1,d2,d3)` per event, then
+/// advance ERDP so the controller can reuse the segment entries.
 unsafe fn drain_events(handle: &mut dyn FnMut(u32, u32, u32, u32)) {
     unsafe {
         let ev = &mut *core::ptr::addr_of_mut!(EV);
@@ -309,18 +346,16 @@ fn wait_cmd() -> (u32, u32) {
     let mut slot = 0u32;
     let mut done = false;
     let mut seen = 0u32;
-    let mut raw: [u32; 4] = [0; 4];
     for _ in 0..4_000_000 {
         if done {
             break;
         }
         unsafe {
-            drain_events(&mut |d0, d1, d2, d3| {
+            drain_events(&mut |_d0, _d1, d2, d3| {
                 seen += 1;
                 if TRB_FIELD_TO_TYPE(d3) == TRB_EV_CMD_COMPLETE {
                     code = GET_COMP_CODE(d2);
                     slot = TRB_TO_SLOT_ID(d3);
-                    raw = [d0, d1, d2, d3];
                     done = true;
                 }
             });
@@ -328,31 +363,20 @@ fn wait_cmd() -> (u32, u32) {
         core::hint::spin_loop();
     }
     if code != COMP_SUCCESS {
-        unsafe {
-            let evidx = core::ptr::addr_of!(EV).cast::<Ring>().read().idx;
-            crate::serial_println!(
-                "usb: cmd-complete code={} slot={} seen={} evidx={}",
-                code, slot, seen, evidx
-            );
-            for i in 0..4 {
-                let s = EV.raw.add(i * 4);
-                crate::serial_println!(
-                    "  ev[{i}] {:08x} {:08x} {:08x} {:08x}",
-                    s.add(0).read(), s.add(1).read(), s.add(2).read(), s.add(3).read()
-                );
-            }
-        }
+        crate::serial_println!(
+            "usb: cmd-complete code={} slot={} seen={}",
+            code,
+            slot,
+            seen
+        );
     }
     (code, slot)
 }
 
-/// Enqueue a command TRB and ring the host command doorbell.
 fn issue_cmd(d3: u32) -> (u32, u32) {
     issue_cmd_ptr(d3, 0)
 }
 
-/// Enqueue a command TRB that carries a 64-bit parameter (used by
-/// Address-Device / Configure-Endpoint for the input-context pointer).
 fn issue_cmd_ptr(d3: u32, ptr: usize) -> (u32, u32) {
     unsafe {
         trb_push(
@@ -368,19 +392,39 @@ fn issue_cmd_ptr(d3: u32, ptr: usize) -> (u32, u32) {
     wait_cmd()
 }
 
-/// Wait for a transfer-complete event on the given device-context endpoint id.
-fn wait_xfer(epid: u32) -> u32 {
-    let mut code = 0u32;
+/// Record a transfer event against the device it belongs to.  Called for
+/// *every* transfer event, including ones drained while waiting for a
+/// control transfer: dropping them would lose an armed device's report and
+/// stall that device forever.
+fn route_event(d2: u32, d3: u32) {
+    if TRB_FIELD_TO_TYPE(d3) != TRB_EV_TRANSFER {
+        return;
+    }
+    let slot = TRB_TO_SLOT_ID(d3);
+    let epid = TRB_TO_EP_ID(d3);
+    let n = NXDEV.load(Ordering::Relaxed) as usize;
+    for i in 0..n {
+        let d = unsafe { dev_mut(i) };
+        if d.slot == slot && d.dci == epid {
+            d.ready = true;
+            d.code = GET_COMP_CODE(d2);
+        }
+    }
+}
+
+/// Wait for a transfer-complete event on (slot, endpoint id).
+fn wait_xfer(slot: u32, epid: u32) -> u32 {
+    let mut code = u32::MAX;
     let mut done = false;
-    let sl = SLOT.load(Ordering::Relaxed);
     for _ in 0..4_000_000 {
         if done {
             break;
         }
         unsafe {
-            drain_events(&mut |_, _, d2, d3| {
+            drain_events(&mut |_d0, _d1, d2, d3| {
+                route_event(d2, d3);
                 if TRB_FIELD_TO_TYPE(d3) == TRB_EV_TRANSFER
-                    && TRB_TO_SLOT_ID(d3) == sl
+                    && TRB_TO_SLOT_ID(d3) == slot
                     && TRB_TO_EP_ID(d3) == epid
                 {
                     code = GET_COMP_CODE(d2);
@@ -390,14 +434,14 @@ fn wait_xfer(epid: u32) -> u32 {
         }
         core::hint::spin_loop();
     }
-    code
+    if !done { 0 } else { code }
 }
 
 // ---------------------------------------------------------------------------
-// Control transfer on EP0 (setup + optional data + status), per Linux
-// xhci_queue_ctrl_tx.  Returns OK(len) or the failure completion code.
+// Control transfer on a device's EP0 (setup + optional data + status).
 // ---------------------------------------------------------------------------
 fn ctrl_xfer(
+    di: usize,
     bmreq: u8,
     breq: u8,
     wval: u16,
@@ -406,10 +450,10 @@ fn ctrl_xfer(
     buflen: u32,
     dir_in: bool,
 ) -> Result<usize, u32> {
-    let slot = SLOT.load(Ordering::Relaxed);
-    let e0 = unsafe { &mut *core::ptr::addr_of_mut!(EP0) };
     unsafe {
-        // Setup stage (immediate data, 8-byte setup packet in d0/d1).
+        let d = dev_mut(di);
+        let slot = d.slot;
+        let e0 = &mut d.ep0;
         let tx = if buflen > 0 {
             TRB_TX_TYPE(if dir_in { TRB_DATA_IN } else { TRB_DATA_OUT })
         } else {
@@ -431,13 +475,12 @@ fn ctrl_xfer(
                 TYPE(TRB_DATA) | if dir_in { TRB_DIR_IN } else { 0 },
             );
         }
-        // Status stage.
         let stat_dir = if buflen > 0 && dir_in { 0 } else { TRB_DIR_IN };
         trb_push(e0, 0, 0, 0, TYPE(TRB_STATUS) | TRB_IOC | stat_dir);
-        wr(db(slot), 1); // ring EP0 (doorbell target value 1)
+        wr(db(slot), 1);
         let _ = rd(db(slot));
     }
-    let code = wait_xfer(1);
+    let code = wait_xfer(unsafe { XDEVS[di].slot }, 1);
     if code == COMP_SUCCESS || code == COMP_SHORT {
         Ok(buflen as usize)
     } else {
@@ -446,9 +489,9 @@ fn ctrl_xfer(
 }
 
 // ---------------------------------------------------------------------------
-// Context construction helpers
+// Context construction
 // ---------------------------------------------------------------------------
-fn build_addr_ictx(port_speed: u32, port_num: u32) {
+fn build_addr_ictx(port_speed: u32, port_num: u32, ep0: *mut u32, mps0: u32) {
     unsafe {
         core::ptr::write_bytes(INCTX, 0, 32 + 8 * cs());
         let cc = INCTX as *mut u32;
@@ -459,8 +502,8 @@ fn build_addr_ictx(port_speed: u32, port_num: u32) {
         *sc.add(1) = ROOTPORT(port_num + 1);
         let e0 = INCTX.add(32 + cs()) as *mut u32;
         *e0.add(0) = 0;
-        *e0.add(1) = EP_TYPE(CTRL_EP) | MAX_PACKET(MPS0) | ERROR_COUNT(3);
-        (INCTX.add(32 + cs() + 8) as *mut u64).write_volatile((EP0.raw as u64) | 1);
+        *e0.add(1) = EP_TYPE(CTRL_EP) | MAX_PACKET(mps0) | ERROR_COUNT(3);
+        (INCTX.add(32 + cs() + 8) as *mut u64).write_volatile((ep0 as u64) | 1);
         (INCTX.add(32 + cs() + 16) as *mut u32).write_volatile(EP_AVG(8));
     }
 }
@@ -470,13 +513,11 @@ fn build_addr_ictx(port_speed: u32, port_num: u32) {
 // ---------------------------------------------------------------------------
 unsafe fn reset_port(port: u32) -> Result<u32, ()> {
     unsafe {
-        let psc = op(OP_PORTSC) + (port as usize) * 0x10;
-        // Ensure powered.
+        let psc = op(OP_PORTSC) + (port as usize) * PORT_STRIDE;
         wr(psc, rd(psc) | PORT_POWER);
         for _ in 0..1000 {
             core::hint::spin_loop();
         }
-        // Assert reset, wait for the hardware to clear it (reset done).
         wr(psc, rd(psc) | PORT_RESET);
         for _ in 0..4_000_000 {
             if rd(psc) & PORT_RESET == 0 {
@@ -484,7 +525,6 @@ unsafe fn reset_port(port: u32) -> Result<u32, ()> {
             }
             core::hint::spin_loop();
         }
-        // Wait for the port to be enabled.
         for _ in 0..4_000_000 {
             if rd(psc) & PORT_PE != 0 {
                 break;
@@ -499,213 +539,253 @@ unsafe fn reset_port(port: u32) -> Result<u32, ()> {
 }
 
 // ---------------------------------------------------------------------------
-// Enumeration + HID setup
+// Descriptor parsing: first boot HID mouse/keyboard interface
 // ---------------------------------------------------------------------------
-fn read_device_descriptor() -> Result<[u8; 18], u32> {
-    let mut d = [0u8; 18];
-    ctrl_xfer(
-        USB_DIR_IN,
-        USB_REQ_GET_DESCRIPTOR,
-        (DT_DEVICE as u16) << 8,
-        0,
-        d.as_mut_ptr(),
-        18,
-        true,
-    )?;
-    Ok(d)
-}
-
-fn read_config_descriptor(buf: &mut [u8]) -> Result<usize, u32> {
-    let total = ctrl_xfer(
-        USB_DIR_IN,
-        USB_REQ_GET_DESCRIPTOR,
-        (DT_CONFIG as u16) << 8,
-        0,
-        buf.as_mut_ptr(),
-        buf.len() as u32,
-        true,
-    )?;
-    Ok(total)
-}
-
-/// Walk a config descriptor tree looking for a HID interrupt IN endpoint
-/// with an interrupt (transfer type 3).  Returns (mps, interval, eepid).
-fn find_hid_mouse(cfg: &[u8]) -> Option<(u32, u32, u32)> {
+/// Returns (kind, interface number, ep address, mps, interval).
+fn find_hid(cfg: &[u8]) -> Option<(Kind, u8, u8, u32, u32)> {
     let mut i = 0usize;
-    let mut in_iface = false;
+    let mut cand: Option<(Kind, u8)> = None;
     while i + 2 <= cfg.len() {
-        let bLen = cfg[i] as usize;
-        if bLen == 0 {
+        let blen = cfg[i] as usize;
+        if blen < 2 {
             break;
         }
-        let btype = cfg[i + 1];
-        if btype == DT_INTERFACE && i + 9 <= cfg.len() {
-            // interface descriptor: +5 = bInterfaceClass
-            in_iface = cfg[i + 5] == HID_CLASS;
-        } else if btype == DT_ENDPOINT && in_iface && i + 7 <= cfg.len() {
-            let addr = cfg[i + 2];
-            let attrs = cfg[i + 3];
-            let dir_in = addr & 0x80 != 0;
-            let xfer = attrs & 0x03;
-            if dir_in && xfer == 3 {
+        let bt = cfg[i + 1];
+        if bt == DT_INTERFACE && blen >= 9 && i + 9 <= cfg.len() {
+            let iface = cfg[i + 2];
+            let class = cfg[i + 5];
+            let sub = cfg[i + 6];
+            let proto = cfg[i + 7];
+            cand = if class == HID_CLASS && sub == HID_SUBCLASS_BOOT {
+                match proto {
+                    HID_PROTO_KEYBOARD => Some((Kind::Keyboard, iface)),
+                    HID_PROTO_MOUSE => Some((Kind::Mouse, iface)),
+                    _ => None,
+                }
+            } else {
+                None
+            };
+        } else if bt == DT_ENDPOINT && blen >= 7 && i + 7 <= cfg.len() {
+            if let Some((kind, iface)) = cand {
+                let addr = cfg[i + 2];
+                let attrs = cfg[i + 3];
                 let mps = (cfg[i + 4] as u32) | ((cfg[i + 5] as u32) << 8);
                 let interval = cfg[i + 6] as u32;
-                let epnum = (addr & 0x0f) as u32;
-                let euid = 2 * epnum + 1; // device-context index (IN)
-                return Some((mps, interval, euid));
+                if (addr & 0x80) != 0 && (attrs & 0x03) == 0x03 {
+                    return Some((kind, iface, addr, mps, interval));
+                }
             }
         }
-        i += bLen;
+        i += blen;
     }
     None
 }
 
 // ---------------------------------------------------------------------------
-// Bring one port's device through address + configure to an HID mouse
+// Enumeration: slot + address + descriptors + configure IN endpoint
 // ---------------------------------------------------------------------------
 fn enumerate(port: u32, speed: u32) -> bool {
-    let _ = DEV_SPEED;
-    // 1. Enable slot.
-    let (code, slotid) = issue_cmd(TYPE(TRB_ENABLE_SLOT));
-    if code != COMP_SUCCESS || slotid == 0 {
-        crate::serial_println!("usb: enable-slot failed code={} slot={}", code, slotid);
+    let idx = NXDEV.load(Ordering::Relaxed) as usize;
+    if idx >= MAX_XDEV {
+        return false;
+    }
+    // Per-device rings + output device context.
+    let ep0_raw = dma_alloc(8 * 16, 64) as *mut u32;
+    let epin_raw = dma_alloc(8 * 16, 64) as *mut u32;
+    let devctx = dma_alloc((1 + 8) * cs(), 64);
+    let report = unsafe { REPORTS.add(idx * 16) };
+    if ep0_raw.is_null() || epin_raw.is_null() || devctx.is_null() || report.is_null() {
+        crate::serial_println!("usb: DMA allocation failed");
+        return false;
+    }
+
+    // Enable slot.
+    let (code, slot) = issue_cmd(TYPE(TRB_ENABLE_SLOT));
+    if code != COMP_SUCCESS || slot == 0 {
+        crate::serial_println!("usb: enable-slot failed code={}", code);
         return false;
     }
-    SLOT.store(slotid, Ordering::Relaxed);
     unsafe {
-        *DCBAA.add(slotid as usize) = DEVCTX as u64;
+        *DCBAA.add(slot as usize) = devctx as u64;
+        let d = dev_mut(idx);
+        d.slot = slot;
+        d.ep0 = Ring {
+            raw: ep0_raw,
+            count: 8,
+            idx: 0,
+            cyc: 1,
+        };
+        d.epin = Ring {
+            raw: epin_raw,
+            count: 8,
+            idx: 0,
+            cyc: 1,
+        };
+        d.devctx = devctx;
+        d.armed = false;
+        d.ready = false;
+        d.code = 0;
+        d.reports = 0;
+        d.failures = 0;
+        d.kbd = crate::hid::Keyboard::new();
     }
-    // 2. Address device.
-    build_addr_ictx(speed, port);
-    let (code, _) = issue_cmd_ptr(TYPE(TRB_ADDR_DEV) | SLOT_ID(slotid), unsafe { INCTX } as usize);
+
+    // Address device with EP0 max packet 8 (first transactions are tiny).
+    build_addr_ictx(speed, port, ep0_raw, 8);
+    let (code, _) = issue_cmd_ptr(TYPE(TRB_ADDR_DEV) | SLOT_ID(slot), unsafe {
+        INCTX as usize
+    });
     if code != COMP_SUCCESS {
         crate::serial_println!("usb: address-device failed code={}", code);
         return false;
     }
-    // 3. Read the device descriptor (picks up bMaxPacketSize0).
-    match read_device_descriptor() {
+
+    // Device descriptor.
+    match read_device_descriptor(idx) {
         Ok(d) => {
-            let mps = d[7];
-            if mps != 0 {
-                unsafe {
-                    MPS0 = mps as u32;
-                }
-            }
             crate::serial_println!(
-                "usb: dev {:04x}:{:04x} class={:#x} mps0={}",
+                "usb: dev {:04x}:{:04x} class={:#x} mps0={} slot={}",
                 u16::from_le_bytes([d[8], d[9]]),
                 u16::from_le_bytes([d[10], d[11]]),
                 d[4],
-                d[7]
+                d[7],
+                slot
             );
         }
-        Err(c) => crate::serial_println!("usb: get-device-desc failed code={}", c),
+        Err(c) => {
+            crate::serial_println!("usb: get-device-desc failed code={}", c);
+            return false;
+        }
     }
-    // 4. Read the whole config descriptor and find the HID mouse IN ep.
-    let mut cfg = [0u8; 256];
-    let got = match read_config_descriptor(&mut cfg) {
+
+    // Config descriptor -> HID interface.
+    let mut cfg = [0u8; 512];
+    let got = match read_config_descriptor(idx, &mut cfg) {
         Ok(n) => n,
         Err(c) => {
             crate::serial_println!("usb: get-config failed code={}", c);
             return false;
         }
     };
-    let _ = got;
-    match find_hid_mouse(&cfg) {
-        Some((mps, interval, euid)) => {
-            crate::serial_println!(
-                "usb: HID mouse IN ep mps={} interval={} euid={}",
-                mps,
-                interval,
-                euid
-            );
-            // 5. Set configuration = 1 (first config value present).
-            let cfgval = cfg[5];
-            if cfgval == 0 {
-                return false;
-            }
-            if read_configuration(cfgval).is_err() {
-                return false;
-            }
-            // 6. Configure the IN endpoint.
-            if !configure_hid_endpoint(euid, mps, interval) {
-                return false;
-            }
-            HID_EPID.store(euid, Ordering::Relaxed);
-            unsafe {
-                HID_LEN = mps as usize;
-                QUEUED = false;
-            }
-            // 7. Arm the first IN read.
-            queue_report();
-            MOUSEPRESENT.store(true, Ordering::Relaxed);
-            crate::serial_println!(
-                "usb: HID boot mouse armed (slot {})",
-                SLOT.load(Ordering::Relaxed)
-            );
-            true
-        }
-        None => {
-            crate::serial_println!("usb: no HID mouse endpoint in config");
-            false
-        }
+    let Some((kind, iface, _epaddr, mps, interval)) = find_hid(&cfg[..got]) else {
+        crate::serial_println!("usb: no HID mouse/keyboard endpoint in config");
+        return false;
+    };
+    let cfgval = cfg[5];
+    if cfgval == 0 {
+        return false;
     }
+    if ctrl_xfer(idx, 0x00, USB_REQ_SET_CONFIGURATION, cfgval as u16, 0, core::ptr::null_mut(), 0, false)
+        .is_err()
+    {
+        return false;
+    }
+    // Boot protocol: fixed 3-byte mouse / 8-byte keyboard reports.
+    if ctrl_xfer(idx, 0x21, REQ_SET_PROTOCOL, 0, iface as u16, core::ptr::null_mut(), 0, false)
+        .is_err()
+    {
+        crate::serial_println!("usb: set-protocol(boot) stalled (continuing)");
+    }
+    let _ = ctrl_xfer(idx, 0x21, REQ_SET_IDLE, 0, iface as u16, core::ptr::null_mut(), 0, false);
+
+    if !configure_hid_endpoint(idx, _epaddr, mps, interval) {
+        return false;
+    }
+    unsafe {
+        let d = dev_mut(idx);
+        d.kind = kind;
+        d.len = mps.min(8).max(1);
+        d.armed = true;
+    }
+    NXDEV.fetch_add(1, Ordering::Relaxed);
+    queue_report(idx);
+    crate::serial_println!(
+        "usb: HID {} armed (slot {} mps {} interval {})",
+        if kind == Kind::Mouse { "mouse" } else { "keyboard" },
+        slot,
+        mps,
+        interval
+    );
+    true
 }
 
-fn read_configuration(val: u8) -> Result<(), u32> {
+fn read_device_descriptor(di: usize) -> Result<[u8; 18], u32> {
+    let mut d = [0u8; 18];
+    let b = d.as_mut_ptr();
     ctrl_xfer(
+        di,
+        0x80,
+        USB_REQ_GET_DESCRIPTOR,
+        (DT_DEVICE as u16) << 8,
         0,
-        USB_REQ_SET_CONFIGURATION,
-        val.into(),
-        0,
-        core::ptr::null_mut(),
+        b,
+        18,
+        true,
+    )
+    .map(|_| d)
+}
+
+fn read_config_descriptor(di: usize, buf: &mut [u8; 512]) -> Result<usize, u32> {
+    ctrl_xfer(
+        di,
+        0x80,
+        USB_REQ_GET_DESCRIPTOR,
+        (DT_CONFIG as u16) << 8,
         0,
-        false,
+        buf.as_mut_ptr(),
+        buf.len() as u32,
+        true,
     )
-    .map(|_| ())
 }
 
-fn configure_hid_endpoint(euid: u32, mps: u32, interval: u32) -> bool {
+/// Commit the interrupt-IN endpoint context and ring its doorbell.
+fn configure_hid_endpoint(di: usize, epaddr: u8, mps: u32, interval: u32) -> bool {
     unsafe {
-        let slot = SLOT.load(Ordering::Relaxed);
+        let d = dev_mut(di);
         core::ptr::write_bytes(INCTX, 0, 32 + 8 * cs());
         let cc = INCTX as *mut u32;
-        *cc.add(0) = 0; // Drop flags
-        *cc.add(1) = (1 << 0) | (1 << euid); // Add slot + this EP
-        // Input slot context: copy the output (post-address) slot context so
-        // route/speed/rootport are preserved, then bump LAST_CTX to the ep.
-        let osc = DEVCTX.offset(0).cast::<u32>();
+        // Add Context flags are DCI bits: bit0 = slot, bit(2*ep+1) = IN ep.
+        // Endpoint DCI for an IN endpoint is 2 * ep_number + 1 (EP1 IN -> 3).
+        let euid = ((epaddr & 0x0f) as u32) * 2 + 1;
+        *cc.add(0) = 0;
+        *cc.add(1) = (1 << 0) | (1 << euid);
+        d.dci = euid;
+        let osc = d.devctx.cast::<u32>();
         let sc = INCTX.offset(32).cast::<u32>();
         *sc.add(0) = (*osc.add(0) & !LAST_CTX(31)) | LAST_CTX(euid);
         *sc.add(1) = *osc.add(1);
-        // Endpoint IN context.
         let ep = INCTX.add(32 + (euid as usize) * cs()).cast::<u32>();
         *ep.add(0) = EP_INTERVAL(interval);
         *ep.add(1) = EP_TYPE(INT_IN_EP) | MAX_PACKET(mps) | ERROR_COUNT(3);
         *ep.add(4) = EP_AVG(mps);
-        let deq = EPIN.raw as u64 | 1;
+        let deq = d.epin.raw as u64 | 1;
         *ep.offset(2).cast::<u64>() = deq;
+        let slot = d.slot;
         let (code, _) = issue_cmd_ptr(TYPE(TRB_CONFIG_EP) | SLOT_ID(slot), INCTX as usize);
-        code == COMP_SUCCESS
+        if code != COMP_SUCCESS {
+            crate::serial_println!("usb: configure-endpoint failed code={}", code);
+            return false;
+        }
+        true
     }
 }
 
-fn queue_report() {
-    let slot = SLOT.load(Ordering::Relaxed);
-    let epid = HID_EPID.load(Ordering::Relaxed);
-    let epin = unsafe { &mut *core::ptr::addr_of_mut!(EPIN) };
+/// Arm one interrupt-IN transfer on the device's endpoint ring.
+fn queue_report(di: usize) {
     unsafe {
-        let b = core::ptr::addr_of_mut!(HID_BUF) as *mut u8;
+        let d = dev_mut(di);
+        let slot = d.slot;
+        let epid = d.dci;
         trb_push(
-            epin,
-            b as u32,
-            (b as usize >> 32) as u32,
-            TRB_LEN(HID_LEN as u32),
+            &mut d.epin,
+            REPORTS.add(di * 16) as u32,
+            (REPORTS.add(di * 16) as usize >> 32) as u32,
+            TRB_LEN(d.len),
             TYPE(TRB_NORMAL) | TRB_IOC,
         );
+        d.ready = false;
         wr(db(slot), epid);
         let _ = rd(db(slot));
-        QUEUED = true;
     }
 }
 
@@ -728,7 +808,6 @@ pub fn init() -> bool {
     };
     crate::serial_println!("usb: xHCI at {:02x}:{:02x}.{}", b, dev, func);
 
-    // Enable MMIO + bus-mastering on the PCI command register.
     unsafe {
         let cmd = crate::pci::config_read(b, dev, func, 0x04);
         crate::pci::config_write(
@@ -736,48 +815,55 @@ pub fn init() -> bool {
             dev,
             func,
             0x04,
-            // Memory space + bus master + INTx disable (bit 10).  The
-            // driver is polled (event ring drained from the cursor task),
-            // so the controller must never assert INTx: an unhandled
-            // xHCI interrupt would hit the kernel's catch-all handler
-            // and (being level-triggered) re-fire forever on real
-            // hardware.
             (cmd & 0xFFFF_0000) | ((cmd & 0xFFFF) | 0x6 | 0x400),
         );
         let bar = crate::pci::bar64(b, dev, func, 0);
-        let base = bar.unwrap() as usize;
+        let base = bar.unwrap_or(0) as usize;
+        if base == 0 {
+            crate::serial_println!("usb: bad BAR");
+            return false;
+        }
         M = base;
         let caplen = rd(base + CAP_CAPLENGTH);
         let hcs1 = rd(base + HCSPARAMS1);
+        let hcs2 = rd(base + HCSPARAMS2);
         let hcc = rd(base + HCCPARAMS1);
-        crate::serial_println!(
-            "usb: mmio={:#x} caplen={:#x} hcs1={:#x} hcc1={:#x}",
-            base, caplen, hcs1, hcc
-        );
         OPB = base + (caplen & 0xff) as usize;
         DBB = base + rd(base + DBOFF) as usize;
         RTB = base + rd(base + RTSOFF) as usize;
-        MAXPORTS = (hcs1 >> 24) & 0xff; // MaxPorts is bits 31:24 (bits 23:16 are MaxIntrs)
-        let _ = hcc;
+        MAXPORTS = (hcs1 >> 24) & 0xff;
+        N_SLOTS = (hcs1 & 0xff).clamp(1, MAX_XDEV as u32);
         CTRL_CS = if hcc & (1 << 2) != 0 { 64 } else { 32 };
+        // HCSPARAMS2: Max Scratchpad Buffers = Hi[31:27] << 5 | Lo[25:21].
+        NUM_SPB = (((hcs2 >> 27) & 0x1f) << 5) | ((hcs2 >> 21) & 0x1f);
+        crate::serial_println!(
+            "usb: mmio={:#x} caplen={:#x} hcs1={:#x} hcs2={:#x} hcc1={:#x}",
+            base,
+            caplen,
+            hcs1,
+            hcs2,
+            hcc
+        );
+        let (maxports, nslots, csz, nspb) = (MAXPORTS, N_SLOTS, CTRL_CS, NUM_SPB);
+        crate::serial_println!(
+            "usb: ports={} slots={} context={}B scratchpad={}",
+            maxports,
+            nslots,
+            csz,
+            nspb
+        );
 
-        // Claim USB Legacy ownership from the BIOS before resetting the
-        // controller (same SMM-legacy rationale as the EHCI driver).
         match crate::pci::claim_usb_legacy(b, dev, func, base, 0x04, true) {
-            0 => crate::serial_println!("usb: xHCI USB legacy: already released (no claim needed)"),
+            0 => crate::serial_println!("usb: xHCI USB legacy: already released"),
             1 => crate::serial_println!("usb: xHCI USB legacy handed over by BIOS"),
-            2 => crate::serial_println!("usb: xHCI USB legacy claim timed out (BIOS sem stuck)"),
-            _ => crate::serial_println!("usb: xHCI USB legacy: no capability found"),
+            2 => crate::serial_println!("usb: xHCI USB legacy claim timed out"),
+            _ => crate::serial_println!("usb: xHCI USB legacy: no capability"),
         }
     }
 
-    // Reset the controller (stop, then reset).  Wait for reset completion
-    // on the CMD_RESET bit clearing (per Linux xhci_reset), then for the
-    // Controller-Not-Ready flag to clear before any doorbell/op-register
-    // activity.
+    // Reset: stop, then RESET, then wait for CNR to clear.
     unsafe {
         wr(op(OP_USBCMD), rd(op(OP_USBCMD)) & !CMD_RUN);
-        // Wait for halt.
         let mut t = 0;
         while t < 1_000_000 && rd(op(OP_USBSTS)) & STS_HALT == 0 {
             t += 1;
@@ -796,42 +882,80 @@ pub fn init() -> bool {
         }
     }
 
-    // Allocate rings / contexts.
+    // Rings, contexts, scratchpads.
     unsafe {
-        CMD.raw = dma_alloc(64 * 16, 64) as *mut u32;
-        EV.raw = dma_alloc(32 * 16, 64) as *mut u32;
-        EP0.raw = dma_alloc(8 * 16, 64) as *mut u32;
-        EPIN.raw = dma_alloc(8 * 16, 64) as *mut u32;
-        ERST = dma_alloc(16, 64);
-        DCBAA = dma_alloc(64 * 8, 64) as *mut u64;
-        DEVCTX = dma_alloc((1 + 8) * cs(), 64);
-        INCTX = dma_alloc(32 + 8 * cs(), 64);
-        // ERST segment 0 -> event ring.
+        let cmd_ring = dma_alloc(64 * 16, 64) as *mut u32;
+        let ev_ring = dma_alloc(32 * 16, 64) as *mut u32;
+        let erst = dma_alloc(16, 64);
+        let dcb = dma_alloc((N_SLOTS as usize + 1) * 8, 64) as *mut u64;
+        let ictx = dma_alloc(32 + 8 * cs(), 64);
+        let reports = dma_alloc(4096, 64);
+        if cmd_ring.is_null()
+            || ev_ring.is_null()
+            || erst.is_null()
+            || dcb.is_null()
+            || ictx.is_null()
+            || reports.is_null()
+        {
+            crate::serial_println!("usb: DMA allocation failed");
+            return false;
+        }
+        CMD = Ring {
+            raw: cmd_ring,
+            count: 64,
+            idx: 0,
+            cyc: 1,
+        };
+        EV = Ring {
+            raw: ev_ring,
+            count: 32,
+            idx: 0,
+            cyc: 1,
+        };
+        ERST = erst;
+        DCBAA = dcb;
+        INCTX = ictx;
+        REPORTS = reports;
+
+        // Scratchpad buffers (§4.20): a 64-byte array of buffer addresses
+        // whose entries the controller requires to exist when HCSPARAMS2
+        // reports them.  DCBAAP[0] must point at that array.
+        if NUM_SPB > 0 {
+            let arr = dma_alloc(NUM_SPB as usize * 8 + 64, 64) as *mut u64;
+            if !arr.is_null() {
+                for i in 0..NUM_SPB as usize {
+                    let p = dma_alloc(4096, 64);
+                    if p.is_null() {
+                        crate::serial_println!("usb: scratchpad buffer {} failed", i);
+                        return false;
+                    }
+                    arr.add(i).write_volatile(p as u64);
+                }
+                SPB_ARRAY = arr;
+                *DCBAA.add(0) = arr as u64;
+                let nspb = NUM_SPB;
+                crate::serial_println!("usb: allocated {} scratchpad buffer(s)", nspb);
+            }
+        }
+
         let e = ERST as *mut u32;
-        *e.add(0) = EV.raw as u32;
-        *e.add(1) = (EV.raw as usize >> 32) as u32;
-        *e.add(2) = 32; // segment size (TRBs)
+        *e.add(0) = ev_ring as u32;
+        *e.add(1) = (ev_ring as usize >> 32) as u32;
+        *e.add(2) = 32;
     }
 
-    // Program registers.
     unsafe {
         wr64(op(OP_DCBAAP), DCBAA as u64);
         wr64(
             op(OP_CRCR),
             ((CMD.raw as u64) & !0x3f) | CMD_RING_CYCLE as u64,
         );
-        // MaxSlotsEn = 1 (we use a single slot).
-        wr(op(OP_CONFIG), (rd(op(OP_CONFIG)) & !0xff) | 1);
-        // Event ring.
+        wr(op(OP_CONFIG), (rd(op(OP_CONFIG)) & !0xff) | N_SLOTS);
         wr(rt(RT_ERSTSZ), 1);
         wr64(rt(RT_ERSTBA), ERST as u64);
         wr64(rt(RT_ERDP), EV.raw as u64);
-        // Interrupter 0: keep the interrupt DISABLED (IE=0, clear any
-        // pending IP) — the driver is polled and the controller must not
-        // assert INTx (see the PCI command-register comment above).
         wr(rt(RT_IMAN), (rd(rt(RT_IMAN)) & !0x2) | 0x1);
         wr(rt(RT_IMOD), 0);
-        // Run.
         wr(op(OP_USBCMD), rd(op(OP_USBCMD)) | CMD_RUN);
         let mut t = 0;
         while t < 4_000_000 && rd(op(OP_USBSTS)) & STS_HALT != 0 {
@@ -845,36 +969,38 @@ pub fn init() -> bool {
     }
     INITOK.store(true, Ordering::Relaxed);
     crate::serial_println!(
-        "usb: controller up ({} ports) — mouse discovery deferred to cursor task",
+        "usb: controller up ({} ports) — HID discovery deferred to polling",
         unsafe { MAXPORTS }
     );
     true
 }
 
-/// Bitmask of root ports whose attempt already failed; skip them so we don't
-/// re-reset the port on every poll.
-static DISCOVERED_TRIED: AtomicU32 = AtomicU32::new(0);
-
-/// Periodic, cheap scan called from the cursor task until a HID mouse is
-/// found.  A USB device can finish connecting long after the controller
-/// starts, so discovery is driven asynchronously rather than blocking boot.
+/// Scan ports until all slots are used or every port has been tried.
 fn maybe_discover() {
-    if MOUSEPRESENT.load(Ordering::Relaxed) {
+    if !INITOK.load(Ordering::Relaxed) {
         return;
     }
+    if NXDEV.load(Ordering::Relaxed) as usize >= MAX_XDEV {
+        return;
+    }
+    let now = crate::interrupts::uptime_ms() as u32;
+    if now.wrapping_sub(LAST_SCAN.load(Ordering::Relaxed)) < 200 {
+        return;
+    }
+    LAST_SCAN.store(now, Ordering::Relaxed);
     let maxports = unsafe { MAXPORTS };
     for p in 0..maxports {
         if DISCOVERED_TRIED.load(Ordering::Relaxed) & (1 << p) != 0 {
             continue;
         }
         unsafe {
-            let psc = op(OP_PORTSC) + (p as usize) * 0x10;
-            // Only touch ports that are both connected and not yet enabled.
+            let psc = op(OP_PORTSC) + (p as usize) * PORT_STRIDE;
             if rd(psc) & (PORT_CONNECT | PORT_PE) == PORT_CONNECT {
                 crate::serial_println!("usb: port {} connect (speed {})", p, DEV_SPEED(rd(psc)));
-                if let Ok(speed) = reset_port(p)
-                    && enumerate(p, speed) {
-                    return;
+                if let Ok(speed) = reset_port(p) {
+                    if enumerate(p, speed) {
+                        continue; // keep scanning for the other HID device
+                    }
                 }
                 DISCOVERED_TRIED.fetch_or(1 << p, Ordering::Relaxed);
             }
@@ -882,237 +1008,249 @@ fn maybe_discover() {
     }
 }
 
+fn has_kind(kind: Kind) -> bool {
+    let n = NXDEV.load(Ordering::Relaxed) as usize;
+    for i in 0..n {
+        if unsafe { XDEVS[i].kind } == kind {
+            return true;
+        }
+    }
+    false
+}
+
 pub fn mouse_present() -> bool {
-    MOUSEPRESENT.load(Ordering::Relaxed)
+    has_kind(Kind::Mouse)
 }
 
-/// True once the xHCI controller is brought up (regardless of whether a
-/// device has attached yet).  The cursor task is spawned as long as this is
-/// set so async mouse discovery can run.
 pub fn controller_up() -> bool {
     INITOK.load(Ordering::Relaxed)
 }
 
-/// Reset discovery state (used by the `usb` shell command to rescan).
 pub fn rescan() {
     DISCOVERED_TRIED.store(0, Ordering::Relaxed);
+    LAST_SCAN.store(0, Ordering::Relaxed);
+}
+
+/// Drain completion events into the device table.
+fn pump_reports() {
+    unsafe {
+        drain_events(&mut |_d0, _d1, d2, d3| route_event(d2, d3));
+    }
 }
 
-/// Non-blocking poll: drain the event ring, then decode one mouse report if
-/// one arrived (re-arming the IN transfer so reports keep flowing).
-/// Returns `None` when no report is pending (or no mouse).
 pub fn poll_mouse() -> Option<MouseEvent> {
-    // Drive discovery while no device is attached, then drain reports.
-    if !MOUSEPRESENT.load(Ordering::Relaxed) {
-        if INITOK.load(Ordering::Relaxed) {
-            maybe_discover();
+    if !INITOK.load(Ordering::Relaxed) {
+        return None;
+    }
+    if !has_kind(Kind::Mouse) {
+        maybe_discover();
+    }
+    pump_reports();
+    let n = NXDEV.load(Ordering::Relaxed) as usize;
+    for i in 0..n {
+        let d = unsafe { dev_mut(i) };
+        if d.kind != Kind::Mouse || !d.armed || !d.ready {
+            continue;
+        }
+        if d.code != COMP_SUCCESS && d.code != COMP_SHORT {
+            d.armed = false;
+            d.failures += 1;
+            d.ready = false;
+            continue;
+        }
+        let hb = unsafe { REPORTS.add(i * 16) };
+        let (b0, b1, b2) = unsafe {
+            (
+                core::ptr::read_volatile(hb),
+                core::ptr::read_volatile(hb.add(1)),
+                core::ptr::read_volatile(hb.add(2)),
+            )
+        };
+        let ev = MouseEvent {
+            dx: (b1 as i8) as i16,
+            dy: (b2 as i8) as i16,
+            left: b0 & 0x01 != 0,
+            right: b0 & 0x02 != 0,
+            middle: b0 & 0x04 != 0,
+        };
+        d.reports += 1;
+        if d.reports == 1 {
+            crate::serial_println!("usb: mouse report dx={} dy={} b={:#x}", ev.dx, ev.dy, b0);
         }
+        queue_report(i);
+        return Some(ev);
+    }
+    None
+}
+
+pub fn try_poll_key() -> Option<crate::ps2::KeyEvent> {
+    if !INITOK.load(Ordering::Relaxed) {
         return None;
     }
-    let epid = HID_EPID.load(Ordering::Relaxed);
-    let sl = SLOT.load(Ordering::Relaxed);
-    unsafe {
-        drain_events(&mut |_, _, d2, d3| {
-            if TRB_FIELD_TO_TYPE(d3) == TRB_EV_TRANSFER
-                && TRB_TO_SLOT_ID(d3) == sl
-                && TRB_TO_EP_ID(d3) == epid
-            {
-                let c = GET_COMP_CODE(d2);
-                if c == COMP_SUCCESS || c == COMP_SHORT {
-                    // A fresh report sits in HID_BUF.
-                    let p = core::ptr::addr_of!(HID_BUF) as *const u8;
-                    RBTN = core::ptr::read_volatile(p);
-                    RDX = core::ptr::read_volatile(p.add(1)) as i8;
-                    RDY = core::ptr::read_volatile(p.add(2)) as i8;
-                    GOT_REPORT = true;
-                    QUEUED = false;
-                }
-            }
-        });
-        if GOT_REPORT {
-            GOT_REPORT = false;
-            let ev = Some(MouseEvent {
-                dx: RDX as i16,
-                dy: RDY as i16,
-                left: RBTN & 0x01 != 0,
-                right: RBTN & 0x02 != 0,
-                middle: RBTN & 0x04 != 0,
-            });
-            queue_report(); // re-arm for the next report
-            return ev;
+    if !has_kind(Kind::Keyboard) {
+        maybe_discover();
+    }
+    if let Some(b) = crate::hid::pop_byte() {
+        return crate::ps2::feed_scancode(b);
+    }
+    pump_reports();
+    let n = NXDEV.load(Ordering::Relaxed) as usize;
+    for i in 0..n {
+        let d = unsafe { dev_mut(i) };
+        if d.kind != Kind::Keyboard || !d.armed || !d.ready {
+            continue;
+        }
+        if d.code != COMP_SUCCESS && d.code != COMP_SHORT {
+            d.armed = false;
+            d.failures += 1;
+            d.ready = false;
+            continue;
         }
-        if !QUEUED {
-            queue_report(); // keep exactly one IN read outstanding
+        let hb = unsafe { REPORTS.add(i * 16) };
+        let mut rep = [0u8; 8];
+        let len = (d.len as usize).min(8);
+        for (k, b) in rep.iter_mut().take(len).enumerate() {
+            *b = unsafe { core::ptr::read_volatile(hb.add(k)) };
         }
+        d.kbd.diff(&rep);
+        d.reports += 1;
+        if d.reports == 1 {
+            crate::serial_println!(
+                "usb: kbd report mods={:#x} keys={:02x} {:02x} {:02x} {:02x} {:02x} {:02x}",
+                rep[0], rep[2], rep[3], rep[4], rep[5], rep[6], rep[7]
+            );
+        }
+        queue_report(i);
+        return crate::hid::pop_byte().and_then(crate::ps2::feed_scancode);
     }
     None
 }
 
-/// One-line status for the shell's `usb` command.
+// ---------------------------------------------------------------------------
+// Diagnostics (shell `usb`)
+// ---------------------------------------------------------------------------
 pub fn diagnose() -> alloc::string::String {
     let mut s = alloc::string::String::new();
     s.push_str(&alloc::format!(
-        "xhci mmio={:#x} mapped={} ports={} mouse={} slot={}\n",
+        "xhci mmio={:#x} mapped={} ports={} slots={} scratchpad={}\n",
         unsafe { M },
-        if crate::paging::is_mapped(unsafe { M }) { "yes" } else { "no" },
-        unsafe { MAXPORTS },
-        if MOUSEPRESENT.load(Ordering::Relaxed) {
+        if crate::paging::is_mapped(unsafe { M }) {
             "yes"
         } else {
             "no"
         },
-        SLOT.load(Ordering::Relaxed)
+        unsafe { MAXPORTS },
+        unsafe { N_SLOTS },
+        unsafe { NUM_SPB }
     ));
-    unsafe {
-        // Page-table walk for the MMIO base (physical == virtual here).
-        let phys = M;
-        let pml4 = crate::paging::cr3() as *const u64;
-        let l4 = (phys >> 39) & 0x1FF;
-        let l3 = (phys >> 30) & 0x1FF;
-        let l2 = (phys >> 21) & 0x1FF;
-        let l1 = (phys >> 12) & 0x1FF;
-        let pml4e = pml4.add(l4).read_volatile();
-        let pdpte = if pml4e & 1 != 0 {
-            ((pml4e & 0x000f_ffff_ffff_f000) as *const u64).add(l3).read_volatile()
-        } else { 0 };
-        let pde = if pdpte & 1 != 0 && pdpte & (1 << 7) == 0 {
-            ((pdpte & 0x000f_ffff_ffff_f000) as *const u64).add(l2).read_volatile()
-        } else { 0 };
-        let pte = if pde & 1 != 0 && pde & (1 << 7) == 0 {
-            ((pde & 0x000f_ffff_ffff_f000) as *const u64).add(l1).read_volatile()
-        } else { 0 };
+    let n = NXDEV.load(Ordering::Relaxed) as usize;
+    s.push_str(&alloc::format!("  {} device(s):\n", n));
+    for i in 0..n {
+        let d = unsafe { &*core::ptr::addr_of!(XDEVS[i]) };
         s.push_str(&alloc::format!(
-            "  pt: pml4e={:#018x} pdpte={:#018x} pde={:#018x} pte={:#018x}\n",
-            pml4e, pdpte, pde, pte
+            "    {} slot={} dci={} armed={} fails={} reports={} ring={:#x}\n",
+            if d.kind == Kind::Mouse { "mouse" } else { "kbd" },
+            d.slot,
+            d.dci,
+            d.armed,
+            d.failures,
+            d.reports,
+            d.epin.raw as usize
         ));
+    }
+    unsafe {
         for p in 0..8 {
-            let psc = op(OP_PORTSC) + (p as usize) * 0x10;
+            let psc = op(OP_PORTSC) + (p as usize) * PORT_STRIDE;
             let v = rd(psc);
             s.push_str(&alloc::format!("  port{}: {:#010x}\n", p, v));
         }
     }
-    if MOUSEPRESENT.load(Ordering::Relaxed) {
-        s.push_str(&alloc::format!(
-            "  hid ep-index={} mps={} armed={}\n",
-            HID_EPID.load(Ordering::Relaxed),
-            unsafe { HID_LEN },
-            unsafe { QUEUED }
-        ));
-    }
 
-    // Cross-controller attach scan: report which PCI USB controller(s)
-    // currently have a device with bit0 (connect) set, across xHCI / EHCI /
-    // UHCI / OHCI.  Read-only diagnostic to locate the mouse on real hw.
+    // Cross-controller attach scan (read-only): locate devices on real hw.
     s.push_str("usbctl: scanning all USB controllers:\n");
     unsafe {
-    for d in crate::pci::probe() {
-        if d.class != 0x0C || d.subclass != 0x03 {
-            continue;
-        }
-        let lbl = match d.prog_if {
-            0x00 => "UHCI",
-            0x10 => "OHCI",
-            0x20 => "EHCI",
-            0x30 => "xHCI",
-            _ => "USB?",
-        };
-        let bar = crate::pci::bar64(d.bus, d.device, d.func, 0);
-        let base = bar.unwrap_or(0);
-        s.push_str(&alloc::format!(
-            "  {} {:02x}:{:02x}.{} bar={:#x}\n",
-            lbl, d.bus, d.device, d.func, base
-        ));
-        if base == 0 || base == u64::MAX {
-            // BAR not decoded / not enabled — nothing to probe.
-            continue;
-        }
-        match d.prog_if {
-            0x30 => {
-                if base == 0 {
-                    continue;
-                }
-                let caplen = rd(base as usize);
-                let hcs1 = rd(base as usize + 0x04);
-                let np = ((hcs1 >> 24) & 0xff) as usize;
-                let opb = base as usize + (caplen & 0xff) as usize;
-                s.push_str(&alloc::format!(
-                    "     xhci ports={} sc=", np
-                ));
-                for p in 0..np.min(16) {
-                    let v = rd(opb + 0x400 + p * 0x10);
-                    s.push_str(&alloc::format!(
-                        "{}{:#x}",
-                        if v & 1 != 0 { "*" } else { "" },
-                        v & 0x3ff
-                    ));
-                    if p + 1 < (np.min(16)) {
-                        s.push(' ');
-                    }
-                }
-                s.push('\n');
+        for d in crate::pci::probe() {
+            if d.class != 0x0C || d.subclass != 0x03 {
+                continue;
             }
-            0x20 => {
-                // EHCI: MMIO BAR.  CAPLENGTH byte0, HcSParams (nports) @0x04.
-                let cl = rd(base as usize) & 0xff;
-                let sp = rd(base as usize + 0x04);
-                let hc = rd(base as usize + 0x08);
-                let np = (sp & 0xf) as usize;
-                let opb = base as usize + cl as usize;
-                let cmd = crate::pci::config_read(d.bus, d.device, d.func, 0x04);
-                let eccp = crate::pci::config_read(d.bus, d.device, d.func, 0x34);
-                s.push_str(&alloc::format!(
-                    "     ehci nports={} hrs={} hcc={:#x} cmd={:#x} eccp={:#x}\n",
-                    np, sp, hc, cmd, (eccp >> 8) & 0xff
-                ));
-                let cfg = rd(opb + 0x40);
-                s.push_str(&alloc::format!(
-                    "     ehci cfgflag={:#x} ", cfg
-                ));
-                for p in 0..np.min(16) {
-                    let v = rd(opb + 0x44 + p * 0x04);
-                    s.push_str(&alloc::format!(
-                        "{}{:#x},",
-                        if v & 1 != 0 { "*" } else { "" },
-                        v & 0xffff
-                    ));
-                }
-                s.push('\n');
-                // Legacy BIOS handoff status (USBLEGSUP at EECP).
-                let eo = ((eccp >> 8) & 0xff) as usize;
-                if eo != 0 && (eccp & 0xff) != 0 {
-                    let hs = crate::pci::config_read(
-                        d.bus,
-                        d.device,
-                        d.func,
-                        (eo & 0xfc) as u8,
-                    );
-                    s.push_str(&alloc::format!(
-                        "     ehci legacy=usblegsup={:#x} os-owner={} bios-owned={}\n",
-                        hs,
-                        (hs & (1 << 24)) != 0,
-                        (hs & (1 << 16)) != 0 || (hs & 0x100) != 0
-                    ));
-                }
+            let lbl = match d.prog_if {
+                0x00 => "UHCI",
+                0x10 => "OHCI",
+                0x20 => "EHCI",
+                0x30 => "xHCI",
+                _ => "USB?",
+            };
+            let bar = crate::pci::bar64(d.bus, d.device, d.func, 0);
+            let base = bar.unwrap_or(0);
+            s.push_str(&alloc::format!(
+                "  {} {:02x}:{:02x}.{} bar={:#x}\n",
+                lbl,
+                d.bus,
+                d.device,
+                d.func,
+                base
+            ));
+            if base == 0 || base == u64::MAX {
+                continue;
             }
-            0x00 => {
-                // UHCI: I/O BAR (bar64 refuses I/O bars — read raw).
-                let lo = crate::pci::config_read(d.bus, d.device, d.func, 0x10);
-                let io = (lo & 0xFFF0) as u16;
-                if io == 0 {
-                    continue;
+            match d.prog_if {
+                0x30 => {
+                    let caplen = rd(base as usize);
+                    let hcs1 = rd(base as usize + 0x04);
+                    let np = ((hcs1 >> 24) & 0xff) as usize;
+                    let opb = base as usize + (caplen & 0xff) as usize;
+                    s.push_str("     xhci ports=");
+                    for p in 0..np.min(16) {
+                        let v = rd(opb + 0x400 + p * 0x10);
+                        s.push_str(&alloc::format!(
+                            "{}{:#x} ",
+                            if v & 1 != 0 { "*" } else { "" },
+                            v & 0x3ff
+                        ));
+                    }
+                    s.push('\n');
                 }
-                for p in 0..2 {
-                    let pr = io_iow16(io + 0x10 + (p as u16) * 2);
+                0x20 => {
+                    let cl = rd(base as usize) & 0xff;
+                    let sp = rd(base as usize + 0x04);
+                    let hc = rd(base as usize + 0x08);
+                    let np = (sp & 0xf) as usize;
+                    let opb = base as usize + cl as usize;
+                    let cfg = rd(opb + 0x40);
                     s.push_str(&alloc::format!(
-                        "{}{:x}:", if pr & 1 != 0 { "*" } else { "" },
-                        (pr & 0xff)
+                        "     ehci nports={} hcc={:#x} cfgflag={:#x} ",
+                        np, hc, cfg
                     ));
+                    for p in 0..np.min(16) {
+                        let v = rd(opb + 0x44 + p * 0x04);
+                        s.push_str(&alloc::format!(
+                            "{}{:#x},",
+                            if v & 1 != 0 { "*" } else { "" },
+                            v & 0xffff
+                        ));
+                    }
+                    s.push('\n');
+                }
+                0x00 => {
+                    let lo = crate::pci::config_read(d.bus, d.device, d.func, 0x10);
+                    let io = (lo & 0xFFF0) as u16;
+                    if io == 0 {
+                        continue;
+                    }
+                    s.push_str("     uhci ports=");
+                    for p in 0..2 {
+                        let pr = io_iow16(io + 0x10 + (p as u16) * 2);
+                        s.push_str(&alloc::format!(
+                            "{}{:x},",
+                            if pr & 1 != 0 { "*" } else { "" },
+                            pr & 0xff
+                        ));
+                    }
+                    s.push('\n');
                 }
-                s.push('\n');
+                _ => {}
             }
-            _ => {}
         }
     }
-    }
     s
 }