SigDB Update: WAL Crash Recovery and the /db VFS Provider

Two commits have landed on top of the initial SigDB implementation. The first closes the STABLE pillar: a crashed transaction is fully absent after reboot, never partially visible. The second makes SigDB a first-class filesystem citizen: every committed key appears as a file under /db, and its value is the file's content. MANIFEST is now at 327/327 ALL PASS.

WAL crash recovery (RWICVEY)

SigDB's write-ahead log records every field mutation before it touches the B-tree page. On a clean commit, the WAL tail is zeroed and the page reflects the new values. On a crash — power loss, kernel fault, anything that interrupts execution mid-transaction — the WAL tail is left non-zero, pointing at uncommitted mutations that made it into the WAL but not into a final commit record.

sigdb_recover(base) in sigdb/recover.sg runs at boot before any VFS mount:

Open the volume and verify the magic number. If the magic is wrong, return -1 immediately — the volume is not a SigDB volume and recovery would corrupt it.
Check wal_tail. If zero: the last transaction committed cleanly, nothing to do, return 0.
If non-zero: call sigdb_wal_undo() — walk the WAL records in reverse and restore each field to its pre-transaction value (prev_val / prev_epoch). Clear wal_tail.
Return 0 (recovered).

The RWICVEY test demonstrates the full crash scenario end-to-end:

Test ID	What it checks
R	Volume opens cleanly; wal_tail=0 after clean commit
W	Commit key=1 val=100 — WAL records written, then committed; key=1 readable
I	Insert key=2 val=200 — WAL records written, no commit (crash simulation: wal_tail left non-zero)
C	`sigdb_recover(base)` returns 0
V	wal_tail = 0 after recovery (WAL cleared)
E	key=2 is absent — the uncommitted insert was undone
Y	key=1 val=100 is still present — committed data survives recovery

The critical property V + E + Y in combination: recovery is not a replay, it is an undo. Only the uncommitted portion is removed. A transaction that committed successfully before the crash is untouched — its WAL records have already been cleared.

This closes the last gap in SigDB's STABLE guarantee: any transaction is either fully committed or fully absent after a reboot. There is no intermediate state a reader can observe.

The /db VFS provider (DBPRXVF)

SigDB is now VFS provider ID 13. When /db is mounted, the database's committed key set is exposed as a directory of decimal-named files:

$ ls /db

10 20 42 100

$ cat /db/42

1337

$ cat /db/99

cat: /db/99: No such file or directory

# key 99 not committed → resolve returns -1 → ENOENT

Three VFS operations are wired in sigdb/dbfs.sg:

dbfs_vfs_readdir(index, ent) — iterates committed keys in insertion order. Uses sigdb_scan to walk the B-tree; emits each key as a decimal filename. Returns -1 past the last entry.
dbfs_vfs_resolve(name_ptr) — parses the decimal filename to a key via dbfs_atoi, calls sigdb_scan_key to verify the key is committed, returns the key as a node ID. Returns -1 if the key is absent (ENOENT to the caller).
dbfs_vfs_read(key, off, buf, n) — formats the value as a decimal string with a trailing newline via dbfs_emit_u32, slices by off and n. Returns bytes written.

The snapshot epoch passed to sigdb_scan is 2^30-1 — the maximum MVCC epoch, which sees all committed data regardless of when it was written. This means /db is always a view of the latest committed state, not a point-in-time snapshot.

The decimal codec deserves a note. Sigil has no string literal syntax for numeric formatting — dbfs_atoi and dbfs_emit_u32 are hand-coded character-by-character converters. dbfs_emit_u32 writes digits from least-significant to most-significant into a scratch buffer, then reverses in-place before copying to the output. dbfs_atoi multiplies an accumulator by 10 for each digit character and adds the digit value. Both are about a dozen lines each and have no dependencies on any runtime library.

The DBPRXVF test suite covers the expected behaviour:

Test ID	What it checks
D	`readdir(0)` returns filename "10" (first committed key)
B	`readdir(1)` returns filename "20" (second committed key)
P	`readdir(2)` returns -1 (past end of key set)
R	`resolve("10")` returns node_id=10
X	`resolve("99")` returns -1 (key not present)
V	`read(10)` returns "100\n" (value of key 10)
F	`read(20)` returns "200\n" (value of key 20)

MANIFEST at 327/327

The /db VFS commit pushed MANIFEST from 325 to 327. Every sigil-fs subsystem — FAT, NFS, SMB, procfs, sysfs, SigDB (core + WAL + recover + /db), pipe, shmem — runs in the same harness. The 327 count is the combined total with 0 failures.

The VFS provider model is now hosting five distinct backends: fat (ID 3), nfs (ID 1), smb (ID 2), proc (ID 11), sys (ID 12), and now db (ID 13). Each is a set of three functions — resolve, read, readdir — registered in vfs_dev.sg's dispatch table. Adding a new filesystem is three function pointers and a provider ID.