bw24-bench — benchmark artifacts for bw24

Reproduction artifacts for the performance claims in github.com/avifenesh/bw24 — a from-scratch LLM inference engine for RTX 5090 Laptop (sm_120a). If a claim depends on a specific artifact, the artifact is public here.

Drafts (drafts/<model>/) — the standard regime, one file per model

Every bw24 spec board row runs one trimmed draft file built by the documented regime (docs/DRAFT-REGIME.md): FR-Spec ranks derived from the model's own generations (never transferred between models — measured −12 acceptance pts), MTP block extracted byte-verbatim from the published model GGUF, head requantized NVFP4 after trimming (zero measured acceptance cost), block Q4_K_M (faster AND higher acceptance than Q8_0). Serve with BW24_MTP_DRAFT=<draft.gguf> — no other flags.

Directory	Source model (exact bytes)	Board row (e2e tok/s p1/p2/p3, vs llama.cpp same-window)
drafts/qwen35-9b-nvfp4/	Qwen3.5-9B NVFP4 MTP GGUF	281 / 212 / 187 = 2.30x / 1.74x / 1.59x
drafts/qwen36-27b-nvfp4/	nvidia/Qwen3.6-27B-NVFP4 → Q4_K_M GGUF	116 / 101 / 86 = 1.27x / 1.08x / 1.06x
drafts/qwen36-35b-a3b-iq4xs/	unsloth Qwen3.6-35B-A3B UD-IQ4_XS	281 / 260 / 258 = 1.19x / 1.49x / 1.49x
drafts/qwen36-27b-unsloth-nvfp4/	unsloth/Qwen3.6-27B-NVFP4 → GGUF (recipe in bw24 jsonl)	plain parity with the nvidia daily; spec p3 win / p2 loss (jsonl 2026-07-17)

Each directory carries the draft (draft-owntrim-nvfp4head-q4blk.gguf) and the rank file it was built from (owngen-ranks-32768.txt, one token id per line, rank order — derived 2026-07-17/18, 218-prompt mixed corpus, ~110k own-generated tokens per model).

Use ours for these exact models. For any other model, requant, or finetune, build your own in two commands (a finetune's distribution moved, so its draft must too):

./target/release/frspec-owngen model.gguf ranks.gguf 32768        # ranks from the model's OWN generations
tools/make-trimmed-draft.sh model.gguf ranks.gguf.txt draft.gguf  # extract + trim + quantize

NVFP4-KV drafts (drafts/kf4/<model>/) — the BW24_KV_K=nvfp4 BW24_KV_V=nvfp4 config

The nvfp4 KV-cache format (kv-fp4 lane, 2026-07-20: 18 B/32 elems both planes = −38% KV bytes/token; self-consistency PASS, battery green in-config) is a NEW NUMERIC CONFIG — per the regime's law 1, each model's draft is re-derived from its OWN generations under that config. These directories carry the kf4-derived draft + rank files:

Directory	Verdict vs the default-config draft (same config, K=cell, N=2)
drafts/kf4/qwen35-9b-nvfp4/	wash (old file marginally ahead; either serves)
drafts/kf4/qwen36-27b-nvfp4/	kf4 draft WINS (+2.5% e2e, acceptance 60.1 vs 57.1) — use this one under nvfp4 KV
drafts/kf4/qwen36-35b-a3b-iq4xs/	old draft stays (−1% / acceptance 67.0 vs 69.2) — kept as evidence

Gemma models keep their existing rank files under nvfp4 KV (measured never-worse, 2026-07-20 jsonl): load-time trim + the serve-time adaptive trim absorb the format shift.

Gemma ranks (drafts/gemma4-*/)

Gemma drafters ship as standalone MTP/assistant GGUFs (byte-verbatim by provenance — law 2 built in); the FR-Spec trim applies at load: BW24_GEMMA_DRAFT_RANKS=<ranks.txt>. Own-gen rank files per model (law 1):

Directory	Model	Serving note
drafts/gemma4-26b-a4b-qat/	Gemma-4 26B-A4B QAT Q4_0	trim adopted (e2e-neutral vs corpus ranks, correct provenance)
drafts/gemma4-31b-qat/	Gemma-4 31B QAT Q4_0	trim adopted on BOTH cells via the serve-time ADAPTIVE trim (BW24_GEMMA_TRIM_ADAPT, on by default with ranks): coverage escapes learned live from prompt+verify tokens, persisted to <ranks>.learned — flipped the chat cell from −17% to +2.5% (bw24 jsonl 2026-07-19)
drafts/gemma4-e4b-qat/	Gemma-4 E4B QAT Q4_0	ranks published; serving stays UNTRIMMED (measured e2e wash — small head)

Legacy trims (top level, Qwen3.6-27B)

The pre-regime artifacts the earlier boards used — superseded by drafts/ but kept because published claims referenced them:

File	Ranking	Status
mtp-Qwen3.6-27B-Q4_K_M-frspec32768.gguf	generic corpus-frequency	superseded by drafts/qwen36-27b-nvfp4/
mtp-Qwen3.6-27B-Q4_K_M-frspec-code75-32768.gguf	code-skewed corpus	superseded
mtp-Qwen3.6-27B-Q4_K_M-frspec-balanced32768.gguf	balanced corpus	superseded

Prompts + configs

prompts/ holds the exact p1/p2/p3 board prompts; CONFIGS.md the per-cell engine configs and llama.cpp pairing flags. Protocol: interleaved same-window N≥2, power state pinned, one engine on the GPU at a time — research/benchmarks.md.

Related

• Engine: github.com/avifenesh/bw24 — every published number's raw JSONL under research/, wins and losses both.
• Avifenesh/Hy3-REAP-Layer103p5-bw24 — the ~100 GB Hy3 expert overlay bw24 serves on a 24 GB card via VRAM→RAM→NVMe spill.