huginnfork/Qwen3.8-Flash-Next-mixed-NVFP4-FP8-noPLE

Qwen3.8-Flash-Next mixed NVFP4/FP8, without the BF16 PLE table

primitive-ai/Qwen3.8-Flash-Next-mixed-NVFP4-FP8 at revision fd7dbb007b83d4ac41db93ccd1cf2b352c6e8bd2, minus the 43 ple-bf16-*.safetensors shards, with model.safetensors.index.json trimmed to match (296630 -> 296502 tensors). Nothing else is changed.

Those shards are the 95.4 GiB BF16 n-gram table, and they hold nothing else -- all 128 dropped tensors are model.language_model.layers.1.ple.ple_embedding.ngram_embedding.shard_N.weight.

Why

Serve this with the quantised PLE sidecars from primitive-ai/Qwen3.8-Flash-Next-PLE-quant and nothing ever reads the BF16 table: GPU workers skip PLE layer init under VLLM_PLE_CPU_OFFLOAD, and the offload worker filters those tensors out of the checkpoint stream once VLLM_PLE_QUANT_DIR is set. vLLM still fetches every *.safetensors in a repo before it loads anything, though, so pointing it at the upstream repo id means downloading and keeping 95.4 GiB in order to ignore it. On a host with a small disk, or a small amount of RAM to spare for page cache, that is the difference between fitting and not.

Use the upstream repo instead if you want the BF16 table -- for the in-RAM path (~100 GB of host RAM) or the BF16-on-NVMe path (worker_image_disk.py, VLLM_PLE_DISK_OFFLOAD_DIR). This fork only makes sense with the quantised sidecars.

All credit for the checkpoint to primitive-ai; this is a repackaging.