StillDeadcode/swift1.5-qwen3.8-next-flash-fp8-iq4r-moe

Swift 1.5 Qwen3.8-Flash-Next, 4-bit codebook experts + int8 trunk — radiance container

ukisai/Swift1.5-Qwen3.8-Flash-Next — UkisAI's reasoning-efficient finetune of Qwen3.8-Flash-Next — quantised from its bf16 checkpoint into a single .rad container for the radiance inference engine (AMD RDNA4, ROCm), with the model's MTP head for speculative decoding and its vision tower. It is the recipe of StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe, calibrated on this finetune.

The source model's card describes how it was trained, how it was evaluated and what it is for.

File	swift1.5-qwen3.8-next-flash-fp8-iq4r-moe.rad — 113.5 GiB
Routed experts	4-bit codes into a 16-level non-uniform codebook after a 128-point Walsh–Hadamard rotation, one E4M3 scale per 64 weights; codes chosen by GPTQ and refined by three sweeps of coordinate descent. The Hessians (pooled and per-expert down-projection) were recorded from this finetune's own bf16 weights over 1M tokens of chat, tool-use and code
Protected experts	sixteen experts that carry most of their layer's down-projection energy, kept bf16, chosen from that recording
Trunk	attention, delta-net and shared-expert linears and the lm_head int8 (W8A8), a scale per 128 columns searched for least error; hyper-connection mixing matrices E4M3
Speculator	the model's MTP head (depth 3 by default)
Vision	the vision tower, bf16: images and video in chat requests
Context	262,144 tokens trained

The finetune changed the attention, delta-net and shared-expert projections and left the routed experts as they were. Its new attention drives the experts differently, though: with the base model's protection rule applied to its own recording, sixteen experts qualify instead of the base model's ten, among them early-layer experts the base model hardly used (layer 4's expert 108 carries 15.8% of its layer's down-projection energy from 0.18% of the routing).

Quality

This file was not measured against its own bf16 weights. For the recipe's error, the same recipe on the base model (with a 10M-token calibration; on held-out data a 1M-token one left the experts' output error within 1% of it), as full-vocabulary KL divergence against its bf16, teacher-forced over ~75K positions of chat, tool-use and code transcripts:

	mean KL	median	p99	p99.9	top-1 agreement
all tokens	0.0789	0.0037	1.65	5.33	92.85%
text and assistant turns	0.0239	0.0029	0.32	1.03	94.57%
reference: a second bf16 implementation, all tokens	0.0399	0.0013	0.82	3.43	95.13%

Before upload it was served on one Radeon AI PRO R9700 (--tp 1, 32K context): arithmetic, a multi-step word problem, a tool call, an image description and four concurrent requests all answered correctly, with 62–88% of the MTP drafts accepted.

Serve

The routed experts do not fit two 32 GB cards; the engine keeps the hottest in VRAM and streams the rest from a pinned host pool, so the host needs about 32 GB of free RAM.

radiance --model swift1.5-qwen3.8-next-flash-fp8-iq4r-moe.rad --tp 2 --tp-wire wht6 \
    --max-model-len 200000 --max-num-seqs 8 --kv-cache-dtype fp8 \
    --placement expert_tiered --host-pool-mib 12288 --gpu-headroom-mib 96 \
    --num-speculative-tokens 3 --max-num-batched-tokens 2048 \
    --host 0.0.0.0 --port 8000

On one 32 GB card, with about 40 GB of free host RAM, reading the experts that fit nowhere from the file:

radiance --model swift1.5-qwen3.8-next-flash-fp8-iq4r-moe.rad --tp 1 \
    --max-model-len 32768 --max-num-seqs 4 --kv-cache-dtype fp8 \
    --placement expert_tiered --weights-disk-tier --ngram-placement disk --host-pool-mib 38600 \
    --num-speculative-tokens 3 --max-num-batched-tokens 2048 \
    --host 0.0.0.0 --port 8000

The server speaks the OpenAI API (/v1/chat/completions, /v1/completions), with tool calls, structured output, and image_url / video parts in chat messages.

How this file was made

CALIB=calib/swift-qfn-bf16-train1m-pe rad-convert ukisai/Swift1.5-Qwen3.8-Flash-Next \
    --tokenizer Qwen/Qwen3.8-Flash-Next/tokenizer.json \
    --recipe swift15-w4nl64-i8-hc8m.recipe -o swift1.5-qwen3.8-next-flash-fp8-iq4r-moe.rad

The finetune's tokenizer.json was re-serialised (its NOTICE says so) and has the same vocabulary, merges and special tokens as the base model's; the base model's file was baked in.

The recipe (swift15-w4nl64-i8-hc8m.recipe in this repository; $CALIB is its calibration data, which is not published). The first rule that matches a weight decides it, and everything no rule names is the checkpoint's bf16:

blk.2.ffn_*_exps.322.weight cast dtype=bf16
blk.2.ffn_*_exps.413.weight cast dtype=bf16
blk.4.ffn_*_exps.108.weight cast dtype=bf16
blk.4.ffn_*_exps.168.weight cast dtype=bf16
blk.4.ffn_*_exps.373.weight cast dtype=bf16
blk.4.ffn_*_exps.426.weight cast dtype=bf16
blk.14.ffn_*_exps.169.weight cast dtype=bf16
blk.14.ffn_*_exps.212.weight cast dtype=bf16
blk.42.ffn_*_exps.43.weight cast dtype=bf16
blk.42.ffn_*_exps.124.weight cast dtype=bf16
blk.46.ffn_*_exps.290.weight cast dtype=bf16
blk.46.ffn_*_exps.392.weight cast dtype=bf16
blk.47.ffn_*_exps.122.weight cast dtype=bf16
blk.47.ffn_*_exps.143.weight cast dtype=bf16
blk.47.ffn_*_exps.399.weight cast dtype=bf16
blk.47.ffn_*_exps.445.weight cast dtype=bf16
blk.*.ffn_*_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=calib/swift-qfn-bf16-train1m-pe
*_hc_down.weight rtn codes=fp8_e4m3 group=128 scale=f32
*_hc_up.weight rtn codes=fp8_e4m3 group=80 scale=f32
mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32
mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32
blk.*.attn_qg.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.attn_k.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.attn_v.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.attn_output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ssm_inz.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ffn_gate_up_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ffn_down_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4
mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16

License

Copyright 2026 UkisAI. Swift Contribution licensed under the Swift Open License v1.0 (https://huggingface.co/ukisai/Swift-Qwen3.8-Flash-Next/blob/main/LICENSE). Derivative of Qwen3.8-Flash-Next, Copyright (c) 2026 Qwen, Qwen Community License 1.0.

This repository carries the Swift Open License v1.0 (LICENSE), the base model's Qwen Community License 1.0 (LICENSE-QWEN) and UkisAI's NOTICE, unchanged. Under the Swift Open License, commercial use of the Swift contribution by an organisation with US$1,000,000 or more in annual revenue needs a separate license from UkisAI, and the Qwen Community License's own terms apply to the base model; read both files.

Changes made here: the finetune's weights were quantised as described above and, with the base model's tokenizer and the chat template, written into the single swift1.5-qwen3.8-next-flash-fp8-iq4r-moe.rad file. No weight was trained. This conversion is not made or endorsed by UkisAI.