Qwen3.8-27B Trim B7

We ran out of compute while finishing the broader FFN pruning study. This release packages the complete, verified block-7-removed B7 checkpoint so the work is available instead of being lost. No unfinished FFN width cut is merged into these weights.

Support the project

Buy Me a Coffee

Donations will be used to fund the project and AI subscriptions.

Donate here: https://buymeacoffee.com/dgresesjr

Included weights

File / directory Format Size
Root safetensors shards HF BF16, 18 shards 50.478 GB of tensors
gguf/qwen3.8-27b-trim-b7-bf16.gguf GGUF BF16 49.567 GB
gguf/qwen3.8-27b-trim-b7-q4_k_m.gguf GGUF Q4_K_M 15.324 GB

The Q4_K_M file is the practical option for a 24 GB GPU. The BF16 checkpoint requires more than 50 GB of weight memory.

Model details

  • 25.239B tensor elements
  • 60 main layers, plus one MTP layer in GGUF
  • Original block 7 (layers 28-31) physically removed and later layers reindexed
  • Three linear-attention layers followed by one full-attention layer
  • Hidden size: 5,120
  • FFN intermediate width: 17,408
  • 1.522B parameters and 3.044 GB of BF16 tensors removed relative to the trimmed input

Validation

  • All 1,146 HF tensors and every FFN shape verified
  • B7 loss matched the earlier block-7 skip run within 0.00108 maximum bucket difference
  • B7 hard-benchmark result: 88/160 (math 22/40, code 40/40, tools 12/40, knowledge 14/40)
  • Corrected original-model tools baseline: 13/40
  • Q4_K_M loaded with full GPU offload on an RTX 3090 using llama.cpp b12359 and correctly answered 17 + 25 = 42

Q4 usage

llama-completion \
  -m qwen3.8-27b-trim-b7-q4_k_m.gguf \
  -p "17 + 25 =" -n 128 --temp 0 -ngl 99 -c 512 -t 6 -no-cnv

FFN checkpoints

The 10%, 20%, and 30% repair plans reached 38/60, 40/60, and 33/60 layers respectively before compute ran out. They are preserved separately as checkpoints, not complete models, and are intentionally not included in this deployable checkpoint.

Downloads last month
204
Safetensors
Model size
25B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support