empero-ai commited on
Commit
486faa5
·
verified ·
1 Parent(s): 5783620

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -48
README.md CHANGED
@@ -32,23 +32,20 @@ A Gated-DeltaNet-aware mixed GGUF of official
32
  for [llama.cpp](https://github.com/ggml-org/llama.cpp), Ollama, LM Studio,
33
  jan, KoboldCpp, and other stock GGUF runtimes.
34
 
35
- This is a quantization of the stock Qwen checkpoint — not a finetune, and
36
- not a new codec. Ridge is a probed mix of types llama.cpp already ships,
37
  written for this architecture: 64 layers =
38
  16 × `(3 × GatedDeltaNet → FFN + 1 × GatedAttn → FFN)`. Generic `IQ2_XS`
39
- and Unsloth UD-IQ2 do not treat GDN state (`ssm_alpha` / `ssm_beta`) or
40
- the GDN mixers as first-class. That is the gap.
41
 
42
  Nothing was stripped to make the file fit. The native MTP draft head
43
  (`blk.64` / `nextn`) stays in the GGUF. Vision is a separate BF16
44
  `mmproj`.
45
 
46
- The name is the measured bitrate: **3.69 bpw, 11.73 GiB**. It is not a
47
- 2-bit file, and it is not lossless.
48
-
49
- This card is about choosing the file and running it. The official
50
- capability writeup lives on the
51
- **[base model card](https://huggingface.co/Qwen/Qwen3.8-27B)**.
52
 
53
  ---
54
 
@@ -68,7 +65,7 @@ If you only want text, download the Ridge GGUF. Add the `mmproj` for image input
68
 
69
  These are practical **weight-size-based estimates**, not a VRAM benchmark.
70
  They assume a modest context and leave room for runtime and the KV cache.
71
- Image input adds the 0.87 GiB `mmproj`. The native 262k window — and the
72
  1M YaRN extension — make KV the dominant cost and may need offload
73
  regardless of weight quant.
74
 
@@ -92,18 +89,6 @@ layer. GDN state is disproportionately sensitive to low-bit quantization,
92
  so Ridge holds that path high and spends the saved bits by dropping
93
  mid-stack FFN.
94
 
95
- | Role | Tensors | Type |
96
- |---|---|---|
97
- | Embeddings, `lm_head` | `token_embd`, `output.weight` | Q6_K |
98
- | GDN state | `ssm_alpha`, `ssm_beta`, `ssm_conv1d`, `ssm_a`, `ssm_dt`, `ssm_norm` | Q8_0 |
99
- | Norms | `attn_norm`, `post_attention_norm`, `output_norm`, `attn_q_norm`, `attn_k_norm` | Q8_0 |
100
- | GDN mixers (48 linear-attn layers) | `attn_qkv`, `attn_gate`, `ssm_out` | Q4_K |
101
- | Full attention (layers 3, 7, …, 63) | `attn_q`, `attn_k`, `attn_v`, `attn_output` | Q5_K |
102
- | FFN first / last 4 layers (`blk.0–3`, `blk.60–63`) | `ffn_{gate,up,down}` | IQ3_S |
103
- | FFN mid (bulk of the bits) | `ffn_{gate,up,down}` | IQ2_M |
104
- | MTP draft (`blk.64`; no imatrix) | `nextn.*`, `blk.64.ffn_*` | Q6_K |
105
- | Vision | separate file | BF16 |
106
-
107
  **The Gated-DeltaNet state path is Q8_0.** Mixers are Q4_K, not IQ2.
108
  That is the difference between this file and a flat 2-bit dump of the
109
  same model.
@@ -125,11 +110,6 @@ Same box, same calibration file, `llama-perplexity`, 80 chunks,
125
  | BF16 GGUF (this convert) | 50.89 GiB | 16.00 | **7.15 ± 0.12** | — |
126
  | **Ridge-3.7bpw** | **11.73 GiB** | **3.69** | **7.82 ± 0.14** | **+9.3 %** |
127
 
128
- Not lossless. +9 % PPL at 23 % of BF16 size is the first-bake number.
129
- A same-suite bake-off against Unsloth `UD-Q3_K_XL` and bartowski
130
- `IQ3_XXS` has not been run on this box yet — do not read the size table
131
- below as a quality ranking.
132
-
133
  ---
134
 
135
  ## Comparison
@@ -143,24 +123,10 @@ where we measured the file ourselves.
143
  | `UD-IQ2_XXS` | [unsloth](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) | 8.39 GiB | ~2.1 bpw | *not measured here* (Unsloth quotes 82.5 % top-1 vs BF16) |
144
  | `UD-IQ2_M` | unsloth | 9.61 GiB | ~2.4 bpw | *not measured* |
145
  | `IQ2_XXS` | [bartowski](https://huggingface.co/bartowski/Qwen3.8-27B-GGUF) | 8.75 GiB | ~2.2 bpw | *not measured* |
146
- | `IQ2_M` | bartowski | 10.13 GiB | ~2.5 bpw | *not measured* |
147
- | `UD-IQ3_XXS` | unsloth | 11.10 GiB | ~2.8 bpw | *not measured* |
148
- | `Q2_K` | bartowski | 11.03 GiB | ~2.7 bpw | *not measured* |
149
  | `Q3_K_S` | unsloth | 11.71 GiB | ~3.1 bpw | *not measured* |
150
  | **Ridge-3.7bpw** | **empero-ai** | **11.73 GiB** | **3.69 bpw** | **7.82 (+9 %)** |
151
  | `IQ3_XXS` | bartowski | 11.76 GiB | ~2.9 bpw | *not measured* |
152
  | `UD-Q3_K_XL` | unsloth | 12.52 GiB | ~3.4 bpw | *not measured* |
153
- | `Q3_K_M` | unsloth | 12.87 GiB | ~3.5 bpw | *not measured* |
154
- | `Q4_K_M` | unsloth | 15.93 GiB | ~4.3 bpw | *not measured* |
155
-
156
- How to read this:
157
-
158
- - Ridge-3.7bpw sits in the **~12 GiB** band, next to Unsloth `Q3_K_S` and
159
- bartowski `IQ3_XXS` — not next to UD-IQ2_XXS.
160
- - The extra bits versus a flat IQ2 go into **GDN state (Q8_0)**, **GDN
161
- mixers (Q4_K)**, and **full attention (Q5_K)**. Mid-stack FFN stays IQ2_M.
162
- - We are not claiming this file beats Unsloth 2-bit at the same size. We
163
- spent the bits on GDN on purpose.
164
 
165
  ---
166
 
@@ -288,12 +254,7 @@ YaRN. Set `-c` to what you actually need — the KV cache, not the
288
 
289
  ## Limitations
290
 
291
- - **Not lossless.** +9 % wiki-style PPL vs our BF16 convert is a real
292
- gap. Do not describe this as an almost-lossless 2-bit.
293
- - **Not a 2-bit file.** The name is Ridge-**3.7bpw** because that is
294
- what llama-quantize reported (3.69).
295
- - **Same-suite bake-off pending.** Unsloth / bartowski PPL on this
296
- calibration set has not been measured here.
297
  - **Context costs memory.** Weight size is only part of the hardware
298
  budget.
299
  - **MTP is runtime-dependent.** The head is in the file; the speedup
 
32
  for [llama.cpp](https://github.com/ggml-org/llama.cpp), Ollama, LM Studio,
33
  jan, KoboldCpp, and other stock GGUF runtimes.
34
 
35
+ This is a quantization of the Qwen3.8-27B checkpoint. Ridge is a probed mix of types
 
36
  written for this architecture: 64 layers =
37
  16 × `(3 × GatedDeltaNet → FFN + 1 × GatedAttn → FFN)`. Generic `IQ2_XS`
38
+ and UD-IQ2 do not treat GDN state (`ssm_alpha` / `ssm_beta`) or
39
+ the GDN mixers as first-class. We fixed that.
40
 
41
  Nothing was stripped to make the file fit. The native MTP draft head
42
  (`blk.64` / `nextn`) stays in the GGUF. Vision is a separate BF16
43
  `mmproj`.
44
 
45
+ > [!Note]
46
+ > This card is about choosing the file and running it. The official
47
+ > capability writeup lives on the
48
+ > **[base model card](https://huggingface.co/Qwen/Qwen3.8-27B)**.
 
 
49
 
50
  ---
51
 
 
65
 
66
  These are practical **weight-size-based estimates**, not a VRAM benchmark.
67
  They assume a modest context and leave room for runtime and the KV cache.
68
+ Image input adds the 0.87 GiB `mmproj`. The native 262k window and the
69
  1M YaRN extension — make KV the dominant cost and may need offload
70
  regardless of weight quant.
71
 
 
89
  so Ridge holds that path high and spends the saved bits by dropping
90
  mid-stack FFN.
91
 
 
 
 
 
 
 
 
 
 
 
 
 
92
  **The Gated-DeltaNet state path is Q8_0.** Mixers are Q4_K, not IQ2.
93
  That is the difference between this file and a flat 2-bit dump of the
94
  same model.
 
110
  | BF16 GGUF (this convert) | 50.89 GiB | 16.00 | **7.15 ± 0.12** | — |
111
  | **Ridge-3.7bpw** | **11.73 GiB** | **3.69** | **7.82 ± 0.14** | **+9.3 %** |
112
 
 
 
 
 
 
113
  ---
114
 
115
  ## Comparison
 
123
  | `UD-IQ2_XXS` | [unsloth](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) | 8.39 GiB | ~2.1 bpw | *not measured here* (Unsloth quotes 82.5 % top-1 vs BF16) |
124
  | `UD-IQ2_M` | unsloth | 9.61 GiB | ~2.4 bpw | *not measured* |
125
  | `IQ2_XXS` | [bartowski](https://huggingface.co/bartowski/Qwen3.8-27B-GGUF) | 8.75 GiB | ~2.2 bpw | *not measured* |
 
 
 
126
  | `Q3_K_S` | unsloth | 11.71 GiB | ~3.1 bpw | *not measured* |
127
  | **Ridge-3.7bpw** | **empero-ai** | **11.73 GiB** | **3.69 bpw** | **7.82 (+9 %)** |
128
  | `IQ3_XXS` | bartowski | 11.76 GiB | ~2.9 bpw | *not measured* |
129
  | `UD-Q3_K_XL` | unsloth | 12.52 GiB | ~3.4 bpw | *not measured* |
 
 
 
 
 
 
 
 
 
 
 
130
 
131
  ---
132
 
 
254
 
255
  ## Limitations
256
 
257
+ - **Not lossless.** +9 % wiki-style PPL vs our BF16 convert
 
 
 
 
 
258
  - **Context costs memory.** Weight size is only part of the hardware
259
  budget.
260
  - **MTP is runtime-dependent.** The head is in the file; the speedup