microsoft

/

bitnet-b1.58-2B-4T

Like 1.5k

Follow

Microsoft 22.5k

We could not find this number published anywhere, so we measured it. Sharing in case it is useful.

The measurement. microsoft/bitnet-b1.58-2B-4T, the official ggml-model-i2_s.gguf, all 210 ternary tensors, 2,084,044,800 weights:

code 0   28.89%   ->  -1
code 1   42.21%   ->   0
code 2   28.90%   ->  +1
code 3    0.00%   ->  unused

Only three of the four 2-bit codes appear, so the encoding is genuinely ternary. The -1/0/+1 assignment is inferred, not read from the kernel: the two outer codes are symmetric to 0.01% and the middle one is the outlier. If anyone has traced quantize_i2_s in bitnet.cpp, confirming the mapping would close it.

The headline: the zero is the majority state. 42.21% - 880M of 2.08B weights - so 880M multiply-accumulates are skipped outright rather than approximated. That is above the balanced-ternary third (33%), which we cannot explain.

The control. Same tool, two models that were NOT trained ternary:

model                      ternary-trained?   zero       neg/pos
BitNet b1.58 2B (i2_s)     yes                42.21%     28.89 / 28.90
Ornith-1.5-9B (Q5_K_M)     no                  0.444%    49.77 / 49.78
Qwen3.8-27B (IQ4_XS)       no                  0.000%    50.00 / 50.00

3.2 billion weights in the 27B and not one exact zero. So the zero tracks training - not quantization, not architecture, not family.

The second finding, which surprised us more: the +/- symmetry is invariant while the zero is the variable. 50.00/50.00, 49.77/49.78, 28.89/28.90 - every model within 0.23% of perfect symmetry across three architectures, two quant families and two training regimes. Training does not break the balance; it changes how much mass the zero takes. In BitNet the halves shrink to 28.9% each and the zero absorbs the rest.

Where the zero sits, by function:

ffn_gate      24.25%    <- fewest zeros
attn_q        25.34%
attn_k        28.06%
ffn_up        28.61%
attn_output   28.85%
ffn_down      30.55%
attn_v        31.25%    <- most

The matrices that DECIDE keep their zeros few; the matrices that CARRY keep them many. If that is a rule and not a coincidence, it is a design constraint worth having.

Caveat, because it bounds the number: the 42.21% carries our converter's fingerprint. The GGUF conversion for BitNet uses round(w/scale).clamp(-1,1) with scale = mean|W|, so the zero threshold is |w| < 0.5 * mean|W|. A different scale gives a different zero fraction. The training makes the weights ternary-shaped; the converter chooses where to cut. If you have Microsoft's native quantization figure, we would like to compare - we could not find one.

Method, so it can be checked or falsified. The i2_s layout is read from the fork's own ggml.c: nbytes = nelements/4 + 32 - 4 weights per byte at 2 bits, 32 pad bytes, no interleaved scales. The Python gguf library refuses type 36 (np.uint32(36) is not a valid GGMLQuantizationType - the same type-36 collision as TQ1_0/TQ2_0), so we decode the codes directly. About 90 lines, stdlib only.

What would change our mind:

• if quantize_i2_s maps the codes differently, the labels move (the count stands either way)
• if Microsoft's native conversion lands near 33% zeros, then our 42% is a converter artifact and "majority state" weakens to "substantial"
• if the deciding/carrying split does not reproduce on other ternary models, it is one model's quirk

Script and the memory kernel it came out of: https://github.com/jlove2177-jpg/trit-kernel
Running 1.58-bit models on consumer AMD: https://github.com/jlove2177-jpg/rokom-setup
Where this fits: https://wedoc-ai.com - the parameters that don't have to think

Curious whether anyone has measured this, or knows why the zero overshoots the balanced third.