blackfan23/sabi-noema-sharp

Sabi-Noema MLX 4-bit (Affine) — Sharp Variant

This repository contains Apple MLX 4-bit affine quantized weights for sabicap/sabi-noema configured with the Sharp chat template from peculiar-ragdoll/Qwen-Sharp-Chat-Templates (rebased on froggeric v22.5).

What is the Sharp Template?

The Sharp template optimizes Qwen-family models for coding and knowledge work:

• Reduces thinking bloat: Models reach correct answers with fewer thinking tokens and less filler without sacrificing depth or reasoning rigor.
• Fast Problem Resolution: Demonstrated in SWE-bench / ClawEval benchmarks to reach solutions significantly faster.
• Full Tooling & Thinking Support: Supports thinking on/off (enable_thinking), reasoning efforts (reasoning_effort), and Qwen tool calling schemas.
• Configured Everywhere: Replaced both in chat_template.jinja and embedded directly in tokenizer_config.json for compatibility across transformers, MLX, vLLM, and llama.cpp.

Quantization Details

• Base model: sabicap/sabi-noema
• Quantization format: Apple MLX (mlx.core.quantize)
• Bits: 4-bit
• Mode: affine
• Group size: 64
• Quantized weight size: 16.22 GB
• Linear layers quantized: 581
• Preserved layers: Token embeddings, 1D normalization weights, and conv1d kernels

Usage

MLX LM / Apple Silicon

You can run this model directly on Apple Silicon with MLX:

from mlx_lm import load, generate

model, tokenizer = load("blackfan23/sabi-noema-sharp")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Write a Python script to monitor network latency."}],
    tokenize=False,
    add_generation_prompt=True
)
response = generate(model, tokenizer, prompt=prompt, max_tokens=2048)
print(response)

Converted and uploaded by Julius (@blackfan23).