majentik/Qwen-Image-Bench-MLX-8bit

Qwen-Image-Bench - MLX 8-bit

8-bit weight-quantized MLX version of Qwen/Qwen-Image-Bench — Qwen's 27B vision-language judge for text-to-image evaluation (built on Qwen3.6-27B). It scores prompt/image pairs across 5 hierarchical dimensions (Quality, Aesthetics, Alignment, Real-world Fidelity, Creative Generation) and emits JSON-formatted results. The language tower is 8-bit quantized; the vision tower is kept in BF16 (unquantized) for judge fidelity. Smoke-verified on Apple Silicon with this exact payload before publishing (see PROVENANCE.md).

Approximate model size: ~30 GB

Model Specifications

Property	Value
Base Model	Qwen/Qwen-Image-Bench (Qwen3.6-27B based)
Parameters	27 billion (dense)
Architecture	qwen3_5 dense VLM (Qwen3_5ForConditionalGeneration)
Modality	Image + text input, text (JSON scores) output
License	Apache 2.0
Weight Quantization	8-bit affine, group size 64; vision tower BF16
Framework	MLX (Apple Silicon), mlx-vlm >= 0.6

Quickstart

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("majentik/Qwen-Image-Bench-MLX-8bit")

prompt = apply_chat_template(
    processor, model.config,
    "Evaluate this generated image against the prompt: 'a red square'. "
    "Score Quality, Aesthetics, Alignment, Real-world Fidelity, Creative "
    "Generation (0=Fail, 1=Pass, 2=Excel) as JSON.",
    num_images=1,
)
output = generate(model, processor, prompt, image=["generated.png"], max_tokens=512)
print(output.text)

For batch judging, see upstream's judge.py and structured checklists.

Variants in this family

Variant	Approx size	Use case
4bit	~17 GB	Balanced default
6bit	~23 GB	Higher judge fidelity
8bit(https://huggingface.co/majentik/Qwen-Image-Bench-MLX-8bit)	~30 GB	Reference fidelity

No sub-4bit variants are published: judge quality degrades at very low bit widths and a miscalibrated judge is worse than none. Smoke verification covers load + vision-grounded generation gates only; it is not a benchmark of judge calibration — validate against upstream's benchmark data before relying on absolute scores.