AX-Qwen3.5-9B-CUDA-AXQ-NVFP4-W4A16-MTP

Experimental AXQuant NVFP4 W4A16 checkpoint, exported using the AXQuant 2.2.1 encoder from Qwen/Qwen3.5-9B. Source revision: c202236235762e1c871ad0ccb60c8ee5ba337b9a.

Storage and precision

AXQuant-owned RTN encoder; no activation calibration or measured sensitivity. Packed E2M1 weights, E4M3 scales per 16 values, FP32 inverse global scale. Activations stay BF16. The container uses compressed-tensors NVFP4A16. Conversion backend: numpy-reference (CPU reference encoder on Mac Studio M2 Ultra 192 GB); a CUDA pack name describes the intended runtime format, not the conversion device.

248 language attention/MLP matrices are packed; 527 other tensors match the source exactly. Vision, embeddings, normalization, output head, and packaged MTP weights are protected. The artifact manifest binds every exported weight and configuration file; static-validation.json records the complete protected-tensor comparison and bounded row sampling.

Runtime status

Text-only load and bounded greedy generation passed on NVIDIA GeForce RTX 5090 with vLLM 0.31.0, PyTorch 2.13.0+cu130 with cpu_offload_gb=12.0 configured (see runtime-smoke.json). W4A16 uses a weight-only Marlin kernel with BF16 activations, not FP4 activation arithmetic. This is a functional smoke, not a quality or performance qualification. Reference BF16 reconstruction and text generation passed with PyTorch CPU on Mac Studio M2 Ultra 192 GB; see reference-runtime-smoke.json. This does not verify native CUDA or FP4 execution.

The -MTP suffix means the original MTP tensors and configuration are packaged and preserved. MTP decoding is not verified or certified; keep speculative decoding disabled.

The manifest intentionally remains status=development, runtime_verified=false, quality_certified=false. Any separate smoke receipt applies only to its exact manifest digest. Vision and audio generation, long context, numerical parity, task quality, speed, and memory savings during execution have not been qualified. This is not an AX Engine MLX pack.

Licensing

The model weights retain the upstream Apache-2.0 license; see LICENSE and NOTICE. Preserve any included upstream usage policy. The AXQuant toolkit license does not replace the source model license.

Downloads last month
24
Safetensors
Model size
10B params
Tensor type
BF16
·
U8
·
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AutomatosX/AX-Qwen3.5-9B-CUDA-AXQ-NVFP4-W4A16-MTP

Finetuned
Qwen/Qwen3.5-9B
Quantized
(740)
this model

Collections including AutomatosX/AX-Qwen3.5-9B-CUDA-AXQ-NVFP4-W4A16-MTP