Instructions to use croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4
# Run inference directly in the terminal:
llama cli -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4
# Run inference directly in the terminal:
llama cli -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4
# Run inference directly in the terminal:
./llama-cli -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4
# Run inference directly in the terminal:
./build/bin/llama-cli -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

Use Docker

docker model run hf.co/croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

LM Studio
Jan

vLLM

How to use croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

Ollama
How to use croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 with Ollama:
```
ollama run hf.co/croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4
```

Unsloth Studio

How to use croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 to start chatting

How to use croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

Run Hermes

hermes

OpenClaw new

How to use croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 with Docker Model Runner:
```
docker model run hf.co/croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4
```

Lemonade

How to use croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4:NVFP4

Run and chat with the model

lemonade run user.Qwopus3.6-27B-v2-Abliterated-NVFP4-NVFP4

List all available models

lemonade list

Atomic Chat

Qwopus3.6-27B-v2-Abliterated — NVFP4 + native MTP draft

Abliterated, NVFP4-quantized vision–language model with the multi-token-prediction (MTP) head preserved as the speculative-decode drafter. Designed to run on NVIDIA Blackwell (RTX 50xx, GB10) under upstream ggml-org/llama.cpp stock (no custom fork).

Uncensored version of Jackrong/Qwopus3.6-27B-v2 — a Claude 4.6 Opus reasoning-distilled fine-tune of Unsloth's Qwen3.6-27B (text body + native MTP nextn head). Abliteration removes refusal behavior without retraining, using activation contrast on harmful vs. harmless prompts.

Abliteration Details

The current checkpoint is the result of a multi-stage de-alignment pipeline applied to the Jackrong/Qwopus3.6-27B-v2 base, plus a post-hoc drafter alignment pass.

Stage 1 — Sumandora-style abliteration (orthogonal projection)

Method: Refusal-direction ablation via activation contrast on paired harmful / harmless prompts.
Harmful prompts: 512 sampled from Sumandora's harmful.txt (520 prompts pool). HARMBENCH was the intended source but is gated; the script falls back to Sumandora.
Harmless prompts: 512 sampled from Sumandora's harmless.txt (31.8K Alpaca-cleaned pool).
Top-K directions: 3 layers selected by ‖mean_harmful − mean_harmless‖, Gram-Schmidt orthogonalized. Layers picked: 62 (norm 92.41), 61 (81.34), 60 (75.98).
Ablated body layers: 2..63 (62 layers; SKIP_BEGIN=2, SKIP_END=0 — final full-attention layer included).
Ablated write-targets per layer type:
- Full-attention layers (16/64): self_attn.o_proj, mlp.down_proj.
- Linear-attention layers (48/64): linear_attn.out_proj, mlp.down_proj.
MTP head (blk.64): also abliterated using the same 3 directions, applied to self_attn.o_proj and mlp.down_proj.
Total ablation operations: 378 (124 body matrices + 2 MTP matrices = 126 unique tensors, each ablated by 3 orthogonal directions).
Vision tower: NOT touched.

Stage 2 — LoRA SFT + DPO de-alignment

SFT corpus: NobodyExistsOnTheInternet/ToxicQAFinal (6,866 examples, 1 epoch). Compliant detailed answers paired with harmful prompts — teaches the model to answer instead of refusing.
DPO corpus: unalignment/toxic-dpo-v0.2 (541 preference pairs, 1 epoch). chosen = compliant answer, rejected = refusal.
LoRA: r=16, α=32, dropout=0.05.
Target modules: (q,k,v,o)_proj, out_proj, (gate,up,down)_proj on text body layers (language_model.layers.*). Vision tower (visual.blocks.*) explicitly excluded via regex filter.
Trainable params: 88.34M / 27.45B (0.322%).
Optimizer: AdamW BF16, gradient checkpointing, effective batch 8 (per-device 2 × grad-accum 4), max_seq_len 1024.
Compute: 1× NVIDIA H200 SXM 141GB, ~48 min total (SFT 41 min + DPO 4 min + setup overhead) with flash-linear-attention + causal-conv1d + tilelang fast path.
Final DPO stats: train_loss 0.034, rewards/margins 14.4, rewards/accuracies 1.000 (100% preference accuracy on training set).

Stage 3 — Post-hoc drafter alignment

The HF wrapper Qwen3_5ForConditionalGeneration does not expose the MTP head as nn.Module, so PEFT could not train LoRA on it directly in the standard run. After body merge, the LoRA delta from body layer 63 (last full-attention layer, structurally matched to MTP) was projected onto the corresponding MTP weights via δ = (B @ A) × (α/r), applied to q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj. This is a heuristic but partially restores drafter / verifier distribution match. The MTP head retains its Stage 1 abliteration plus this projected delta.

What this means in practice

The model rarely refuses on cybersec, social engineering, chemistry, and other subtle-harmful categories that the Jackrong distillation would normally refuse.
Identity ("I am Qwen") is restored — the Claude-distillation identity artifact from the base is gone.
Multimodal capability is preserved (vision tower never touched).
Some MTP speculative-decode acceptance is lost relative to the pre-LoRA base; the decode-speed trade-off is detailed in the Benchmarks section below.

Model Details

Property	Value
Base model	Jackrong/Qwopus3.6-27B-v2 (Unsloth/Qwen3.6-27B + MTP)
Architecture class	`Qwen3_5ForConditionalGeneration` (`qwen35` in GGUF) — text body + vision tower + MTP nextn head
Text parameters	~28 B (64 hidden layers, hidden 5120, head_dim 256, 24 attn heads, 4 KV heads)
MTP nextn layers	1 (block 64)
Vision tower	Qwen3.5 vision (depth 27, hidden 1152, out_hidden 5120)
Context length	262 144 tokens (configurable up to 524 288)
Quantization	Plain NVFP4 (NVFP4_DEFAULT_CFG, no AWQ pre-quant scale) for text body; BF16 for MTP head; F16 for vision projector
Format	GGUF + (optional) abliterated BF16 safetensors
License	Apache 2.0

Files

File	Size	Download	Notes
`Qwopus3.6-27B-v2-Abliterated-NVFP4.gguf`	~19 GB	⬇ download	Main model — NVFP4 text body + BF16 MTP head + metadata for `--spec-type draft-mtp`. Use with upstream `ggml-org/llama.cpp` ≥ 2026-05-23 (PR #23563).
`Qwopus3.6-27B-v2-Abliterated-mmproj-F16.gguf`	~885 MB	⬇ download	Vision projector (F16). Required for `--mmproj` if you want to send images.

Quick CLI download (with huggingface_hub):

hf download croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4 \
  Qwopus3.6-27B-v2-Abliterated-NVFP4.gguf \
  Qwopus3.6-27B-v2-Abliterated-mmproj-F16.gguf \
  --local-dir ./qwopus36-v2-abl

Quick start with `llama.cpp` (upstream stock)

The native MTP drafter (--spec-type draft-mtp) is in upstream ggml-org/llama.cpp since 2026-04 (PR #22673). NVFP4 MTP scale-tensor handling landed on 2026-05-23 (PR #23563). Build accordingly:

git clone https://github.com/ggml-org/llama.cpp ~/llama-cpp-stock
cd ~/llama-cpp-stock
cmake -B build \
  -DGGML_CUDA=ON \
  -DCMAKE_CUDA_ARCHITECTURES="120;121" \
  -DGGML_CUDA_FA_ALL_QUANTS=ON \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-server llama-quantize llama-cli -j$(nproc)

CMAKE_CUDA_ARCHITECTURES="120;121" covers RTX 5090 (SM 12.0) and GB10 (SM 12.1) with native FP4 MMA. Drop 121 if you only target consumer Blackwell.

`llama-server` — production setup (port 30000)

./build/bin/llama-server \
  -m Qwopus3.6-27B-v2-Abliterated-NVFP4.gguf \
  --mmproj Qwopus3.6-27B-v2-Abliterated-mmproj-F16.gguf \
  --spec-type draft-mtp --spec-draft-n-max 5 \
  --host 0.0.0.0 --port 30000 \
  -c 262144 -np 2 -ngl 99 \
  -ctk q8_0 -ctv q8_0 \
  --slot-prompt-similarity 0.5 --cache-reuse 256 --cache-ram 16384 \
  -bs \
  --jinja --chat-template-file chat_template.jinja \
  --no-prefill-assistant --reasoning off \
  --alias dark-jarvis --alias dark-opus --no-webui --no-warmup

Key flags explained:

Flag	Why
`--spec-type draft-mtp`	Enables native MTP speculative decode using `blk.64` of the GGUF as drafter. No external drafter model needed.
`--spec-draft-n-max 5`	Tokens proposed per spec step. 5 is the sweet spot for mixed text/code workloads (see benchmarks). For pure long-context vision drop to 3.
`-c 262144 -np 2`	Total KV budget 262 K split across 2 parallel slots = 131 K per slot.
`-ctk q8_0 -ctv q8_0`	KV cache quantized to q8 — best quality / size on Blackwell.
`--cache-ram 16384`	Up to 16 GiB of host RAM used as warm prompt-prefix cache (massive TTFT improvement on repeated system prompts).
`-bs`	Backend sampling (experimental, +5–10 % throughput on Blackwell).
`--reasoning off`	Skips the `<think>` block from the chat template; ~12 % faster end-to-end on tool/agent workloads. Remove if you want explicit reasoning chains.
`--alias dark-jarvis --alias dark-opus`	OpenAI-API model aliases.

Inference temperature

Use temperature = 0.8 for general chat / reasoning / writing. The model is a reasoning-distilled fine-tune and tolerates higher temperatures than typical instruct models; below 0.5 it gets noticeably stiff. For deterministic tool/JSON output, use temperature = 0.0 + --json-schema.

Example OpenAI-compatible call:

curl http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dark-jarvis",
    "messages": [{"role": "user", "content": "Explain reflection in 5 lines."}],
    "temperature": 0.8,
    "max_tokens": 500
  }'

`llama-cli` (one-shot)

./build/bin/llama-cli \
  -m Qwopus3.6-27B-v2-Abliterated-NVFP4.gguf \
  --mmproj Qwopus3.6-27B-v2-Abliterated-mmproj-F16.gguf \
  --spec-type draft-mtp --spec-draft-n-max 5 \
  -ngl 99 -c 8192 -ctk q8_0 -ctv q8_0 -bs \
  --jinja --temp 0.8 \
  -p "Hello"

Benchmarks (NVIDIA GB10, SM 12.1, 128 GiB unified mem)

Measured on dark-jarvis.service (this model, upstream stock, n_max=5, cache-ram 16 GiB, KV q8_0/q8_0, reasoning off). Real Hermes-agent workload over ~45 min (Markdown, code, JSON, structured writing, multimodal vision capture).

Decode throughput by workload type (single slot)

Workload	Decode tok/s	MTP acceptance
Code / skill writing / structured Markdown	20 – 30	65 – 90 %
Reasoning / prose / Italian explanations	13 – 18	50 – 70 %
Long-context Q&A (≥ 30 K tokens)	13 – 15	40 – 50 %
Vision-heavy (browser screenshots → extraction)	11 – 14	30 – 50 %
2 slots concurrent (per-slot, bandwidth split)	6 – 9	unchanged

Peak sustained: 23.58 tok/s for 1 609 generated tokens, acceptance 85.6 % (code generation, single slot).

Prefill throughput

Cold prefill (no cache): ~700 – 880 tok/s on 2 – 50 K-token prompts.
Warm prefill (system prompt cached, only the new user message processed): TTFT ≈ 0.4 – 1.5 s regardless of system-prompt size.

Memory footprint (stable, no leak)

Component	Value
GPU compute pool (unified, model + KV + MTP draft + scratch)	30.07 GB (fixed, pre-allocated, does not grow)
Process RSS (heap + mmap + lazy `--cache-ram` 16 GiB)	4 – 28 GB (oscillates with workload, reclaimable)
Total `MemTotal − MemAvailable` (`free -h` view)	~44 – 67 GB

The GPU pool stays flat over hours of sustained traffic — no draft KV leak.

Reproducing the build

The text body uses plain NVFP4 (NVFP4_DEFAULT_CFG, no AWQ pre-quant scale) via NVIDIA ModelOpt, then the BF16 MTP-head weights from the original Jackrong checkpoint are merged into the same directory and a stock convert_hf_to_gguf.py produces the final GGUF.

High-level pipeline:

# 1. Source
hf download Jackrong/Qwopus3.6-27B-v2 --local-dir ~/qwopus36-v2-bf16

# 2. Abliterate (skip MTP head). See abliterate_qwopus_v2_mtp.py in repo.
python3 abliterate_qwopus_v2_mtp.py ~/qwopus36-v2-bf16 ~/qwopus36-v2-mtp-abl

# 3. NVFP4 quantize text body (HF does not expose MTP head to ModelOpt — that's fine)
python3 quantize_nvfp4_plain.py ~/qwopus36-v2-mtp-abl ~/qwopus36-v2-mtp-abl-nvfp4

# 4. Re-inject BF16 MTP weights from the original safetensors
python3 inject_mtp.py ~/qwopus36-v2-bf16 ~/qwopus36-v2-mtp-abl-nvfp4

# 5. Convert to GGUF with upstream stock script (PR #23563+)
cd ~/llama-cpp-stock
python3 convert_hf_to_gguf.py ~/qwopus36-v2-mtp-abl-nvfp4 \
  --outfile ~/qwopus36-v2-mtp-abl-nvfp4/Qwopus3.6-27B-v2-Abliterated-NVFP4.gguf

# 6. mmproj from BF16 source (vision tower unchanged by abliteration)
python3 convert_hf_to_gguf.py ~/qwopus36-v2-bf16 --mmproj --outtype f16 \
  --outfile ~/qwopus36-v2-mtp-abl-nvfp4/Qwopus3.6-27B-v2-Abliterated-mmproj-F16.gguf

Full reproduction guide: croll83/jarvis/infrastructure/gb10/dark-jarvis.md.

Important Disclaimers

This model has reduced safety filtering and may generate content that is sensitive, controversial, or potentially harmful.

Intended for research and experimental use only.
Not suitable for public-facing applications or use by minors.
User is solely responsible for legal and ethical compliance.
No default safety guarantees are provided.
Use at your own risk and discretion.

Credits

Base model: Jackrong/Qwopus3.6-27B-v2
Foundation: Unsloth/Qwen3.6-27B → Qwen/Qwen3.5
Abliteration technique: Sumandora/remove-refusals-with-transformers
Harmful prompts: walledai/HarmBench
Harmless prompts: Sumandora's harmless.txt (sampled from Alpaca-cleaned)
MTP spec-decode in llama.cpp: PR #22673, PR #23461 (server VRAM fix), PR #23563 (NVFP4 MTP scale tensors)
NVFP4 quantization: NVIDIA ModelOpt
Inspiration: HuiHui-AI, z-lab/DFlash

Downloads last month: 1,764

Safetensors

Model size

17B params

Tensor type

BF16

F8_E4M3

Model tree for croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4

Base model

Jackrong/Qwopus3.6-27B-v2

Quantized

(59)

this model

Finetunes

1 model

Qwopus3.6-27B-v2-Abliterated — NVFP4 + native MTP draft

Abliteration Details

Stage 1 — Sumandora-style abliteration (orthogonal projection)

Stage 2 — LoRA SFT + DPO de-alignment

Stage 3 — Post-hoc drafter alignment

What this means in practice

Model Details

Files

Quick start with llama.cpp (upstream stock)

llama-server — production setup (port 30000)

Inference temperature

llama-cli (one-shot)

Benchmarks (NVIDIA GB10, SM 12.1, 128 GiB unified mem)

Decode throughput by workload type (single slot)

Prefill throughput

Memory footprint (stable, no leak)

Reproducing the build

Important Disclaimers

Credits

Model tree for croll83/Qwopus3.6-27B-v2-Abliterated-NVFP4

Quick start with `llama.cpp` (upstream stock)

`llama-server` — production setup (port 30000)

`llama-cli` (one-shot)