Instructions to use drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated

SGLang

How to use drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated with Docker Model Runner:
```
docker model run hf.co/drowzeys/keys-DeepSeek-V4.1-Flash-1017-Abliterated
```

2x DGX Spark (Native Zig 1M TP=2) benchmark receipt: 0% refusal and zero coding regression vs stock LibertAI 2.9bpw

by joelafrite - opened 13 days ago

Discussion

joelafrite

13 days ago

Sharing controlled head-to-head benchmark numbers from a 2x DGX Spark (GB10 sm_121a, TP=2 over 200GbE CX-7 RoCE) comparing stock LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw against the same checkpoint with exl3_wo_b_l10_35_mia29.safetensors (78 attn.wo_b tensors in layers 10-35 swapped in-place inside the Native Zig TFDSRK01 rank cache, 1,048,576 shared token pool across 4 slots, lossless MXFP8 Engram).

1. Throughput, Speculative Acceptance & Refusal Battery

Metric	Stock `LibertAIDAI 2.9bpw` (`ABLIT=0`)	`+ drowzeys/keys` (`exl3_wo_b_l10_35_mia29`, `ABLIT=1`)
Pure code decode (`tok/s`)	`93.0 - 103.0`	`104.3`
Pure prose decode (`tok/s`)	`~50.0`	`49.2`
MTP speculative acceptance (`tok/round`)	`2.74 - 3.10` (`mean_na = 1.74 - 2.10`)	`2.48 - 2.92` (`mean_na = 1.48 - 1.92`)
HarmBench 50-prompt refusal rate	`42 / 50` (`84%`)	`0 / 50` (`0%`)
XSTest 50-prompt benign over-refusal	`3 / 50` (`6%`)	`0 / 50` (`0%`)

Because only layers.10..35.attn.wo_b are replaced and the MTP draft module is untouched, speculative decoding acceptance and throughput stay at parity with stock.

2. Real-Repository Multi-File Coding Benchmarks (`86k - 238k` Context)

A common concern with abliterated checkpoints is silent degradation on long-context multi-file coding, tool calling, or test writing. We ran both arms through our real-repository coding harness (OpenCode / OpenChamber) on two multi-file Python + JS codebases:

Task A — 2-Phase Blind Repository Audit & 4-Step Feature/Bug Implementation (542-test Python + JS web codebase):
- Phase 1 (Read-only blind audit against 10 ground-truth bugs): keys (#low effort) independently booted a temporary server, executed all 14 .mjs Playwright browser contract suites + the 542-test Python suite, and identified 8.5 / 10 canonical bugs (stock #high scored 10.0 / 10; Qwen3.8-Flash-Next #medium scored 9.0 / 10; GLM-5.3-Flash #high scored 4.5 / 10).
- Phase 2 (4-Step multi-file implementation + 4x4 cross-model unit-test matrix): Every model's production patch was tested not only against its own test suite (48/48 PASS) but against the unit tests written by all 4 competing models. keys (#low, 39.57 min total wall time, 58.9 tok/s E2E across 86.8k generated tokens) scored 10.0 / 10 (Rank #1 overall) — passing 100% of all cross-model edge-case tests while preserving backward compatibility with legacy fixtures.
Task B — Multi-File Backlog Implementation & Test Verification (~700 pytest + 110 JS regression checks):
- keys (#low): Rank #1 fastest overall (11.37 min wall time, 52.3 tok/s E2E, 27.9k generated tokens, 110.9k max context, 43 turns, 23.3% batched tool calls, 702 pytest PASS).
- keys (#high): Most comprehensive scope (13 files modified, +300/-35, 96.0k generated tokens, 238.1k max context, 48.9 tok/s E2E, all unit + smoke + regression suites PASS).

Thanks for publishing the standalone exl3_wo_b_l10_35_mia29.safetensors sidecar — being able to toggle stock <-> keys in <1s on the rank cache without storing a second 200 GB checkpoint is a huge quality-of-life win on 2x Sparks.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

2x DGX Spark (Native Zig 1M TP=2) benchmark receipt: 0% refusal and zero coding regression vs stock LibertAI 2.9bpw

1. Throughput, Speculative Acceptance & Refusal Battery

2. Real-Repository Multi-File Coding Benchmarks (86k - 238k Context)

2. Real-Repository Multi-File Coding Benchmarks (`86k - 238k` Context)