Instructions to use kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP # Run inference directly in the terminal: llama cli -hf kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP # Run inference directly in the terminal: llama cli -hf kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP # Run inference directly in the terminal: ./build/bin/llama cli -hf kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
Use Docker
docker model run hf.co/kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
- Llama new
- LM Studio
- Jan
- vLLM
How to use kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
- Ollama
How to use kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF with Ollama:
ollama run hf.co/kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
- Unsloth Desktop
- Pi
How to use kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF with Docker Model Runner:
docker model run hf.co/kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
- Lemonade
How to use kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
Run and chat with the model
lemonade run user.GLiDE-27B-ROCmFP4-STRIX-GGUF-Q4_0_ROCMFP
List all available models
lemonade list
- Hermes Agent
How to use kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF:Q4_0_ROCMFP" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
GLiDE-27B โ ROCmFP4 for AMD Strix Halo (gfx1151)
A ROCmFP4 STRIX quant of fastino/GLiDE-27B (Fastino Community License; a modified Qwen3.8-27B). Importance-matrix weighted; token embedding at Q6_K (embeddings are tied; there is no separate output.weight). Two Super ROC 4-bit prefill route files, rounded from this model's own BF16 weights, ship next to the GGUF.
Upstream GLiDE is a decision model: a 255-way readout scores typed questions in one forward pass. This GGUF is the text transformer only. The vision encoder and the decision readout from the upstream repo are not in this file. Native context in the GGUF is 262144; the serve command below uses 131072. There are no in-file nextn / MTP tensors. Serve it with the Qwen3.8-27B DFlash2 drafter.
Files
| File | Bytes | sha256 | Role |
|---|---|---|---|
GLiDE-27B-Q4_0_ROCMFP4_STRIX-imatrix-headQ6K.gguf |
14084639424 | 65297b09db336c35975cf696e786f51633c8c4e4736feb638a2be31a708ac504 |
model, recipe STRIX (ggml type 105), 4.39 BPW |
GLiDE-27B-Q4_0_ROCMFP4_STRIX-imatrix-headQ6K.route.a4w |
8640268824 | 747666a48996981685a57a5245bfa70e55da4d63d2b96f8e18a6d6fc5a278a95 |
4-bit prefill route (ffn_gate,ffn_up,attn_gate,attn_q,attn_qkv), clip 3, fine 64 |
GLiDE-27B-Q4_0_ROCMFP4_STRIX-imatrix-headQ6K.down-rot256.a4r |
2878866208 | b90bea9a0a91b612539c0484911cd835fbe7c622b64788f62fa87fca4dd114fb |
rotated ffn_down route, block 256, clip 3, fine 64 |
The route files belong to this GGUF only. The engine checks layer count and width, so using them with any other 27B file still loads and gives wrong answers.
Drafter (not in this repository): Qwen3.8-27B-DFlash2-Q4_K_M.gguf from z-lab/Qwen3.8-27B-DFlash2-GGUF.
How it was built
- Source:
fastino/GLiDE-27BBF16 safetensors, converted to a single BF16 GGUF (qwen35, 25,624,600,064 parameters, 850 tensors, 64 blocks, vocab 248320, tied embeddings). Visual tensors and the 255-way readout were not written into the GGUF. - Importance matrix: 531 chunks ร 512 tokens of the same agentic calibration text used for the Qwen3.8-27B STRIX file, computed on a Q8_0 of GLiDE (496 imatrix entries).
- Quantize:
llama-quantize --imatrix <that file> --output-tensor-type q6_K --token-embedding-type q6_K <BF16> <out> Q4_0_ROCMFP4_STRIX 16. Census: 850 tensors;token_embd.weightconverted to Q6_K; model 48880.09 MiB (16.00 BPW) โ 13421.67 MiB (4.39 BPW). Wall 188.5 s. Tensor types in the STRIX file: 416 ร ggml type 101, 80 ร type 100, 353 ร F32, 1 ร Q6_K. - Prefill route: Super ROC
int4_weightsfrom the GLiDE BF16 GGUF. Column scales dumped on this STRIX file (8K-token prompt, context 4096). Rotated-down scales dumped on the held-out quality text with the five-tensor route on andSUPERROC_INT4_DOWN_ROT=256.
Requires a runtime that loads ROCmFP4 (ggml types 100โ106). Stock llama.cpp rejects these tensor types.
Run it
SUPERROC_ATTN_FOLDED=1 SUPERROC_ATTN_VW=1 \
SUPERROC_INT4_WEIGHTS=/path/to/GLiDE-27B-Q4_0_ROCMFP4_STRIX-imatrix-headQ6K.route.a4w \
SUPERROC_INT4_TENSORS=ffn_gate,ffn_up,attn_gate,attn_q,attn_qkv \
SUPERROC_INT4_DOWN_WEIGHTS=/path/to/GLiDE-27B-Q4_0_ROCMFP4_STRIX-imatrix-headQ6K.down-rot256.a4r \
SUPERROC_INT4_DOWN_ROT=256 \
superroc serve llm \
-m /path/to/GLiDE-27B-Q4_0_ROCMFP4_STRIX-imatrix-headQ6K.gguf \
--served-model-name glide-27b \
--context 131072 --think off -t 0 --presence-penalty 0 \
--speculative dflash2 --dflash-model /path/to/Qwen3.8-27B-DFlash2-Q4_K_M.gguf \
--prompt-lookup
ROCm 7.2.4, LD_LIBRARY_PATH pointing at that ROCm lib. Thinking off. Presence penalty 0.
This GGUF is a next-token server of the text backbone. It does not expose GLiDE's native decision API.
Quality
Perplexity with Super ROC's qwen35_ppl on a held-out text, context 4096, 5 chunks, 10,235 scored tokens (second half of each chunk). Corpus sha256 167d6b72bcaf3e0139fa407889b7b1aeee18e304ec5b5ce6135efb669b9647f8. Lower is better. GLiDE's native head is a 255-way decision readout, not a generative lm_head, so absolute generative PPL is large; the useful figure is the ratio to this file's own Q8_0.
| File / route | PPL | vs Q8_0 |
|---|---|---|
| Q8_0 reference (not shipped) | 373673 | 1.000 |
| STRIX, five-tensor route on | 385580 | 1.0319 |
| STRIX, five-tensor route + rotated down + folded attention VW | 380893 | 1.0193 |
The 1.02 ร Q8_0 bar is 381146.46. The full stack (the serve command above) is 1.0193 ร Q8_0.
Chat completions on this GGUF return HTTP 200 with finish_reason=length; the decoded text is not coherent English (no decision readout, no generative training objective). A weather-tool request also returned HTTP 200 with finish_reason=length and no tool_calls. Second turn of an 1,802-token prefix reused 1,802 / 1,802 cached tokens; the follow-up turn reused 1,802 / 1,899.
Speed
AMD Ryzen AI Max+ 395 (Radeon 8060S, gfx1151), 128 GB unified, ROCm 7.2.4. Super ROC, the serve command above, temperature 0, cold prompt cache (cached_tokens=0 on every timed request), means of two runs.
| Measurement | Result |
|---|---|
| Prompt processing, 1,566-token prompt | 455.9 tok/s |
| Generation after that prompt (up to 256 tokens) | 17.7 tok/s |
| Draft tokens accepted on that prompt | 75 / 263 |
| Prompt processing, 13,384-token prompt | 820.85 tok/s |
| Generation after that prompt | 20.05 tok/s |
| Draft tokens accepted on that prompt | 99 / 212 |
| Prompt processing, 26,821-token prompt | 756.45 tok/s |
| Generation after that prompt | 18.0 tok/s |
| Draft tokens accepted on that prompt | 90 / 233 |
| Code generation, 6 prompts, up to 256 tokens, mean of two runs (range) | 20.73 tok/s (19.9โ22.4) |
| Draft tokens accepted on those code prompts | 1112 / 2554 |
Tool calls valid, tool_choice required / auto |
0/8 / 0/8 |
The DFlash2 drafter is the base Qwen3.8-27B file. Generation on this GGUF fills the 256-token cap; the table is the measured acceptance on this file.
Architecture
GLiDE-27B's text transformer is dense hybrid-attention qwen35: 64 layers, hidden 5120, FFN 17408, SSM inner 6144, vocab 248320, tied embeddings. 850 tensors. Native context in the GGUF is 262144; the serve command above uses 131072.
License
Base model fastino/GLiDE-27B, Fastino Community License (see LICENSE in this repository). GLiDE is a modified Qwen3.8-27B; Qwen3.8-27B remains Apache-2.0 (licenses/Qwen3.8-27B-LICENSE). This repository is the quantised weights, the two route files, and the measurements above. A quantized copy is a Derivative Work under the Fastino Community License: include this license with the files; commercial production use above $10,000,000 annual revenue needs a separate written agreement; commercial use requires the attribution "Built with Fastino GLiDE".
- Downloads last month
- 140
4-bit
Model tree for kingjones777/GLiDE-27B-ROCmFP4-STRIX-GGUF
Base model
fastino/GLiDE-27B