Text Generation
Transformers
Safetensors
GGUF
English
qwen3_5
image-text-to-text
decision-model
typed-decisions
calibration
calibrated-probabilities
classification
tool-selection
agent-routing
decision-index
jevbench
jev-compatible
systemone
wald
wald-q4b
qwen3.5
4b
vllm
reasoning
llama.cpp
conversational
Eval Results (legacy)
Instructions to use org2ai/Wald-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use org2ai/Wald-4B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="org2ai/Wald-4B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("org2ai/Wald-4B") model = AutoModelForMultimodalLM.from_pretrained("org2ai/Wald-4B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use org2ai/Wald-4B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "org2ai/Wald-4B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "org2ai/Wald-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/org2ai/Wald-4B
- SGLang
How to use org2ai/Wald-4B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "org2ai/Wald-4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "org2ai/Wald-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "org2ai/Wald-4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "org2ai/Wald-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use org2ai/Wald-4B with Docker Model Runner:
docker model run hf.co/org2ai/Wald-4B
Model card: clearer summary, FAQ, related projects, JevBench result, model-index; add API reference, CITATION.cff, llms.txt
Browse files- CITATION.cff +28 -0
- MANIFEST.json +18 -6
- README.md +173 -57
- docs/api.md +148 -0
- docs/readmes/README.zh.md +110 -50
- llms.txt +31 -0
- model-info.json +32 -3
CITATION.cff
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
cff-version: 1.2.0
|
| 2 |
+
message: "If you use Wald-Q4B, please cite it as below."
|
| 3 |
+
type: software
|
| 4 |
+
title: "Wald-Q4B v1.1: an open-weight 4B decision model with calibrated option probabilities"
|
| 5 |
+
abstract: >-
|
| 6 |
+
Wald-Q4B is an open-weight 4B decision model built on Qwen3.5-4B-Base. Given a
|
| 7 |
+
state and typed questions (choice, yes/no, score), it returns a calibrated
|
| 8 |
+
probability for every option through a Jev-compatible POST /v1/systemone API,
|
| 9 |
+
in one pass or after an optional short thought. Weights and serving code are
|
| 10 |
+
Apache-2.0. Independent project; not affiliated with TypeSafe AI.
|
| 11 |
+
authors:
|
| 12 |
+
- name: "Wald-4B authors"
|
| 13 |
+
alias: Harry19081
|
| 14 |
+
website: "https://huggingface.co/Harry19081"
|
| 15 |
+
version: "v1.1"
|
| 16 |
+
date-released: "2026-09-29"
|
| 17 |
+
license: Apache-2.0
|
| 18 |
+
url: "https://huggingface.co/Harry19081/Wald-4B"
|
| 19 |
+
repository-code: "https://github.com/Harry19081/wald-4b"
|
| 20 |
+
repository-artifact: "https://huggingface.co/Harry19081/Wald-4B"
|
| 21 |
+
keywords:
|
| 22 |
+
- decision model
|
| 23 |
+
- calibrated probabilities
|
| 24 |
+
- tool selection
|
| 25 |
+
- agent routing
|
| 26 |
+
- classification
|
| 27 |
+
- Jev-compatible API
|
| 28 |
+
- Qwen3.5
|
MANIFEST.json
CHANGED
|
@@ -20,8 +20,8 @@
|
|
| 20 |
"sha256": "953551e746a887044cc0ce65dec9b96ea3a763d69fda7d3a296f524bb37bca19"
|
| 21 |
},
|
| 22 |
"README.md": {
|
| 23 |
-
"bytes":
|
| 24 |
-
"sha256": "
|
| 25 |
},
|
| 26 |
"RUNBOOK.md": {
|
| 27 |
"bytes": 2718,
|
|
@@ -44,8 +44,8 @@
|
|
| 44 |
"sha256": "e385b92e333eaaa9395a09daf26c09420645bee242456552a55500db45919bc7"
|
| 45 |
},
|
| 46 |
"docs/readmes/README.zh.md": {
|
| 47 |
-
"bytes":
|
| 48 |
-
"sha256": "
|
| 49 |
},
|
| 50 |
"evaluation/benchmark-summary.json": {
|
| 51 |
"bytes": 36014,
|
|
@@ -220,7 +220,19 @@
|
|
| 220 |
"sha256": "ad20de47de5bbe7f72417f3d87e6f72aae7987fab913eb814b818e4a4bbc6e1e"
|
| 221 |
},
|
| 222 |
"model-info.json": {
|
| 223 |
-
"bytes":
|
| 224 |
-
"sha256": "
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 225 |
}
|
| 226 |
}
|
|
|
|
| 20 |
"sha256": "953551e746a887044cc0ce65dec9b96ea3a763d69fda7d3a296f524bb37bca19"
|
| 21 |
},
|
| 22 |
"README.md": {
|
| 23 |
+
"bytes": 16370,
|
| 24 |
+
"sha256": "0964da2ec2e05dfa5f46b0008c1fe912f5dddf50a5876cc8970dd3b2baa67cf9"
|
| 25 |
},
|
| 26 |
"RUNBOOK.md": {
|
| 27 |
"bytes": 2718,
|
|
|
|
| 44 |
"sha256": "e385b92e333eaaa9395a09daf26c09420645bee242456552a55500db45919bc7"
|
| 45 |
},
|
| 46 |
"docs/readmes/README.zh.md": {
|
| 47 |
+
"bytes": 13881,
|
| 48 |
+
"sha256": "bc2d14f2e64189657b2142fb19e54d94215be4751fcf30113a7cd30c37c40ef8"
|
| 49 |
},
|
| 50 |
"evaluation/benchmark-summary.json": {
|
| 51 |
"bytes": 36014,
|
|
|
|
| 220 |
"sha256": "ad20de47de5bbe7f72417f3d87e6f72aae7987fab913eb814b818e4a4bbc6e1e"
|
| 221 |
},
|
| 222 |
"model-info.json": {
|
| 223 |
+
"bytes": 3436,
|
| 224 |
+
"sha256": "4afd88d67874d61ddea72e4ceaf6e0ace4199f2ef5f87cb00fe5df4abc87f00b"
|
| 225 |
+
},
|
| 226 |
+
"llms.txt": {
|
| 227 |
+
"bytes": 2537,
|
| 228 |
+
"sha256": "a814eeca0112086b1740378a16173b27c86c240c5222abaaf7fa33287bdd71f1"
|
| 229 |
+
},
|
| 230 |
+
"CITATION.cff": {
|
| 231 |
+
"bytes": 1075,
|
| 232 |
+
"sha256": "4fab11f4e20a79e69a4ebbf23573abe3a8cd9a374bd5ab11e7e4206a4f070bf1"
|
| 233 |
+
},
|
| 234 |
+
"docs/api.md": {
|
| 235 |
+
"bytes": 6085,
|
| 236 |
+
"sha256": "e2e349bc9d17f01e0d52835f664808f94a6d9c88dd120ba49c9f25af47c235b8"
|
| 237 |
}
|
| 238 |
}
|
README.md
CHANGED
|
@@ -3,40 +3,131 @@ license: apache-2.0
|
|
| 3 |
base_model: Qwen/Qwen3.5-4B-Base
|
| 4 |
base_model_relation: finetune
|
| 5 |
library_name: transformers
|
|
|
|
| 6 |
language:
|
| 7 |
- en
|
| 8 |
tags:
|
| 9 |
- decision-model
|
|
|
|
| 10 |
- calibration
|
| 11 |
-
-
|
| 12 |
-
-
|
| 13 |
-
- wald-q4b
|
| 14 |
-
- jev
|
| 15 |
-
- decision-making
|
| 16 |
- tool-selection
|
|
|
|
| 17 |
- agent-routing
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
- reasoning
|
| 19 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
---
|
| 21 |
|
| 22 |
<div align="center">
|
| 23 |
<h1>Wald-Q4B v1.1</h1>
|
| 24 |
<p><strong>Decide directly. Think when needed. Return probabilities.</strong></p>
|
| 25 |
-
<p><a href="
|
| 26 |
</div>
|
| 27 |
|
| 28 |
-
**Wald-Q4B v1.1 is a self-hosted,
|
| 29 |
|
| 30 |
-
|
| 31 |
|
| 32 |
-
|
| 33 |
|
| 34 |
## At a glance
|
| 35 |
|
| 36 |
-
- **4B parameters**, built on Qwen3.5-4B-Base; BF16 weights.
|
| 37 |
-
- **
|
| 38 |
-
- **
|
| 39 |
-
- **
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
## Thinking effort
|
| 42 |
|
|
@@ -50,7 +141,7 @@ Start with `none` for direct decisions, `medium` for confidence-gated thinking,
|
|
| 50 |
| **`high` (default)** | Every eligible question | Up to 512 tokens |
|
| 51 |
| `high-k2` … `high-k8` | Multiple thoughts; average their answer distributions | Up to 512 tokens per thought |
|
| 52 |
|
| 53 |
-
Thinking applies to questions with 2–26 options when context space permits. Larger option sets use grouped readout and a final winner comparison; if a thought cannot fit, the initial answer is kept. Increasing effort spends more computation; it does not guarantee a better answer. **54.59 applies to `high` only.**
|
| 54 |
|
| 55 |
Set the server default with `EFFORT=medium ./run.sh "$PWD"`, or override it per request with `"effort": "none"`.
|
| 56 |
|
|
@@ -58,64 +149,73 @@ Set the server default with `EFFORT=medium ./run.sh "$PWD"`, or override it per
|
|
| 58 |
|
| 59 |
Wald first reads option-letter logits from a plain prompt and turns them into a probability distribution. If the effort policy asks for thinking, it generates a short thought and reads the options again. Bucketed temperature scaling calibrates the returned probabilities.
|
| 60 |
|
| 61 |
-
v1.1 combines full-parameter decision training, LoRA refinement, short-thought distillation and RLCD. Training uses **WaldGen**, our generated decision corpus, together with public training datasets. [Data sources](PROVENANCE.md) · [Evaluation notes](CONTAMINATION.md)
|
| 62 |
|
| 63 |
## Benchmarks
|
| 64 |
|
| 65 |
-
| Benchmark | Configuration | Result |
|
| 66 |
-
|---|---|---:|
|
| 67 |
-
| **Decision Index 0.2.1 — complete suite** | v1.1 · `high` | **54.59** |
|
|
|
|
| 68 |
|
| 69 |
-
150,317/150,317 requests succeeded, including HLE. Measured on one RTX PRO 6000 96 GB with the pinned reproduction kit.
|
| 70 |
|
| 71 |
-
|
| 72 |
|
| 73 |
### Latency
|
| 74 |
|
| 75 |
-
A 32-request serial preflight of
|
| 76 |
|
| 77 |
-
##
|
| 78 |
|
| 79 |
-
|
| 80 |
|
| 81 |
-
``
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
./run.sh "$PWD" # default: high
|
| 85 |
-
# EFFORT=medium ./run.sh "$PWD" # confidence-gated thinking
|
| 86 |
-
```
|
| 87 |
|
| 88 |
-
|
| 89 |
-
curl http://localhost:8000/v1/systemone \
|
| 90 |
-
-H 'Content-Type: application/json' \
|
| 91 |
-
-d '{
|
| 92 |
-
"state": "The customer wants to return a damaged kettle.",
|
| 93 |
-
"effort": "medium",
|
| 94 |
-
"questions": {
|
| 95 |
-
"route": {
|
| 96 |
-
"type": "choice",
|
| 97 |
-
"instructions": "Choose the support queue.",
|
| 98 |
-
"criteria": {
|
| 99 |
-
"returns": "Returns and refunds",
|
| 100 |
-
"delivery": "Delivery tracking",
|
| 101 |
-
"other": "Other enquiries"
|
| 102 |
-
}
|
| 103 |
-
}
|
| 104 |
-
}
|
| 105 |
-
}'
|
| 106 |
-
```
|
| 107 |
|
| 108 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 109 |
|
| 110 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 111 |
|
| 112 |
-
|
| 113 |
|
| 114 |
-
|
| 115 |
|
| 116 |
-
**
|
| 117 |
|
| 118 |
-
**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 119 |
|
| 120 |
## Versioning
|
| 121 |
|
|
@@ -124,8 +224,24 @@ Confidence is not a guarantee of correctness; validate thresholds on your own ta
|
|
| 124 |
| **v1.1 — current** | `022D0-f7` | `high` |
|
| 125 |
| [v1.0 — archive](https://huggingface.co/Harry19081/Wald-4B/tree/v1.0) | `021A0-f10` | `medium` |
|
| 126 |
|
| 127 |
-
The v1.0 model card retains its XL, task-LoRA and latency reports. Those measurements belong to their documented builds.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 128 |
|
| 129 |
---
|
| 130 |
|
| 131 |
-
[
|
|
|
|
| 3 |
base_model: Qwen/Qwen3.5-4B-Base
|
| 4 |
base_model_relation: finetune
|
| 5 |
library_name: transformers
|
| 6 |
+
pipeline_tag: text-generation
|
| 7 |
language:
|
| 8 |
- en
|
| 9 |
tags:
|
| 10 |
- decision-model
|
| 11 |
+
- typed-decisions
|
| 12 |
- calibration
|
| 13 |
+
- calibrated-probabilities
|
| 14 |
+
- classification
|
|
|
|
|
|
|
|
|
|
| 15 |
- tool-selection
|
| 16 |
+
- tool-use
|
| 17 |
- agent-routing
|
| 18 |
+
- clarification
|
| 19 |
+
- decision-index
|
| 20 |
+
- jevbench
|
| 21 |
+
- jev
|
| 22 |
+
- jev-compatible
|
| 23 |
+
- open-jev
|
| 24 |
+
- typesafe-compatible
|
| 25 |
+
- systemone
|
| 26 |
+
- kev
|
| 27 |
+
- laya
|
| 28 |
+
- wald
|
| 29 |
+
- wald-q4b
|
| 30 |
+
- qwen3.5
|
| 31 |
+
- 4b
|
| 32 |
+
- vllm
|
| 33 |
- reasoning
|
| 34 |
+
model-index:
|
| 35 |
+
- name: Wald-Q4B v1.1
|
| 36 |
+
results:
|
| 37 |
+
- task:
|
| 38 |
+
type: text-classification
|
| 39 |
+
name: Structured decisions (option probabilities)
|
| 40 |
+
dataset:
|
| 41 |
+
name: Decision Index 0.2.1, complete suite (150,317 requests)
|
| 42 |
+
type: decision-index
|
| 43 |
+
revision: 87d4650b42b377c0291a89c1f1a879f9b31082bf
|
| 44 |
+
metrics:
|
| 45 |
+
- type: balanced_skill_index
|
| 46 |
+
name: Balanced-skill index, effort high (author-run, not leaderboard-verified)
|
| 47 |
+
value: 54.59
|
| 48 |
+
verified: false
|
| 49 |
+
source:
|
| 50 |
+
name: 'Author-run with the official kit; submission PR #30 awaits maintainer validation'
|
| 51 |
+
url: https://github.com/apolinario/decision-index/pull/30
|
| 52 |
+
- task:
|
| 53 |
+
type: text-classification
|
| 54 |
+
name: Structured decisions (option probabilities)
|
| 55 |
+
dataset:
|
| 56 |
+
name: JevBench public set (231 items)
|
| 57 |
+
type: jevbench
|
| 58 |
+
split: public
|
| 59 |
+
revision: 9ec6f15a
|
| 60 |
+
metrics:
|
| 61 |
+
- type: accuracy
|
| 62 |
+
name: Accuracy, effort none (self-scored, 203/231)
|
| 63 |
+
value: 87.88
|
| 64 |
+
verified: false
|
| 65 |
+
- type: ece
|
| 66 |
+
name: Expected calibration error, 10 bins (self-scored)
|
| 67 |
+
value: 0.041
|
| 68 |
+
verified: false
|
| 69 |
+
- type: brier
|
| 70 |
+
name: Brier score (self-scored)
|
| 71 |
+
value: 0.188
|
| 72 |
+
verified: false
|
| 73 |
+
source:
|
| 74 |
+
name: 'Self-scored with the JevBench harness; row requested in jevbench issue #146'
|
| 75 |
+
url: https://github.com/fstandhartinger/jevbench/issues/146
|
| 76 |
---
|
| 77 |
|
| 78 |
<div align="center">
|
| 79 |
<h1>Wald-Q4B v1.1</h1>
|
| 80 |
<p><strong>Decide directly. Think when needed. Return probabilities.</strong></p>
|
| 81 |
+
<p><a href="https://huggingface.co/Harry19081/Wald-4B">English</a> · <a href="https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/readmes/README.zh.md">简体中文</a> · <a href="https://github.com/Harry19081/wald-4b">GitHub</a> · <a href="https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md">API</a></p>
|
| 82 |
</div>
|
| 83 |
|
| 84 |
+
**Wald-Q4B v1.1 is an open-weight 4B decision model: give it a state and a set of options, and it returns a calibrated probability for every option.** It is for developers who build agents and pipelines and need a fast, self-hosted component to pick a tool, route a request, classify an input or decide whether to ask the user. Unlike a chat model, it does not write an answer you have to parse. It reads the options in one pass (33 ms median on one RTX PRO 6000 with effort `none`) and can optionally think first. It serves a Jev-compatible `POST /v1/systemone` API, is built on Qwen3.5-4B-Base and is released under Apache-2.0.
|
| 85 |
|
| 86 |
+
Wald-Q4B is an independent, self-hosted alternative to TypeSafe's hosted Jev API. It is not Jev, contains no Jev weights, and is not affiliated with or endorsed by TypeSafe AI. The Hugging Face repository is `Harry19081/Wald-4B` (earlier name: Wald-4B).
|
| 87 |
|
| 88 |
+
**W**ait **A** bit, **L**ook, then **D**ecide. Also named after Abraham Wald, the pioneer of sequential analysis: stop when the evidence is enough.
|
| 89 |
|
| 90 |
## At a glance
|
| 91 |
|
| 92 |
+
- **4B parameters**, built on Qwen3.5-4B-Base; BF16 weights (8.4 GB).
|
| 93 |
+
- **Every option gets a probability.** Question types: `choice` (1–255 named options), `noul` (yes/no) and `score` (ordered levels).
|
| 94 |
+
- **Adjustable thinking:** `none`, `low`, `medium`, `high`, or several thoughts with `high-k`.
|
| 95 |
+
- **Decision Index 0.2.1: 54.59** on the complete suite with `high`; 150,317 requests, all successful. Author-run, pending maintainer validation.
|
| 96 |
+
- **JevBench public set: 203/231** with `none`, ECE 0.041, p50 33 ms / p95 168 ms. Self-scored with JevBench's own harness.
|
| 97 |
+
- **Self-hosted API:** `POST /v1/systemone`, up to 131,072 prompt tokens.
|
| 98 |
+
|
| 99 |
+
## Quick start
|
| 100 |
+
|
| 101 |
+
On a Linux machine with an NVIDIA GPU and [`uv`](https://docs.astral.sh/uv/):
|
| 102 |
+
|
| 103 |
+
```sh
|
| 104 |
+
hf download Harry19081/Wald-4B --revision v1.1 --local-dir ./Wald-Q4B
|
| 105 |
+
cd Wald-Q4B
|
| 106 |
+
EFFORT=none ./run.sh "$PWD" # one pass, lowest latency
|
| 107 |
+
# ./run.sh "$PWD" # default: high, the evaluated Decision Index configuration
|
| 108 |
+
```
|
| 109 |
+
|
| 110 |
+
```sh
|
| 111 |
+
curl http://localhost:8000/v1/systemone \
|
| 112 |
+
-H 'Content-Type: application/json' \
|
| 113 |
+
-d '{
|
| 114 |
+
"state": "The customer wants to return a damaged kettle.",
|
| 115 |
+
"effort": "medium",
|
| 116 |
+
"questions": {
|
| 117 |
+
"route": {
|
| 118 |
+
"type": "choice",
|
| 119 |
+
"instructions": "Choose the support queue.",
|
| 120 |
+
"criteria": {
|
| 121 |
+
"returns": "Returns and refunds",
|
| 122 |
+
"delivery": "Delivery tracking",
|
| 123 |
+
"other": "Other enquiries"
|
| 124 |
+
}
|
| 125 |
+
}
|
| 126 |
+
}
|
| 127 |
+
}'
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
The answer for `route` contains the chosen key and a probability for each of `returns`, `delivery` and `other`. Request and response fields, yes/no and score questions, and a clarification example: [API reference](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md). `GET /health` reports the effective policy. The server uses vLLM 0.30.0 and the included `wald-serve` package; Docker and exact evaluation settings are in [RUNBOOK.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md). Generic text-generation calls do not reproduce the decision API's readout.
|
| 131 |
|
| 132 |
## Thinking effort
|
| 133 |
|
|
|
|
| 141 |
| **`high` (default)** | Every eligible question | Up to 512 tokens |
|
| 142 |
| `high-k2` … `high-k8` | Multiple thoughts; average their answer distributions | Up to 512 tokens per thought |
|
| 143 |
|
| 144 |
+
Thinking applies to questions with 2–26 options when context space permits. Larger option sets use grouped readout and a final winner comparison; if a thought cannot fit, the initial answer is kept. Increasing effort spends more computation; it does not guarantee a better answer. **54.59 applies to `high` only; 203/231 applies to `none` only.**
|
| 145 |
|
| 146 |
Set the server default with `EFFORT=medium ./run.sh "$PWD"`, or override it per request with `"effort": "none"`.
|
| 147 |
|
|
|
|
| 149 |
|
| 150 |
Wald first reads option-letter logits from a plain prompt and turns them into a probability distribution. If the effort policy asks for thinking, it generates a short thought and reads the options again. Bucketed temperature scaling calibrates the returned probabilities.
|
| 151 |
|
| 152 |
+
v1.1 combines full-parameter decision training, LoRA refinement, short-thought distillation and RLCD. Training uses **WaldGen**, our generated decision corpus, together with public training datasets. [Data sources](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md) · [Evaluation notes](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md)
|
| 153 |
|
| 154 |
## Benchmarks
|
| 155 |
|
| 156 |
+
| Benchmark | Configuration | Result | Status |
|
| 157 |
+
|---|---|---:|---|
|
| 158 |
+
| **Decision Index 0.2.1 — complete suite** | v1.1 · `high` | **54.59** | Author-run; [PR #30](https://github.com/apolinario/decision-index/pull/30) awaits maintainer validation |
|
| 159 |
+
| **JevBench public set (231 items)** | v1.1 · `none` | **203/231** (87.9%) · ECE 0.041 · Brier 0.188 | Self-scored; [issue #146](https://github.com/fstandhartinger/jevbench/issues/146) asks the maintainers to measure it |
|
| 160 |
|
| 161 |
+
**Decision Index:** 150,317/150,317 requests succeeded, including HLE. Measured on one RTX PRO 6000 96 GB with the pinned reproduction kit. [Full results](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results/tree/805716601b2466be324ed6716407b4c3d9267faa/runs/wald-q4b-22d0-f7-full021) · [Per-benchmark scores](https://huggingface.co/Harry19081/Wald-4B/blob/main/evaluation/benchmark-summary.json) · [Reproduction guide](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md)
|
| 162 |
|
| 163 |
+
**JevBench:** JevBench's own CLI (`fstandhartinger/jevbench` at `9ec6f15a`, `typesafe` adapter) against the packaged server on loopback, one request at a time, on one RTX PRO 6000. Easy 48/48, original 72/72, hard 83/111; no tokens generated. With `medium`: 205/231, p95 1.80 s. The public items were used as a development scoreboard (never as training data), so this is not a held-out result. The JevBench leaderboard publishes a score only after its maintainers run the model themselves.
|
| 164 |
|
| 165 |
### Latency
|
| 166 |
|
| 167 |
+
With `none`, the JevBench run above measured **33 ms median and 168 ms p95** per decision. A 32-request serial preflight of `high` measured **821 ms median** on the same GPU; this small preflight is not a full-suite latency result or the Decision Index maintainers' admission test. Effort, context length, option count and concurrency all affect speed.
|
| 168 |
|
| 169 |
+
## Related projects and how Wald compares
|
| 170 |
|
| 171 |
+
Several projects implement or approximate structured decisions with calibrated option probabilities. The names below belong to their owners; Wald is not affiliated with any of them.
|
| 172 |
|
| 173 |
+
- **[Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev)** is TypeSafe AI's hosted decision model behind the `/v1/systemone` API; its weights are closed. Wald accepts the same request shape and runs on your own GPU.
|
| 174 |
+
- **[Kev](https://github.com/jaredpalmer/kev)** by Jared Palmer is an open-weight project that adds a LoRA and a pointer head to Qwen3.5 base models (0.8B, 4B and 9B). Wald's request parsing adapts Kev's Apache-2.0 code ([NOTICE](https://huggingface.co/Harry19081/Wald-4B/blob/main/NOTICE)); Wald reads option letters from the language-model head instead of a separate head.
|
| 175 |
+
- **[Laya](https://huggingface.co/convaiinnovations/laya)** ([code](https://github.com/NandhaKishorM/laya)) is an open-weight 421M ModernBERT-large encoder with a decision head. It is much smaller than Wald and reads up to 512 tokens.
|
|
|
|
|
|
|
|
|
|
| 176 |
|
| 177 |
+
**JevBench public set, same 231 items (identical dataset hash), JevBench CLI, run by us:**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 178 |
|
| 179 |
+
| System | How it was run | Correct |
|
| 180 |
+
|---|---|---:|
|
| 181 |
+
| Wald-Q4B v1.1 · `none` | Self-hosted, RTX PRO 6000, 2026-09-29 | 203/231 |
|
| 182 |
+
| Jev (`jev-1.13.0`) | TypeSafe's hosted API, 2026-09-25 | 200/231 |
|
| 183 |
+
| Laya (English checkpoint `55cf4c4e`) | Self-hosted, NVIDIA L4, 2026-09-26 | 134/231 |
|
| 184 |
|
| 185 |
+
A 3-item difference on 231 items is within run-to-run and sampling noise. The public items informed Wald's development, and 52 of the 231 states are longer than Laya's 512-token window.
|
| 186 |
+
|
| 187 |
+
**Decision Index 0.2.1:**
|
| 188 |
+
|
| 189 |
+
| System | Index | Source |
|
| 190 |
+
|---|---:|---|
|
| 191 |
+
| Jev (`jev-1.13.0`) | 57.91 | [Leaderboard](https://huggingface.co/spaces/multimodalart/jev-decision-index), maintainer-run (data of 2026-09-28) |
|
| 192 |
+
| Wald-Q4B v1.1 · `high` | 54.59 | Author-run complete suite; not on the leaderboard yet ([PR #30](https://github.com/apolinario/decision-index/pull/30)) |
|
| 193 |
+
| Kev 9B | 38.48 | Leaderboard, maintainer-run (data of 2026-09-28) |
|
| 194 |
+
| Kev 4B | 34.64 | Leaderboard, maintainer-run (data of 2026-09-28) |
|
| 195 |
+
|
| 196 |
+
Leaderboard rows are scored by the maintainers; Wald's number is self-run with the official kit and may change after validation.
|
| 197 |
+
|
| 198 |
+
## FAQ
|
| 199 |
+
|
| 200 |
+
**Is there an open-source alternative to Jev?** Wald-Q4B is one open-weight option: Apache-2.0 weights and serving code that you run yourself, with a Jev-compatible `/v1/systemone` API. Kev and Laya (above) are other open projects. Wald is independent and is not a TypeSafe release.
|
| 201 |
+
|
| 202 |
+
**Can I use a Jev client with a self-hosted model?** Point the client at your own endpoint. The included server accepts `state` plus typed `questions` (`choice`, `noul`, `score`) at `POST /v1/systemone` and answers with TypeSafe's answer keys. It does not check API keys. See the [API reference](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md).
|
| 203 |
|
| 204 |
+
**How do I route tools or decide whether to ask the user?** Send the conversation or task as `state`. For tool routing, ask a `choice` question whose options are your tools. To decide whether to ask a clarifying question, ask a `noul` question such as "Is the request specific enough to act on without asking?" Act when the probability is high, ask when it is low, and set both thresholds on your own validation data. The model picks the tool; it does not write the tool's arguments.
|
| 205 |
|
| 206 |
+
**How calibrated are the probabilities?** On the JevBench public set with `none`, expected calibration error is 0.041 (10 bins) and the Brier score is 0.188. Temperatures were fitted on held-out rows of our own development data, with no JevBench items and with known Decision Index matches excluded. A confidence is not a guarantee; check calibration on your task.
|
| 207 |
|
| 208 |
+
**Does it run on a single GPU or a laptop?** The shipped server needs one NVIDIA GPU on Linux (vLLM 0.30.0); the BF16 weights are 8.4 GB. The v1.1 measurements come from an RTX PRO 6000 96 GB; earlier builds of the same 4B architecture have also been served with vLLM on a 24 GB NVIDIA L4 at a 16K context limit. CPU, Apple Silicon and laptop setups are not supported by the shipped server and have not been tested.
|
| 209 |
|
| 210 |
+
**Kev vs Wald, or Laya vs Wald?** All three are open-weight. Kev adds a pointer head and LoRA to Qwen3.5 base models; Laya is a small encoder with a decision head; Wald is a fully trained 4B decoder with optional thinking. Our same-protocol measurements are in the tables above. Choose on your own task, latency budget and hardware.
|
| 211 |
+
|
| 212 |
+
**Can I fine-tune it for my task?** It is a standard Transformers checkpoint, so common LoRA tooling applies. For v1.0 we trained per-task LoRAs for $0.12–$1.81 of GPU time each; that tooling is not public yet, and those adapters are not validated on v1.1 ([v1.0 notes](https://huggingface.co/Harry19081/Wald-4B/blob/main/history/v1.0/README.md)).
|
| 213 |
+
|
| 214 |
+
**What is the license?** Apache-2.0 for the weights and code. The base model, Qwen3.5-4B-Base, is also Apache-2.0. Some public training sources have their own terms or no stated licence; they are listed in [PROVENANCE.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md). The model and code licence does not grant rights in those texts.
|
| 215 |
+
|
| 216 |
+
## Limits
|
| 217 |
+
|
| 218 |
+
Confidence is not a guarantee of correctness; validate thresholds on your own task. Oversized prompts are rejected rather than truncated. Known exact training overlaps were filtered, but semantic overlap and pretraining contamination are not ruled out; development used visible benchmark samples. Source-text rights vary. See [evaluation notes](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md) and [source attribution](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md).
|
| 219 |
|
| 220 |
## Versioning
|
| 221 |
|
|
|
|
| 224 |
| **v1.1 — current** | `022D0-f7` | `high` |
|
| 225 |
| [v1.0 — archive](https://huggingface.co/Harry19081/Wald-4B/tree/v1.0) | `021A0-f10` | `medium` |
|
| 226 |
|
| 227 |
+
The v1.0 model card retains its XL, task-LoRA and latency reports, and the [v1.0 walkthrough slides](https://claude.ai/artifact/XfCHVaCuj9A5ectrpaWzV5) describe v1.0 only. Those measurements belong to their documented builds.
|
| 228 |
+
|
| 229 |
+
## Citation
|
| 230 |
+
|
| 231 |
+
Wald-Q4B v1.1 (2026), an open-weight 4B decision model with calibrated option probabilities. https://huggingface.co/Harry19081/Wald-4B
|
| 232 |
+
|
| 233 |
+
```bibtex
|
| 234 |
+
@misc{wald_q4b_2026,
|
| 235 |
+
title = {Wald-Q4B v1.1: an open-weight 4B decision model with calibrated option probabilities},
|
| 236 |
+
author = {{Wald-4B authors}},
|
| 237 |
+
year = {2026},
|
| 238 |
+
howpublished = {\url{https://huggingface.co/Harry19081/Wald-4B}},
|
| 239 |
+
note = {Revision v1.1}
|
| 240 |
+
}
|
| 241 |
+
```
|
| 242 |
+
|
| 243 |
+
Machine-readable: [CITATION.cff](https://huggingface.co/Harry19081/Wald-4B/blob/main/CITATION.cff) · [llms.txt](https://huggingface.co/Harry19081/Wald-4B/blob/main/llms.txt) · [model-info.json](https://huggingface.co/Harry19081/Wald-4B/blob/main/model-info.json)
|
| 244 |
|
| 245 |
---
|
| 246 |
|
| 247 |
+
[Model](https://huggingface.co/Harry19081/Wald-4B) · [GitHub](https://github.com/Harry19081/wald-4b) · [Decision Index results](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results) · Apache-2.0 for weights and code. [Third-party notices](https://huggingface.co/Harry19081/Wald-4B/blob/main/NOTICE) · [Training-data usage notes](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md)
|
docs/api.md
ADDED
|
@@ -0,0 +1,148 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Wald-Q4B decision API
|
| 2 |
+
|
| 3 |
+
Wald-Q4B's server, `wald-serve`, exposes one decision endpoint, `POST /v1/systemone`. Its request and answer shapes follow TypeSafe's `/v1/systemone` format, so clients written for Jev can point at a self-hosted Wald server. Wald is independent and is not affiliated with TypeSafe AI.
|
| 4 |
+
|
| 5 |
+
Start the server with `./run.sh "$PWD"` in the downloaded model directory ([model card](https://huggingface.co/Harry19081/Wald-4B), [RUNBOOK.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md)). It listens on port 8000 and does not check API keys.
|
| 6 |
+
|
| 7 |
+
## Endpoints
|
| 8 |
+
|
| 9 |
+
| Method and path | Purpose |
|
| 10 |
+
|---|---|
|
| 11 |
+
| `POST /v1/systemone` (also `POST /`) | Answer one or more typed questions about a state |
|
| 12 |
+
| `GET /health` | Effective policy: effort, gate, thought budget, prompt format, context limit |
|
| 13 |
+
| `GET /v1/models` | The served model name |
|
| 14 |
+
|
| 15 |
+
## Request
|
| 16 |
+
|
| 17 |
+
| Field | Type | Meaning |
|
| 18 |
+
|---|---|---|
|
| 19 |
+
| `state` | string, object, array or null | What the decision is about: a message, a conversation, a document, an agent trace. Objects and arrays are flattened to text with their field names kept. |
|
| 20 |
+
| `questions` | object, at least one entry | Question id → question. Every question is answered about the same `state`. |
|
| 21 |
+
| `effort` | string, optional | `none`, `low`, `medium`, `high` or `high-k2` … `high-k8`. Overrides the server default for this request. |
|
| 22 |
+
| `model` | string, optional | Accepted for client compatibility; the server answers with the model it serves. |
|
| 23 |
+
|
| 24 |
+
Each question has a `type`, optional `instructions` (any JSON, usually a sentence) and `criteria`:
|
| 25 |
+
|
| 26 |
+
| `type` | `criteria` | Answer |
|
| 27 |
+
|---|---|---|
|
| 28 |
+
| `choice` | Object of 1–255 options: key → description (description may be null) | `choice` (the most probable key), `probabilities` (key → probability), `confidence` |
|
| 29 |
+
| `noul` | Optional object with `true` and/or `false` descriptions | `noul`: the probability of yes |
|
| 30 |
+
| `score` | Array of 1–255 ordered levels | `score` (expected level index), `probabilities` (level index → probability), `confidence` |
|
| 31 |
+
|
| 32 |
+
Every answer also carries `mode`: `A` for a one-pass read, `B` for a read after a thought, `K` for grouped (knockout) reading of wide option sets.
|
| 33 |
+
|
| 34 |
+
Prompts longer than the context limit (131,072 tokens by default) are rejected with HTTP 422, never truncated. A malformed request or unknown effort returns HTTP 400.
|
| 35 |
+
|
| 36 |
+
## Example: route a tool call
|
| 37 |
+
|
| 38 |
+
```sh
|
| 39 |
+
curl http://localhost:8000/v1/systemone \
|
| 40 |
+
-H 'Content-Type: application/json' \
|
| 41 |
+
-d '{
|
| 42 |
+
"state": {
|
| 43 |
+
"user": "What will the weather be in Lisbon tomorrow afternoon?",
|
| 44 |
+
"tools_available": ["web_search", "weather_api", "calendar"]
|
| 45 |
+
},
|
| 46 |
+
"effort": "none",
|
| 47 |
+
"questions": {
|
| 48 |
+
"tool": {
|
| 49 |
+
"type": "choice",
|
| 50 |
+
"instructions": "Which tool should the agent call next?",
|
| 51 |
+
"criteria": {
|
| 52 |
+
"web_search": "General web search",
|
| 53 |
+
"weather_api": "Forecast for a city and time",
|
| 54 |
+
"calendar": "Read or create calendar events",
|
| 55 |
+
"none": "Answer directly without a tool"
|
| 56 |
+
}
|
| 57 |
+
}
|
| 58 |
+
}
|
| 59 |
+
}'
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
Response shape (the numbers here are illustrative, not a measured output):
|
| 63 |
+
|
| 64 |
+
```json
|
| 65 |
+
{
|
| 66 |
+
"model": "wald-4b",
|
| 67 |
+
"answers": {
|
| 68 |
+
"tool": {
|
| 69 |
+
"type": "choice",
|
| 70 |
+
"choice": "weather_api",
|
| 71 |
+
"probabilities": {"web_search": 0.04, "weather_api": 0.94, "calendar": 0.0, "none": 0.02},
|
| 72 |
+
"confidence": 0.92,
|
| 73 |
+
"mode": "A"
|
| 74 |
+
}
|
| 75 |
+
},
|
| 76 |
+
"usage": {"input_tokens": 142, "output_tokens": 0},
|
| 77 |
+
"latency_s": 0.03
|
| 78 |
+
}
|
| 79 |
+
```
|
| 80 |
+
|
| 81 |
+
For `choice`, `confidence` rescales the top probability so that 0 means uniform and 1 means certain: `(p_top − 1/K) / (1 − 1/K)` for K options.
|
| 82 |
+
|
| 83 |
+
## Example: decide whether to ask the user
|
| 84 |
+
|
| 85 |
+
Ask several questions about one state in a single request. Each question is read separately.
|
| 86 |
+
|
| 87 |
+
```json
|
| 88 |
+
{
|
| 89 |
+
"state": "User: book me a table for Friday",
|
| 90 |
+
"effort": "medium",
|
| 91 |
+
"questions": {
|
| 92 |
+
"specific_enough": {
|
| 93 |
+
"type": "noul",
|
| 94 |
+
"instructions": "Is the request specific enough to act on without asking a follow-up question?",
|
| 95 |
+
"criteria": {"true": "Enough detail to act", "false": "Needs clarification first"}
|
| 96 |
+
},
|
| 97 |
+
"urgency": {
|
| 98 |
+
"type": "score",
|
| 99 |
+
"instructions": "How urgent is this request?",
|
| 100 |
+
"criteria": ["low", "medium", "high"]
|
| 101 |
+
}
|
| 102 |
+
}
|
| 103 |
+
}
|
| 104 |
+
```
|
| 105 |
+
|
| 106 |
+
Response shape (illustrative numbers):
|
| 107 |
+
|
| 108 |
+
```json
|
| 109 |
+
{
|
| 110 |
+
"model": "wald-4b",
|
| 111 |
+
"answers": {
|
| 112 |
+
"specific_enough": {"type": "noul", "noul": 0.08, "mode": "B"},
|
| 113 |
+
"urgency": {"type": "score", "score": 1.1, "probabilities": {"0": 0.15, "1": 0.6, "2": 0.25}, "confidence": 0.6, "mode": "A"}
|
| 114 |
+
},
|
| 115 |
+
"usage": {"input_tokens": 310, "output_tokens": 96},
|
| 116 |
+
"latency_s": 0.6
|
| 117 |
+
}
|
| 118 |
+
```
|
| 119 |
+
|
| 120 |
+
A typical policy: act when `noul` is above a high threshold, ask the user when it is below a low one, and escalate the cases in between. Set both thresholds on validation data from your own task.
|
| 121 |
+
|
| 122 |
+
## Choosing an effort
|
| 123 |
+
|
| 124 |
+
| Effort | Use it for |
|
| 125 |
+
|---|---|
|
| 126 |
+
| `none` | Lowest latency. One pass, no generated tokens. |
|
| 127 |
+
| `low` / `medium` | Think only when the top initial probability is below 0.5 / 0.7. |
|
| 128 |
+
| `high` | Think on every eligible question. The published Decision Index score uses this setting. |
|
| 129 |
+
| `high-k2` … `high-k8` | Several thoughts, averaged. Slowest. |
|
| 130 |
+
|
| 131 |
+
Thinking applies to questions with 2–26 options when the context has room; otherwise the one-pass answer is returned.
|
| 132 |
+
|
| 133 |
+
## Python client
|
| 134 |
+
|
| 135 |
+
```python
|
| 136 |
+
import requests
|
| 137 |
+
|
| 138 |
+
r = requests.post("http://localhost:8000/v1/systemone", json={
|
| 139 |
+
"state": "Refund request for order 1182; the parcel arrived damaged.",
|
| 140 |
+
"effort": "none",
|
| 141 |
+
"questions": {"route": {"type": "choice", "criteria": {
|
| 142 |
+
"refunds": "Refunds and returns", "shipping": "Delivery problems", "other": "Anything else"}}},
|
| 143 |
+
}, timeout=30)
|
| 144 |
+
answer = r.json()["answers"]["route"]
|
| 145 |
+
print(answer["choice"], answer["probabilities"])
|
| 146 |
+
```
|
| 147 |
+
|
| 148 |
+
Source: [`server/src/wald_serve/wire.py`](https://huggingface.co/Harry19081/Wald-4B/blob/main/server/src/wald_serve/wire.py) (request shapes) and [`server.py`](https://huggingface.co/Harry19081/Wald-4B/blob/main/server/src/wald_serve/server.py) (endpoints).
|
docs/readmes/README.zh.md
CHANGED
|
@@ -1,21 +1,56 @@
|
|
| 1 |
<div align="center">
|
| 2 |
<h1>Wald-Q4B v1.1</h1>
|
| 3 |
<p><strong>直接决策,按需思考,返回概率。</strong></p>
|
| 4 |
-
<p><a href="
|
| 5 |
</div>
|
| 6 |
|
| 7 |
-
**Wald-Q4B v1.1 是面向
|
| 8 |
|
| 9 |
-
|
| 10 |
|
| 11 |
-
|
| 12 |
|
| 13 |
## 一览
|
| 14 |
|
| 15 |
-
- **4B 参数**,基于 Qwen3.5-4B-Base,BF16 权重。
|
|
|
|
| 16 |
- **可调思考程度**:`none`、`low`、`medium`、`high`,以及多次思考的 `high-k`。
|
| 17 |
-
- **完整 Decision Index 0.2.1:54.59**,使用 `high`,150,317 个请求全部成功。
|
| 18 |
-
- **
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
## 思考程度
|
| 21 |
|
|
@@ -29,7 +64,7 @@
|
|
| 29 |
| **`high`(默认)** | 每个符合条件的问题 | 最多 512 token |
|
| 30 |
| `high-k2` … `high-k8` | 多次思考,平均答案分布 | 每次最多 512 token |
|
| 31 |
|
| 32 |
-
思考适
|
| 33 |
|
| 34 |
通过 `EFFORT=medium ./run.sh "$PWD"` 设置服务默认值,也可在单次请求中传入 `"effort": "none"` 覆盖。
|
| 35 |
|
|
@@ -37,64 +72,73 @@
|
|
| 37 |
|
| 38 |
Wald 先从普通文本提示末尾读取选项字母的 logits,得到初始概率。如果 effort 策略触发思考,就生成一段短思考,再读取选项。返回的概率经过分桶温度校准。
|
| 39 |
|
| 40 |
-
v1.1 结合全参数决策训练、LoRA 精修、短思考蒸馏和 RLCD。训练使用我们的自生成决策语料 **WaldGen**,并混合公开训练数据。[数据来源](
|
| 41 |
|
| 42 |
## 评测
|
| 43 |
|
| 44 |
-
| 基准 | 配置 |
|
| 45 |
-
|---|---|---:|
|
| 46 |
-
| **Decision Index 0.2.1 完整套件** | v1.1 · `high` | **54.59** |
|
|
|
|
| 47 |
|
| 48 |
-
150,317/150,317 个请求全部成功,包含 HLE。在单张 RTX PRO 6000 96 GB 上使用固定版本的复现工具运行。
|
| 49 |
|
| 50 |
-
|
| 51 |
|
| 52 |
### 速度
|
| 53 |
|
| 54 |
-
|
| 55 |
|
| 56 |
-
##
|
| 57 |
|
| 58 |
-
在
|
| 59 |
|
| 60 |
-
``
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
./run.sh "$PWD" # 默认 high
|
| 64 |
-
# EFFORT=medium ./run.sh "$PWD" # 按置信度触发思考
|
| 65 |
-
```
|
| 66 |
|
| 67 |
-
|
| 68 |
-
curl http://localhost:8000/v1/systemone \
|
| 69 |
-
-H 'Content-Type: application/json' \
|
| 70 |
-
-d '{
|
| 71 |
-
"state": "The customer wants to return a damaged kettle.",
|
| 72 |
-
"effort": "medium",
|
| 73 |
-
"questions": {
|
| 74 |
-
"route": {
|
| 75 |
-
"type": "choice",
|
| 76 |
-
"instructions": "Choose the support queue.",
|
| 77 |
-
"criteria": {
|
| 78 |
-
"returns": "Returns and refunds",
|
| 79 |
-
"delivery": "Delivery tracking",
|
| 80 |
-
"other": "Other enquiries"
|
| 81 |
-
}
|
| 82 |
-
}
|
| 83 |
-
}
|
| 84 |
-
}'
|
| 85 |
-
```
|
| 86 |
|
| 87 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
|
| 89 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
|
| 91 |
-
|
| 92 |
|
| 93 |
-
|
| 94 |
|
| 95 |
-
**
|
| 96 |
|
| 97 |
-
**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
|
| 99 |
## 版本
|
| 100 |
|
|
@@ -103,8 +147,24 @@ curl http://localhost:8000/v1/systemone \
|
|
| 103 |
| **v1.1 — 当前版本** | `022D0-f7` | `high` |
|
| 104 |
| [v1.0 — 历史版本](https://huggingface.co/Harry19081/Wald-4B/tree/v1.0) | `021A0-f10` | `medium` |
|
| 105 |
|
| 106 |
-
v1.0 模型卡保留其 XL、任务 LoRA 和延迟报告
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
|
| 108 |
---
|
| 109 |
|
| 110 |
-
[
|
|
|
|
| 1 |
<div align="center">
|
| 2 |
<h1>Wald-Q4B v1.1</h1>
|
| 3 |
<p><strong>直接决策,按需思考,返回概率。</strong></p>
|
| 4 |
+
<p><a href="https://huggingface.co/Harry19081/Wald-4B">English</a> · <a href="https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/readmes/README.zh.md">简体中文</a> · <a href="https://github.com/Harry19081/wald-4b">GitHub</a> · <a href="https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md">API</a></p>
|
| 5 |
</div>
|
| 6 |
|
| 7 |
+
**Wald-Q4B v1.1 是一个开放权重的 4B 决策模型:给它一段状态和一组选项,它为每个选项返回校准过的概率。** 它面向构建 agent 和数据流水线的开发者:需要一个快速、可自行部署的组件来选择工具、路由请求、分类输入,或判断是否需要向用户澄清。和聊天模型不同,它不生成需要再解析的答案,而是一遍读出所有选项的概率(effort `none` 时单张 RTX PRO 6000 上中位延迟 33 ms),也可以先思考再回答。它提供与 Jev 兼容的 `POST /v1/systemone` API,基于 Qwen3.5-4B-Base,以 Apache-2.0 发布。
|
| 8 |
|
| 9 |
+
Wald-Q4B 是 TypeSafe 托管 Jev API 之外、可自行部署的独立替代方案。它不是 Jev,不含 Jev 权重,与 TypeSafe AI 没有隶属或背书关系。Hugging Face 仓库为 `Harry19081/Wald-4B`(旧名 Wald-4B)。
|
| 10 |
|
| 11 |
+
**W**ait **A** bit, **L**ook, then **D**ecide:稍等一下,看清楚,再决定。名字也致敬序贯分析先驱 Abraham Wald:证据足够时就停止。
|
| 12 |
|
| 13 |
## 一览
|
| 14 |
|
| 15 |
+
- **4B 参数**,基于 Qwen3.5-4B-Base,BF16 权重(8.4 GB)。
|
| 16 |
+
- **每个选项都有概率。** 题型:`choice`(1–255 个命名选项)、`noul`(是/否)、`score`(有序等级)。
|
| 17 |
- **可调思考程度**:`none`、`low`、`medium`、`high`,以及多次思考的 `high-k`。
|
| 18 |
+
- **完整 Decision Index 0.2.1:54.59**,使用 `high`,150,317 个请求全部成功。作者自测,等待维护者验证。
|
| 19 |
+
- **JevBench 公开集:203/231**,使用 `none`,ECE 0.041,p50 33 ms / p95 168 ms。用 JevBench 自己的评测工具自评。
|
| 20 |
+
- **可自行部署的 API**:`POST /v1/systemone`,最多 131,072 个提示 token。
|
| 21 |
+
|
| 22 |
+
## 快速开始
|
| 23 |
+
|
| 24 |
+
在装有 NVIDIA GPU 和 [`uv`](https://docs.astral.sh/uv/) 的 Linux 机器上:
|
| 25 |
+
|
| 26 |
+
```sh
|
| 27 |
+
hf download Harry19081/Wald-4B --revision v1.1 --local-dir ./Wald-Q4B
|
| 28 |
+
cd Wald-Q4B
|
| 29 |
+
EFFORT=none ./run.sh "$PWD" # 一遍读出,延迟最低
|
| 30 |
+
# ./run.sh "$PWD" # 默认 high,即 Decision Index 的评测配置
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
```sh
|
| 34 |
+
curl http://localhost:8000/v1/systemone \
|
| 35 |
+
-H 'Content-Type: application/json' \
|
| 36 |
+
-d '{
|
| 37 |
+
"state": "The customer wants to return a damaged kettle.",
|
| 38 |
+
"effort": "medium",
|
| 39 |
+
"questions": {
|
| 40 |
+
"route": {
|
| 41 |
+
"type": "choice",
|
| 42 |
+
"instructions": "Choose the support queue.",
|
| 43 |
+
"criteria": {
|
| 44 |
+
"returns": "Returns and refunds",
|
| 45 |
+
"delivery": "Delivery tracking",
|
| 46 |
+
"other": "Other enquiries"
|
| 47 |
+
}
|
| 48 |
+
}
|
| 49 |
+
}
|
| 50 |
+
}'
|
| 51 |
+
```
|
| 52 |
+
|
| 53 |
+
`route` 的答案包含选中的键,以及 `returns`、`delivery`、`other` 各自的概率。请求与响应字段、是非题和评分题、澄清判断示例见 [API 说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md)。`GET /health` 返回当前生效策略。服务使用 vLLM 0.30.0 和仓库内的 `wald-serve`;Docker 与精确评测配置见 [RUNBOOK.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md)。普通文本生成接口不会复现决策 API 的读出流程。
|
| 54 |
|
| 55 |
## 思考程度
|
| 56 |
|
|
|
|
| 64 |
| **`high`(默认)** | 每个符合条件的问题 | 最多 512 token |
|
| 65 |
| `high-k2` … `high-k8` | 多次思考,平均答案分布 | 每次最多 512 token |
|
| 66 |
|
| 67 |
+
思考适���于 2–26 个选项且上下文空间足够的问题。更多选项采用分组读取,再比较各组优胜项;若放不下思考文本,则保留初次答案。提高 effort 会增加计算量,但不保证每道题都更准确。**54.59 仅对应 `high`;203/231 仅对应 `none`。**
|
| 68 |
|
| 69 |
通过 `EFFORT=medium ./run.sh "$PWD"` 设置服务默认值,也可在单次请求中传入 `"effort": "none"` 覆盖。
|
| 70 |
|
|
|
|
| 72 |
|
| 73 |
Wald 先从普通文本提示末尾读取选项字母的 logits,得到初始概率。如果 effort 策略触发思考,就生成一段短思考,再读取选项。返回的概率经过分桶温度校准。
|
| 74 |
|
| 75 |
+
v1.1 结合全参数决策训练、LoRA 精修、短思考蒸馏和 RLCD。训练使用我们的自生成决策语料 **WaldGen**,并混合公开训练数据。[数据来源](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md) · [评测说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md)
|
| 76 |
|
| 77 |
## 评测
|
| 78 |
|
| 79 |
+
| 基准 | 配置 | 结果 | 状态 |
|
| 80 |
+
|---|---|---:|---|
|
| 81 |
+
| **Decision Index 0.2.1 完整套件** | v1.1 · `high` | **54.59** | 作者自测;[PR #30](https://github.com/apolinario/decision-index/pull/30) 等待维护者验证 |
|
| 82 |
+
| **JevBench 公开集(231 题)** | v1.1 · `none` | **203/231**(87.9%)· ECE 0.041 · Brier 0.188 | 自评;[issue #146](https://github.com/fstandhartinger/jevbench/issues/146) 请维护者自行测量 |
|
| 83 |
|
| 84 |
+
**Decision Index:** 150,317/150,317 个请求全部成功,包含 HLE。在单张 RTX PRO 6000 96 GB 上使用固定版本的复现工具运行。[完整结果](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results/tree/805716601b2466be324ed6716407b4c3d9267faa/runs/wald-q4b-22d0-f7-full021) · [分项成绩](https://huggingface.co/Harry19081/Wald-4B/blob/main/evaluation/benchmark-summary.json) · [复现指南](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md)
|
| 85 |
|
| 86 |
+
**JevBench:** 使用 JevBench 自己的命令行工具(`fstandhartinger/jevbench` @ `9ec6f15a`,`typesafe` 适配器),在单张 RTX PRO 6000 上通过本机回环逐条请求打包服务。easy 48/48、original 72/72、hard 83/111;不生成任何 token。`medium` 为 205/231,p95 1.80 s。公开题在开发中被用作计分板(从未作为训练数据),因此这不是留出集结果。JevBench 排行榜只在维护者自己跑过模型后才公布分数。
|
| 87 |
|
| 88 |
### 速度
|
| 89 |
|
| 90 |
+
使用 `none` 时,上述 JevBench 运行的单次决策**中位延迟 33 ms、p95 168 ms**。`high` 在同一 GPU 上的 32 请求串行预检中,**中位延迟为 821 ms**;这只是小规模预检,不代表完整套件延迟或 Decision Index 维护者的准入测试。effort、上下文长度、选项数量和并发都会影响速度。
|
| 91 |
|
| 92 |
+
## 相关项目与对比
|
| 93 |
|
| 94 |
+
多个项目在做带校准选项概率的结构化决策。以下名称归各自所有者;Wald 与它们都没有隶属关系。
|
| 95 |
|
| 96 |
+
- **[Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev)** 是 TypeSafe AI 通过 `/v1/systemone` API 提供的托管决策模型,权重不公开。Wald 接受相同的请求格式,运行在你自己的 GPU 上。
|
| 97 |
+
- **[Kev](https://github.com/jaredpalmer/kev)** 是 Jared Palmer 的开放权重项目,在 Qwen3.5 基座(0.8B、4B、9B)上加 LoRA 和指针头。Wald 的请求解析改编自 Kev 的 Apache-2.0 代码(见 [NOTICE](https://huggingface.co/Harry19081/Wald-4B/blob/main/NOTICE));Wald 从语言模型头读取选项字母,不使用单独的头。
|
| 98 |
+
- **[Laya](https://huggingface.co/convaiinnovations/laya)**([代码](https://github.com/NandhaKishorM/laya))是开放权重的 421M ModernBERT-large ���码器加决策头。它比 Wald 小得多,最多读取 512 个 token。
|
|
|
|
|
|
|
|
|
|
| 99 |
|
| 100 |
+
**JevBench 公开集,同一批 231 题(数据集哈希相同),JevBench 命令行工具,由我们运行:**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
|
| 102 |
+
| 系统 | 运行方式 | 答对 |
|
| 103 |
+
|---|---|---:|
|
| 104 |
+
| Wald-Q4B v1.1 · `none` | 自行部署,RTX PRO 6000,2026-09-29 | 203/231 |
|
| 105 |
+
| Jev(`jev-1.13.0`) | TypeSafe 托管 API,2026-09-25 | 200/231 |
|
| 106 |
+
| Laya(英文 checkpoint `55cf4c4e`) | 自行部署,NVIDIA L4,2026-09-26 | 134/231 |
|
| 107 |
|
| 108 |
+
231 题上 3 题的差距在多次运行和抽样的噪声范围内。公开题影响过 Wald 的开发;231 题中有 52 题的状态超过 Laya 的 512 token 窗口。
|
| 109 |
+
|
| 110 |
+
**Decision Index 0.2.1:**
|
| 111 |
+
|
| 112 |
+
| 系统 | 指数 | 来源 |
|
| 113 |
+
|---|---:|---|
|
| 114 |
+
| Jev(`jev-1.13.0`) | 57.91 | [排行榜](https://huggingface.co/spaces/multimodalart/jev-decision-index),维护者运行(2026-09-28 数据) |
|
| 115 |
+
| Wald-Q4B v1.1 · `high` | 54.59 | 作者自测完整套件;尚未上榜([PR #30](https://github.com/apolinario/decision-index/pull/30)) |
|
| 116 |
+
| Kev 9B | 38.48 | 排行榜,维护者运行(2026-09-28 数据) |
|
| 117 |
+
| Kev 4B | 34.64 | 排行榜,维护者运行(2026-09-28 数据) |
|
| 118 |
+
|
| 119 |
+
排行榜各行由维护者评分;Wald 的分数是用官方工具自测的,验证后可能变化。
|
| 120 |
+
|
| 121 |
+
## 常见问题
|
| 122 |
+
|
| 123 |
+
**有开源的 Jev 替代品吗?** Wald-Q4B 是一个开放权重的选择:Apache-2.0 的权重和服务代码,自己部署,提供与 Jev 兼容的 `/v1/systemone` API。Kev 和 Laya(见上)是其他开放项目。Wald 是独立项目,不是 TypeSafe 的发布。
|
| 124 |
+
|
| 125 |
+
**能用 Jev 客户端调用自部署模型吗?** 把客户端指向你自己的端点。内置服务在 `POST /v1/systemone` 接收 `state` 和带类型的 `questions`(`choice`、`noul`、`score`),按 TypeSafe 的答案键返回。服务不校验 API key。见 [API 说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md)。
|
| 126 |
+
|
| 127 |
+
**如何做工具路由,或判断是否需要向用户提问?** 把对话或任务作为 `state` 发送。工具路由:提一个 `choice` 问题,选项就是你的工具。是否澄清:提一个 `noul` 问题,例如“这个请求是否足够具体,可以不问就执行?”概率高就执行,概率低就提问,两个阈值都在你自己的验证数据上确定。模型只选工具,不生成工具参数。
|
| 128 |
|
| 129 |
+
**概率校准得怎么样?** 在 JevBench 公开集上使用 `none`,期望校准误差为 0.041(10 个分桶),Brier 分数为 0.188。温度是在我们自己开发数据的留出行上拟合的,不含 JevBench 题目,并排除了已知的 Decision Index 匹配项。置信度不是保证,请在你的任务上检查校准。
|
| 130 |
|
| 131 |
+
**能在单张 GPU 或笔记本上运行吗?** 内置服务需要 Linux 上的一张 NVIDIA GPU(vLLM 0.30.0);BF16 权重为 8.4 GB。v1.1 的测量来自 RTX PRO 6000 96 GB;同一 4B 架构的早期版本也曾用 vLLM 在 24 GB 的 NVIDIA L4 上以 16K 上下文上限运行。内置服务不支持 CPU、Apple Silicon 和笔记本环境,也没有测试过。
|
| 132 |
|
| 133 |
+
**Kev 和 Wald、Laya 和 Wald 怎么选?** 三者都开放权重。Kev 在 Qwen3.5 基座上加指针头和 LoRA;Laya 是带决策头的小型编码器;Wald 是完整训练的 4B 解码器,可选思考。我们在同一协议下的测量见上表。请根据你自己的任务、延迟预算和硬件选择。
|
| 134 |
|
| 135 |
+
**能针对我的任务微调吗?** 它是标准的 Transformers checkpoint,常见的 LoRA 工具都适用。v1.0 时我们为单个任务训练 LoRA,每个花费 $0.12–$1.81 的 GPU 时间;这套工具尚未公开,这些 adapter 也未在 v1.1 上验证([v1.0 说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/history/v1.0/README.md))。
|
| 136 |
+
|
| 137 |
+
**许可证是什么?** 权重与代码为 Apache-2.0。基座 Qwen3.5-4B-Base 也是 Apache-2.0。部分公开训练数据有各自的条款或没有注明许可证,列在 [PROVENANCE.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md)。模型与代码的许可证不授予这些文本的权利。
|
| 138 |
+
|
| 139 |
+
## 使用限制
|
| 140 |
+
|
| 141 |
+
置信度不保证正确性,应在自己的任务上验证阈值。超过上下文限制的提示会被拒绝,不会截断。已过滤已知的严格训练重叠,但无法排除语义重叠和预训练污染;开发过程中使用了可见的基准样本。各来源文本的使用权不同,详见[评测说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md)与[来源声明](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md)。
|
| 142 |
|
| 143 |
## 版本
|
| 144 |
|
|
|
|
| 147 |
| **v1.1 — 当前版本** | `022D0-f7` | `high` |
|
| 148 |
| [v1.0 — 历史版本](https://huggingface.co/Harry19081/Wald-4B/tree/v1.0) | `021A0-f10` | `medium` |
|
| 149 |
|
| 150 |
+
v1.0 模型卡保留其 XL、任务 LoRA 和延迟报告,[v1.0 讲解幻灯片](https://claude.ai/artifact/XfCHVaCuj9A5ectrpaWzV5)只描述 v1.0。这些测量属于各自注明的模型版本。
|
| 151 |
+
|
| 152 |
+
## 引用
|
| 153 |
+
|
| 154 |
+
Wald-Q4B v1.1 (2026), an open-weight 4B decision model with calibrated option probabilities. https://huggingface.co/Harry19081/Wald-4B
|
| 155 |
+
|
| 156 |
+
```bibtex
|
| 157 |
+
@misc{wald_q4b_2026,
|
| 158 |
+
title = {Wald-Q4B v1.1: an open-weight 4B decision model with calibrated option probabilities},
|
| 159 |
+
author = {{Wald-4B authors}},
|
| 160 |
+
year = {2026},
|
| 161 |
+
howpublished = {\url{https://huggingface.co/Harry19081/Wald-4B}},
|
| 162 |
+
note = {Revision v1.1}
|
| 163 |
+
}
|
| 164 |
+
```
|
| 165 |
+
|
| 166 |
+
机器可读:[CITATION.cff](https://huggingface.co/Harry19081/Wald-4B/blob/main/CITATION.cff) · [llms.txt](https://huggingface.co/Harry19081/Wald-4B/blob/main/llms.txt) · [model-info.json](https://huggingface.co/Harry19081/Wald-4B/blob/main/model-info.json)
|
| 167 |
|
| 168 |
---
|
| 169 |
|
| 170 |
+
[模型](https://huggingface.co/Harry19081/Wald-4B) · [GitHub](https://github.com/Harry19081/wald-4b) · [Decision Index 结果](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results) · 权重与代码:Apache-2.0。[第三方声明](https://huggingface.co/Harry19081/Wald-4B/blob/main/NOTICE) · [训练数据使用说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md)
|
llms.txt
ADDED
|
@@ -0,0 +1,31 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Wald-Q4B v1.1
|
| 2 |
+
|
| 3 |
+
> Open-weight 4B decision model. Given a state and typed questions (choice, yes/no, score), it returns a calibrated probability for every option through a Jev-compatible `POST /v1/systemone` API. Built on Qwen3.5-4B-Base; Apache-2.0 weights and serving code; self-hosted on one NVIDIA GPU with vLLM. Independent project, not affiliated with or endorsed by TypeSafe AI.
|
| 4 |
+
|
| 5 |
+
Key facts:
|
| 6 |
+
- Hugging Face repository: Harry19081/Wald-4B (current release tag `v1.1`; earlier name Wald-4B). GitHub: Harry19081/wald-4b.
|
| 7 |
+
- Uses: tool selection, agent routing, classification, deciding whether to ask the user a clarifying question.
|
| 8 |
+
- Effort levels: `none` (one pass, no generated tokens), `low`, `medium`, `high` (default), `high-k2`…`high-k8`.
|
| 9 |
+
- JevBench public set (231 items), effort `none`: 203/231, ECE 0.041, p50 33 ms / p95 168 ms on one RTX PRO 6000. Self-scored with JevBench's harness; the public items were a development scoreboard, not held out. Leaderboard row requested in fstandhartinger/jevbench issue #146.
|
| 10 |
+
- Decision Index 0.2.1 complete suite, effort `high`: 54.59 balanced-skill index. Author-run; submission apolinario/decision-index PR #30 awaits maintainer validation.
|
| 11 |
+
|
| 12 |
+
## Docs
|
| 13 |
+
|
| 14 |
+
- [Model card](https://huggingface.co/Harry19081/Wald-4B): what it is, benchmarks, comparisons with Jev, Kev and Laya, FAQ, limits
|
| 15 |
+
- [API reference](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md): request and response JSON with examples
|
| 16 |
+
- [RUNBOOK.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md): serving and exact evaluation settings
|
| 17 |
+
- [Chinese model card](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/readmes/README.zh.md)
|
| 18 |
+
|
| 19 |
+
## Evidence
|
| 20 |
+
|
| 21 |
+
- [Decision Index results dataset](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results): untouched responses and scores
|
| 22 |
+
- [Decision Index submission PR #30](https://github.com/apolinario/decision-index/pull/30)
|
| 23 |
+
- [JevBench request issue #146](https://github.com/fstandhartinger/jevbench/issues/146)
|
| 24 |
+
- [model-info.json](https://huggingface.co/Harry19081/Wald-4B/blob/main/model-info.json): machine-readable facts
|
| 25 |
+
|
| 26 |
+
## Optional
|
| 27 |
+
|
| 28 |
+
- [PROVENANCE.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md): training-data sources and their terms
|
| 29 |
+
- [CONTAMINATION.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md): evaluation caveats
|
| 30 |
+
- [CITATION.cff](https://huggingface.co/Harry19081/Wald-4B/blob/main/CITATION.cff)
|
| 31 |
+
- [Server source](https://github.com/Harry19081/wald-4b/tree/main/server)
|
model-info.json
CHANGED
|
@@ -5,7 +5,7 @@
|
|
| 5 |
"Wald-4B"
|
| 6 |
],
|
| 7 |
"checkpoint": "022D0-f7",
|
| 8 |
-
"description": "
|
| 9 |
"base_model": "Qwen/Qwen3.5-4B-Base",
|
| 10 |
"model_url": "https://huggingface.co/Harry19081/Wald-4B",
|
| 11 |
"weights_revision": "50f94ecd5d6e7e8459e9c198ef811bfaba12a3fe",
|
|
@@ -72,7 +72,36 @@
|
|
| 72 |
"zh": "docs/readmes/README.zh.md",
|
| 73 |
"serving": "RUNBOOK.md",
|
| 74 |
"sources": "PROVENANCE.md",
|
| 75 |
-
"evaluation_notes": "CONTAMINATION.md"
|
|
|
|
|
|
|
|
|
|
| 76 |
},
|
| 77 |
-
"affiliation": "Independent; not affiliated with TypeSafe AI"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
}
|
|
|
|
| 5 |
"Wald-4B"
|
| 6 |
],
|
| 7 |
"checkpoint": "022D0-f7",
|
| 8 |
+
"description": "Open-weight 4B decision model that returns a calibrated probability for every option through a Jev-compatible POST /v1/systemone API; one pass or optional thinking; self-hosted. Independent, not affiliated with TypeSafe AI.",
|
| 9 |
"base_model": "Qwen/Qwen3.5-4B-Base",
|
| 10 |
"model_url": "https://huggingface.co/Harry19081/Wald-4B",
|
| 11 |
"weights_revision": "50f94ecd5d6e7e8459e9c198ef811bfaba12a3fe",
|
|
|
|
| 72 |
"zh": "docs/readmes/README.zh.md",
|
| 73 |
"serving": "RUNBOOK.md",
|
| 74 |
"sources": "PROVENANCE.md",
|
| 75 |
+
"evaluation_notes": "CONTAMINATION.md",
|
| 76 |
+
"api": "docs/api.md",
|
| 77 |
+
"citation": "CITATION.cff",
|
| 78 |
+
"llms": "llms.txt"
|
| 79 |
},
|
| 80 |
+
"affiliation": "Independent; not affiliated with TypeSafe AI",
|
| 81 |
+
"evaluations": [
|
| 82 |
+
{
|
| 83 |
+
"benchmark": "JevBench",
|
| 84 |
+
"split": "public",
|
| 85 |
+
"items": 231,
|
| 86 |
+
"effort": "none",
|
| 87 |
+
"correct": 203,
|
| 88 |
+
"accuracy": 0.8788,
|
| 89 |
+
"ece": 0.041,
|
| 90 |
+
"brier": 0.188,
|
| 91 |
+
"latency_p50_ms": 33,
|
| 92 |
+
"latency_p95_ms": 168,
|
| 93 |
+
"hardware": "NVIDIA RTX PRO 6000 96 GB",
|
| 94 |
+
"harness": "fstandhartinger/jevbench@9ec6f15a (jevbench.cli, typesafe adapter, serial, loopback)",
|
| 95 |
+
"status": "self-scored; public items were a development scoreboard, not held out; leaderboard measurement requested",
|
| 96 |
+
"request": "https://github.com/fstandhartinger/jevbench/issues/146"
|
| 97 |
+
}
|
| 98 |
+
],
|
| 99 |
+
"links": {
|
| 100 |
+
"model": "https://huggingface.co/Harry19081/Wald-4B",
|
| 101 |
+
"github": "https://github.com/Harry19081/wald-4b",
|
| 102 |
+
"results_dataset": "https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results",
|
| 103 |
+
"decision_index_submission": "https://github.com/apolinario/decision-index/pull/30",
|
| 104 |
+
"jevbench_request": "https://github.com/fstandhartinger/jevbench/issues/146",
|
| 105 |
+
"api": "https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md"
|
| 106 |
+
}
|
| 107 |
}
|