Harry19081 commited on
Commit
1f96041
·
verified ·
1 Parent(s): b1b4c4c

Model card: clearer summary, FAQ, related projects, JevBench result, model-index; add API reference, CITATION.cff, llms.txt

Browse files
Files changed (7) hide show
  1. CITATION.cff +28 -0
  2. MANIFEST.json +18 -6
  3. README.md +173 -57
  4. docs/api.md +148 -0
  5. docs/readmes/README.zh.md +110 -50
  6. llms.txt +31 -0
  7. model-info.json +32 -3
CITATION.cff ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ cff-version: 1.2.0
2
+ message: "If you use Wald-Q4B, please cite it as below."
3
+ type: software
4
+ title: "Wald-Q4B v1.1: an open-weight 4B decision model with calibrated option probabilities"
5
+ abstract: >-
6
+ Wald-Q4B is an open-weight 4B decision model built on Qwen3.5-4B-Base. Given a
7
+ state and typed questions (choice, yes/no, score), it returns a calibrated
8
+ probability for every option through a Jev-compatible POST /v1/systemone API,
9
+ in one pass or after an optional short thought. Weights and serving code are
10
+ Apache-2.0. Independent project; not affiliated with TypeSafe AI.
11
+ authors:
12
+ - name: "Wald-4B authors"
13
+ alias: Harry19081
14
+ website: "https://huggingface.co/Harry19081"
15
+ version: "v1.1"
16
+ date-released: "2026-09-29"
17
+ license: Apache-2.0
18
+ url: "https://huggingface.co/Harry19081/Wald-4B"
19
+ repository-code: "https://github.com/Harry19081/wald-4b"
20
+ repository-artifact: "https://huggingface.co/Harry19081/Wald-4B"
21
+ keywords:
22
+ - decision model
23
+ - calibrated probabilities
24
+ - tool selection
25
+ - agent routing
26
+ - classification
27
+ - Jev-compatible API
28
+ - Qwen3.5
MANIFEST.json CHANGED
@@ -20,8 +20,8 @@
20
  "sha256": "953551e746a887044cc0ce65dec9b96ea3a763d69fda7d3a296f524bb37bca19"
21
  },
22
  "README.md": {
23
- "bytes": 6659,
24
- "sha256": "5124931bb00d7c1f28ab1f35b440f3837b1efd8e08ac9a6f9ac75f20ae1bdf5f"
25
  },
26
  "RUNBOOK.md": {
27
  "bytes": 2718,
@@ -44,8 +44,8 @@
44
  "sha256": "e385b92e333eaaa9395a09daf26c09420645bee242456552a55500db45919bc7"
45
  },
46
  "docs/readmes/README.zh.md": {
47
- "bytes": 6035,
48
- "sha256": "7e9b1b096443066b801d9e144da275095fa70c109fdd46f48f9b14f862accef0"
49
  },
50
  "evaluation/benchmark-summary.json": {
51
  "bytes": 36014,
@@ -220,7 +220,19 @@
220
  "sha256": "ad20de47de5bbe7f72417f3d87e6f72aae7987fab913eb814b818e4a4bbc6e1e"
221
  },
222
  "model-info.json": {
223
- "bytes": 2165,
224
- "sha256": "f1766a5c20e9dad5da245ad06f68fd6dd39f93d87d44bd6820423e2e59eb6063"
 
 
 
 
 
 
 
 
 
 
 
 
225
  }
226
  }
 
20
  "sha256": "953551e746a887044cc0ce65dec9b96ea3a763d69fda7d3a296f524bb37bca19"
21
  },
22
  "README.md": {
23
+ "bytes": 16370,
24
+ "sha256": "0964da2ec2e05dfa5f46b0008c1fe912f5dddf50a5876cc8970dd3b2baa67cf9"
25
  },
26
  "RUNBOOK.md": {
27
  "bytes": 2718,
 
44
  "sha256": "e385b92e333eaaa9395a09daf26c09420645bee242456552a55500db45919bc7"
45
  },
46
  "docs/readmes/README.zh.md": {
47
+ "bytes": 13881,
48
+ "sha256": "bc2d14f2e64189657b2142fb19e54d94215be4751fcf30113a7cd30c37c40ef8"
49
  },
50
  "evaluation/benchmark-summary.json": {
51
  "bytes": 36014,
 
220
  "sha256": "ad20de47de5bbe7f72417f3d87e6f72aae7987fab913eb814b818e4a4bbc6e1e"
221
  },
222
  "model-info.json": {
223
+ "bytes": 3436,
224
+ "sha256": "4afd88d67874d61ddea72e4ceaf6e0ace4199f2ef5f87cb00fe5df4abc87f00b"
225
+ },
226
+ "llms.txt": {
227
+ "bytes": 2537,
228
+ "sha256": "a814eeca0112086b1740378a16173b27c86c240c5222abaaf7fa33287bdd71f1"
229
+ },
230
+ "CITATION.cff": {
231
+ "bytes": 1075,
232
+ "sha256": "4fab11f4e20a79e69a4ebbf23573abe3a8cd9a374bd5ab11e7e4206a4f070bf1"
233
+ },
234
+ "docs/api.md": {
235
+ "bytes": 6085,
236
+ "sha256": "e2e349bc9d17f01e0d52835f664808f94a6d9c88dd120ba49c9f25af47c235b8"
237
  }
238
  }
README.md CHANGED
@@ -3,40 +3,131 @@ license: apache-2.0
3
  base_model: Qwen/Qwen3.5-4B-Base
4
  base_model_relation: finetune
5
  library_name: transformers
 
6
  language:
7
  - en
8
  tags:
9
  - decision-model
 
10
  - calibration
11
- - typesafe
12
- - decision-index
13
- - wald-q4b
14
- - jev
15
- - decision-making
16
  - tool-selection
 
17
  - agent-routing
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  - reasoning
19
- pipeline_tag: text-generation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  ---
21
 
22
  <div align="center">
23
  <h1>Wald-Q4B v1.1</h1>
24
  <p><strong>Decide directly. Think when needed. Return probabilities.</strong></p>
25
- <p><a href="README.md">English</a> · <a href="docs/readmes/README.zh.md">简体中文</a></p>
26
  </div>
27
 
28
- **Wald-Q4B v1.1 is a self-hosted, open-weight alternative to Jev for structured decisions, with adjustable thinking effort and a TypeSafe-compatible `/v1/systemone` API.** Built on Qwen3.5-4B-Base, it supports tool selection, agent routing and option-probability scoring.
29
 
30
- **W**ait **A** bit, **L**ook, then **D**ecide. Also named after Abraham Wald, the pioneer of sequential analysis: stop when the evidence is enough.
31
 
32
- Give Wald a state, a question and a set of options. It returns a probability for every option—useful for tool selection, routing, classification and deciding whether to ask for clarification. Choose how much it thinks with one `effort` setting. The answer stays a structured distribution at every level.
33
 
34
  ## At a glance
35
 
36
- - **4B parameters**, built on Qwen3.5-4B-Base; BF16 weights.
37
- - **Adjustable thinking:** `none`, `low`, `medium`, `high`, or multiple thoughts with `high-k`.
38
- - **Decision Index 0.2.1: 54.59** on the complete suite with `high`; 150,317 requests, all successful.
39
- - **Self-hosted API:** `POST /v1/systemone`, up to 131,072 prompt tokens and 255 options per question.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
 
41
  ## Thinking effort
42
 
@@ -50,7 +141,7 @@ Start with `none` for direct decisions, `medium` for confidence-gated thinking,
50
  | **`high` (default)** | Every eligible question | Up to 512 tokens |
51
  | `high-k2` … `high-k8` | Multiple thoughts; average their answer distributions | Up to 512 tokens per thought |
52
 
53
- Thinking applies to questions with 2–26 options when context space permits. Larger option sets use grouped readout and a final winner comparison; if a thought cannot fit, the initial answer is kept. Increasing effort spends more computation; it does not guarantee a better answer. **54.59 applies to `high` only.**
54
 
55
  Set the server default with `EFFORT=medium ./run.sh "$PWD"`, or override it per request with `"effort": "none"`.
56
 
@@ -58,64 +149,73 @@ Set the server default with `EFFORT=medium ./run.sh "$PWD"`, or override it per
58
 
59
  Wald first reads option-letter logits from a plain prompt and turns them into a probability distribution. If the effort policy asks for thinking, it generates a short thought and reads the options again. Bucketed temperature scaling calibrates the returned probabilities.
60
 
61
- v1.1 combines full-parameter decision training, LoRA refinement, short-thought distillation and RLCD. Training uses **WaldGen**, our generated decision corpus, together with public training datasets. [Data sources](PROVENANCE.md) · [Evaluation notes](CONTAMINATION.md)
62
 
63
  ## Benchmarks
64
 
65
- | Benchmark | Configuration | Result |
66
- |---|---|---:|
67
- | **Decision Index 0.2.1 — complete suite** | v1.1 · `high` | **54.59** |
 
68
 
69
- 150,317/150,317 requests succeeded, including HLE. Measured on one RTX PRO 6000 96 GB with the pinned reproduction kit. This is an author-run result; [submission #30](https://github.com/apolinario/decision-index/pull/30) awaits maintainer validation.
70
 
71
- [Full results](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results/tree/805716601b2466be324ed6716407b4c3d9267faa/runs/wald-q4b-22d0-f7-full021) · [Per-benchmark scores](evaluation/benchmark-summary.json) · [Reproduction guide](RUNBOOK.md)
72
 
73
  ### Latency
74
 
75
- A 32-request serial preflight of v1.1 `high` measured **821 ms median** on an RTX PRO 6000. This small preflight is not a full-suite latency result or the maintainer's admission test. Effort, context length, option count and concurrency all affect speed.
76
 
77
- ## Serving
78
 
79
- On a Linux CUDA machine with `uv`:
80
 
81
- ```sh
82
- hf download Harry19081/Wald-4B --revision v1.1 --local-dir ./Wald-Q4B
83
- cd Wald-Q4B
84
- ./run.sh "$PWD" # default: high
85
- # EFFORT=medium ./run.sh "$PWD" # confidence-gated thinking
86
- ```
87
 
88
- ```sh
89
- curl http://localhost:8000/v1/systemone \
90
- -H 'Content-Type: application/json' \
91
- -d '{
92
- "state": "The customer wants to return a damaged kettle.",
93
- "effort": "medium",
94
- "questions": {
95
- "route": {
96
- "type": "choice",
97
- "instructions": "Choose the support queue.",
98
- "criteria": {
99
- "returns": "Returns and refunds",
100
- "delivery": "Delivery tracking",
101
- "other": "Other enquiries"
102
- }
103
- }
104
- }
105
- }'
106
- ```
107
 
108
- `GET /health` reports the effective policy. The server uses vLLM 0.30.0 and the included `wald-serve` package. Docker and exact evaluation settings: [RUNBOOK.md](RUNBOOK.md). Generic text-generation calls do not reproduce the decision API's readout.
 
 
 
 
109
 
110
- ## Limits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
111
 
112
- Confidence is not a guarantee of correctness; validate thresholds on your own task. Oversized prompts are rejected rather than truncated. Known exact training overlaps were filtered, but semantic overlap and pretraining contamination are not ruled out; development used visible benchmark samples. Source-text rights vary. See [evaluation notes](CONTAMINATION.md) and [source attribution](PROVENANCE.md).
113
 
114
- ## Jev and agent integration
115
 
116
- **Can I use it with a Jev-style client?** The included server accepts `state` and typed `questions` at `/v1/systemone`, including choice and yes/no decisions. Point your client to your own endpoint and check its supported schema in the [serving guide](RUNBOOK.md). Wald is an independent model, not Jev weights or a TypeSafe AI release.
117
 
118
- **Which effort should my agent use?** Use `none` for direct readout, `medium` for confidence-gated thinking, and `high` to reproduce the published score. All return option probabilities; you do not need to parse a generated answer.
 
 
 
 
 
 
 
 
119
 
120
  ## Versioning
121
 
@@ -124,8 +224,24 @@ Confidence is not a guarantee of correctness; validate thresholds on your own ta
124
  | **v1.1 — current** | `022D0-f7` | `high` |
125
  | [v1.0 — archive](https://huggingface.co/Harry19081/Wald-4B/tree/v1.0) | `021A0-f10` | `medium` |
126
 
127
- The v1.0 model card retains its XL, task-LoRA and latency reports. Those measurements belong to their documented builds.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
128
 
129
  ---
130
 
131
- [Machine-readable model facts](model-info.json) · Apache-2.0 for weights and code. [Third-party notices](NOTICE) · [Training-data usage notes](PROVENANCE.md)
 
3
  base_model: Qwen/Qwen3.5-4B-Base
4
  base_model_relation: finetune
5
  library_name: transformers
6
+ pipeline_tag: text-generation
7
  language:
8
  - en
9
  tags:
10
  - decision-model
11
+ - typed-decisions
12
  - calibration
13
+ - calibrated-probabilities
14
+ - classification
 
 
 
15
  - tool-selection
16
+ - tool-use
17
  - agent-routing
18
+ - clarification
19
+ - decision-index
20
+ - jevbench
21
+ - jev
22
+ - jev-compatible
23
+ - open-jev
24
+ - typesafe-compatible
25
+ - systemone
26
+ - kev
27
+ - laya
28
+ - wald
29
+ - wald-q4b
30
+ - qwen3.5
31
+ - 4b
32
+ - vllm
33
  - reasoning
34
+ model-index:
35
+ - name: Wald-Q4B v1.1
36
+ results:
37
+ - task:
38
+ type: text-classification
39
+ name: Structured decisions (option probabilities)
40
+ dataset:
41
+ name: Decision Index 0.2.1, complete suite (150,317 requests)
42
+ type: decision-index
43
+ revision: 87d4650b42b377c0291a89c1f1a879f9b31082bf
44
+ metrics:
45
+ - type: balanced_skill_index
46
+ name: Balanced-skill index, effort high (author-run, not leaderboard-verified)
47
+ value: 54.59
48
+ verified: false
49
+ source:
50
+ name: 'Author-run with the official kit; submission PR #30 awaits maintainer validation'
51
+ url: https://github.com/apolinario/decision-index/pull/30
52
+ - task:
53
+ type: text-classification
54
+ name: Structured decisions (option probabilities)
55
+ dataset:
56
+ name: JevBench public set (231 items)
57
+ type: jevbench
58
+ split: public
59
+ revision: 9ec6f15a
60
+ metrics:
61
+ - type: accuracy
62
+ name: Accuracy, effort none (self-scored, 203/231)
63
+ value: 87.88
64
+ verified: false
65
+ - type: ece
66
+ name: Expected calibration error, 10 bins (self-scored)
67
+ value: 0.041
68
+ verified: false
69
+ - type: brier
70
+ name: Brier score (self-scored)
71
+ value: 0.188
72
+ verified: false
73
+ source:
74
+ name: 'Self-scored with the JevBench harness; row requested in jevbench issue #146'
75
+ url: https://github.com/fstandhartinger/jevbench/issues/146
76
  ---
77
 
78
  <div align="center">
79
  <h1>Wald-Q4B v1.1</h1>
80
  <p><strong>Decide directly. Think when needed. Return probabilities.</strong></p>
81
+ <p><a href="https://huggingface.co/Harry19081/Wald-4B">English</a> · <a href="https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/readmes/README.zh.md">简体中文</a> · <a href="https://github.com/Harry19081/wald-4b">GitHub</a> · <a href="https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md">API</a></p>
82
  </div>
83
 
84
+ **Wald-Q4B v1.1 is an open-weight 4B decision model: give it a state and a set of options, and it returns a calibrated probability for every option.** It is for developers who build agents and pipelines and need a fast, self-hosted component to pick a tool, route a request, classify an input or decide whether to ask the user. Unlike a chat model, it does not write an answer you have to parse. It reads the options in one pass (33 ms median on one RTX PRO 6000 with effort `none`) and can optionally think first. It serves a Jev-compatible `POST /v1/systemone` API, is built on Qwen3.5-4B-Base and is released under Apache-2.0.
85
 
86
+ Wald-Q4B is an independent, self-hosted alternative to TypeSafe's hosted Jev API. It is not Jev, contains no Jev weights, and is not affiliated with or endorsed by TypeSafe AI. The Hugging Face repository is `Harry19081/Wald-4B` (earlier name: Wald-4B).
87
 
88
+ **W**ait **A** bit, **L**ook, then **D**ecide. Also named after Abraham Wald, the pioneer of sequential analysis: stop when the evidence is enough.
89
 
90
  ## At a glance
91
 
92
+ - **4B parameters**, built on Qwen3.5-4B-Base; BF16 weights (8.4 GB).
93
+ - **Every option gets a probability.** Question types: `choice` (1–255 named options), `noul` (yes/no) and `score` (ordered levels).
94
+ - **Adjustable thinking:** `none`, `low`, `medium`, `high`, or several thoughts with `high-k`.
95
+ - **Decision Index 0.2.1: 54.59** on the complete suite with `high`; 150,317 requests, all successful. Author-run, pending maintainer validation.
96
+ - **JevBench public set: 203/231** with `none`, ECE 0.041, p50 33 ms / p95 168 ms. Self-scored with JevBench's own harness.
97
+ - **Self-hosted API:** `POST /v1/systemone`, up to 131,072 prompt tokens.
98
+
99
+ ## Quick start
100
+
101
+ On a Linux machine with an NVIDIA GPU and [`uv`](https://docs.astral.sh/uv/):
102
+
103
+ ```sh
104
+ hf download Harry19081/Wald-4B --revision v1.1 --local-dir ./Wald-Q4B
105
+ cd Wald-Q4B
106
+ EFFORT=none ./run.sh "$PWD" # one pass, lowest latency
107
+ # ./run.sh "$PWD" # default: high, the evaluated Decision Index configuration
108
+ ```
109
+
110
+ ```sh
111
+ curl http://localhost:8000/v1/systemone \
112
+ -H 'Content-Type: application/json' \
113
+ -d '{
114
+ "state": "The customer wants to return a damaged kettle.",
115
+ "effort": "medium",
116
+ "questions": {
117
+ "route": {
118
+ "type": "choice",
119
+ "instructions": "Choose the support queue.",
120
+ "criteria": {
121
+ "returns": "Returns and refunds",
122
+ "delivery": "Delivery tracking",
123
+ "other": "Other enquiries"
124
+ }
125
+ }
126
+ }
127
+ }'
128
+ ```
129
+
130
+ The answer for `route` contains the chosen key and a probability for each of `returns`, `delivery` and `other`. Request and response fields, yes/no and score questions, and a clarification example: [API reference](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md). `GET /health` reports the effective policy. The server uses vLLM 0.30.0 and the included `wald-serve` package; Docker and exact evaluation settings are in [RUNBOOK.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md). Generic text-generation calls do not reproduce the decision API's readout.
131
 
132
  ## Thinking effort
133
 
 
141
  | **`high` (default)** | Every eligible question | Up to 512 tokens |
142
  | `high-k2` … `high-k8` | Multiple thoughts; average their answer distributions | Up to 512 tokens per thought |
143
 
144
+ Thinking applies to questions with 2–26 options when context space permits. Larger option sets use grouped readout and a final winner comparison; if a thought cannot fit, the initial answer is kept. Increasing effort spends more computation; it does not guarantee a better answer. **54.59 applies to `high` only; 203/231 applies to `none` only.**
145
 
146
  Set the server default with `EFFORT=medium ./run.sh "$PWD"`, or override it per request with `"effort": "none"`.
147
 
 
149
 
150
  Wald first reads option-letter logits from a plain prompt and turns them into a probability distribution. If the effort policy asks for thinking, it generates a short thought and reads the options again. Bucketed temperature scaling calibrates the returned probabilities.
151
 
152
+ v1.1 combines full-parameter decision training, LoRA refinement, short-thought distillation and RLCD. Training uses **WaldGen**, our generated decision corpus, together with public training datasets. [Data sources](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md) · [Evaluation notes](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md)
153
 
154
  ## Benchmarks
155
 
156
+ | Benchmark | Configuration | Result | Status |
157
+ |---|---|---:|---|
158
+ | **Decision Index 0.2.1 — complete suite** | v1.1 · `high` | **54.59** | Author-run; [PR #30](https://github.com/apolinario/decision-index/pull/30) awaits maintainer validation |
159
+ | **JevBench public set (231 items)** | v1.1 · `none` | **203/231** (87.9%) · ECE 0.041 · Brier 0.188 | Self-scored; [issue #146](https://github.com/fstandhartinger/jevbench/issues/146) asks the maintainers to measure it |
160
 
161
+ **Decision Index:** 150,317/150,317 requests succeeded, including HLE. Measured on one RTX PRO 6000 96 GB with the pinned reproduction kit. [Full results](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results/tree/805716601b2466be324ed6716407b4c3d9267faa/runs/wald-q4b-22d0-f7-full021) · [Per-benchmark scores](https://huggingface.co/Harry19081/Wald-4B/blob/main/evaluation/benchmark-summary.json) · [Reproduction guide](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md)
162
 
163
+ **JevBench:** JevBench's own CLI (`fstandhartinger/jevbench` at `9ec6f15a`, `typesafe` adapter) against the packaged server on loopback, one request at a time, on one RTX PRO 6000. Easy 48/48, original 72/72, hard 83/111; no tokens generated. With `medium`: 205/231, p95 1.80 s. The public items were used as a development scoreboard (never as training data), so this is not a held-out result. The JevBench leaderboard publishes a score only after its maintainers run the model themselves.
164
 
165
  ### Latency
166
 
167
+ With `none`, the JevBench run above measured **33 ms median and 168 ms p95** per decision. A 32-request serial preflight of `high` measured **821 ms median** on the same GPU; this small preflight is not a full-suite latency result or the Decision Index maintainers' admission test. Effort, context length, option count and concurrency all affect speed.
168
 
169
+ ## Related projects and how Wald compares
170
 
171
+ Several projects implement or approximate structured decisions with calibrated option probabilities. The names below belong to their owners; Wald is not affiliated with any of them.
172
 
173
+ - **[Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev)** is TypeSafe AI's hosted decision model behind the `/v1/systemone` API; its weights are closed. Wald accepts the same request shape and runs on your own GPU.
174
+ - **[Kev](https://github.com/jaredpalmer/kev)** by Jared Palmer is an open-weight project that adds a LoRA and a pointer head to Qwen3.5 base models (0.8B, 4B and 9B). Wald's request parsing adapts Kev's Apache-2.0 code ([NOTICE](https://huggingface.co/Harry19081/Wald-4B/blob/main/NOTICE)); Wald reads option letters from the language-model head instead of a separate head.
175
+ - **[Laya](https://huggingface.co/convaiinnovations/laya)** ([code](https://github.com/NandhaKishorM/laya)) is an open-weight 421M ModernBERT-large encoder with a decision head. It is much smaller than Wald and reads up to 512 tokens.
 
 
 
176
 
177
+ **JevBench public set, same 231 items (identical dataset hash), JevBench CLI, run by us:**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
178
 
179
+ | System | How it was run | Correct |
180
+ |---|---|---:|
181
+ | Wald-Q4B v1.1 · `none` | Self-hosted, RTX PRO 6000, 2026-09-29 | 203/231 |
182
+ | Jev (`jev-1.13.0`) | TypeSafe's hosted API, 2026-09-25 | 200/231 |
183
+ | Laya (English checkpoint `55cf4c4e`) | Self-hosted, NVIDIA L4, 2026-09-26 | 134/231 |
184
 
185
+ A 3-item difference on 231 items is within run-to-run and sampling noise. The public items informed Wald's development, and 52 of the 231 states are longer than Laya's 512-token window.
186
+
187
+ **Decision Index 0.2.1:**
188
+
189
+ | System | Index | Source |
190
+ |---|---:|---|
191
+ | Jev (`jev-1.13.0`) | 57.91 | [Leaderboard](https://huggingface.co/spaces/multimodalart/jev-decision-index), maintainer-run (data of 2026-09-28) |
192
+ | Wald-Q4B v1.1 · `high` | 54.59 | Author-run complete suite; not on the leaderboard yet ([PR #30](https://github.com/apolinario/decision-index/pull/30)) |
193
+ | Kev 9B | 38.48 | Leaderboard, maintainer-run (data of 2026-09-28) |
194
+ | Kev 4B | 34.64 | Leaderboard, maintainer-run (data of 2026-09-28) |
195
+
196
+ Leaderboard rows are scored by the maintainers; Wald's number is self-run with the official kit and may change after validation.
197
+
198
+ ## FAQ
199
+
200
+ **Is there an open-source alternative to Jev?** Wald-Q4B is one open-weight option: Apache-2.0 weights and serving code that you run yourself, with a Jev-compatible `/v1/systemone` API. Kev and Laya (above) are other open projects. Wald is independent and is not a TypeSafe release.
201
+
202
+ **Can I use a Jev client with a self-hosted model?** Point the client at your own endpoint. The included server accepts `state` plus typed `questions` (`choice`, `noul`, `score`) at `POST /v1/systemone` and answers with TypeSafe's answer keys. It does not check API keys. See the [API reference](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md).
203
 
204
+ **How do I route tools or decide whether to ask the user?** Send the conversation or task as `state`. For tool routing, ask a `choice` question whose options are your tools. To decide whether to ask a clarifying question, ask a `noul` question such as "Is the request specific enough to act on without asking?" Act when the probability is high, ask when it is low, and set both thresholds on your own validation data. The model picks the tool; it does not write the tool's arguments.
205
 
206
+ **How calibrated are the probabilities?** On the JevBench public set with `none`, expected calibration error is 0.041 (10 bins) and the Brier score is 0.188. Temperatures were fitted on held-out rows of our own development data, with no JevBench items and with known Decision Index matches excluded. A confidence is not a guarantee; check calibration on your task.
207
 
208
+ **Does it run on a single GPU or a laptop?** The shipped server needs one NVIDIA GPU on Linux (vLLM 0.30.0); the BF16 weights are 8.4 GB. The v1.1 measurements come from an RTX PRO 6000 96 GB; earlier builds of the same 4B architecture have also been served with vLLM on a 24 GB NVIDIA L4 at a 16K context limit. CPU, Apple Silicon and laptop setups are not supported by the shipped server and have not been tested.
209
 
210
+ **Kev vs Wald, or Laya vs Wald?** All three are open-weight. Kev adds a pointer head and LoRA to Qwen3.5 base models; Laya is a small encoder with a decision head; Wald is a fully trained 4B decoder with optional thinking. Our same-protocol measurements are in the tables above. Choose on your own task, latency budget and hardware.
211
+
212
+ **Can I fine-tune it for my task?** It is a standard Transformers checkpoint, so common LoRA tooling applies. For v1.0 we trained per-task LoRAs for $0.12–$1.81 of GPU time each; that tooling is not public yet, and those adapters are not validated on v1.1 ([v1.0 notes](https://huggingface.co/Harry19081/Wald-4B/blob/main/history/v1.0/README.md)).
213
+
214
+ **What is the license?** Apache-2.0 for the weights and code. The base model, Qwen3.5-4B-Base, is also Apache-2.0. Some public training sources have their own terms or no stated licence; they are listed in [PROVENANCE.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md). The model and code licence does not grant rights in those texts.
215
+
216
+ ## Limits
217
+
218
+ Confidence is not a guarantee of correctness; validate thresholds on your own task. Oversized prompts are rejected rather than truncated. Known exact training overlaps were filtered, but semantic overlap and pretraining contamination are not ruled out; development used visible benchmark samples. Source-text rights vary. See [evaluation notes](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md) and [source attribution](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md).
219
 
220
  ## Versioning
221
 
 
224
  | **v1.1 — current** | `022D0-f7` | `high` |
225
  | [v1.0 — archive](https://huggingface.co/Harry19081/Wald-4B/tree/v1.0) | `021A0-f10` | `medium` |
226
 
227
+ The v1.0 model card retains its XL, task-LoRA and latency reports, and the [v1.0 walkthrough slides](https://claude.ai/artifact/XfCHVaCuj9A5ectrpaWzV5) describe v1.0 only. Those measurements belong to their documented builds.
228
+
229
+ ## Citation
230
+
231
+ Wald-Q4B v1.1 (2026), an open-weight 4B decision model with calibrated option probabilities. https://huggingface.co/Harry19081/Wald-4B
232
+
233
+ ```bibtex
234
+ @misc{wald_q4b_2026,
235
+ title = {Wald-Q4B v1.1: an open-weight 4B decision model with calibrated option probabilities},
236
+ author = {{Wald-4B authors}},
237
+ year = {2026},
238
+ howpublished = {\url{https://huggingface.co/Harry19081/Wald-4B}},
239
+ note = {Revision v1.1}
240
+ }
241
+ ```
242
+
243
+ Machine-readable: [CITATION.cff](https://huggingface.co/Harry19081/Wald-4B/blob/main/CITATION.cff) · [llms.txt](https://huggingface.co/Harry19081/Wald-4B/blob/main/llms.txt) · [model-info.json](https://huggingface.co/Harry19081/Wald-4B/blob/main/model-info.json)
244
 
245
  ---
246
 
247
+ [Model](https://huggingface.co/Harry19081/Wald-4B) · [GitHub](https://github.com/Harry19081/wald-4b) · [Decision Index results](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results) · Apache-2.0 for weights and code. [Third-party notices](https://huggingface.co/Harry19081/Wald-4B/blob/main/NOTICE) · [Training-data usage notes](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md)
docs/api.md ADDED
@@ -0,0 +1,148 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Wald-Q4B decision API
2
+
3
+ Wald-Q4B's server, `wald-serve`, exposes one decision endpoint, `POST /v1/systemone`. Its request and answer shapes follow TypeSafe's `/v1/systemone` format, so clients written for Jev can point at a self-hosted Wald server. Wald is independent and is not affiliated with TypeSafe AI.
4
+
5
+ Start the server with `./run.sh "$PWD"` in the downloaded model directory ([model card](https://huggingface.co/Harry19081/Wald-4B), [RUNBOOK.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md)). It listens on port 8000 and does not check API keys.
6
+
7
+ ## Endpoints
8
+
9
+ | Method and path | Purpose |
10
+ |---|---|
11
+ | `POST /v1/systemone` (also `POST /`) | Answer one or more typed questions about a state |
12
+ | `GET /health` | Effective policy: effort, gate, thought budget, prompt format, context limit |
13
+ | `GET /v1/models` | The served model name |
14
+
15
+ ## Request
16
+
17
+ | Field | Type | Meaning |
18
+ |---|---|---|
19
+ | `state` | string, object, array or null | What the decision is about: a message, a conversation, a document, an agent trace. Objects and arrays are flattened to text with their field names kept. |
20
+ | `questions` | object, at least one entry | Question id → question. Every question is answered about the same `state`. |
21
+ | `effort` | string, optional | `none`, `low`, `medium`, `high` or `high-k2` … `high-k8`. Overrides the server default for this request. |
22
+ | `model` | string, optional | Accepted for client compatibility; the server answers with the model it serves. |
23
+
24
+ Each question has a `type`, optional `instructions` (any JSON, usually a sentence) and `criteria`:
25
+
26
+ | `type` | `criteria` | Answer |
27
+ |---|---|---|
28
+ | `choice` | Object of 1–255 options: key → description (description may be null) | `choice` (the most probable key), `probabilities` (key → probability), `confidence` |
29
+ | `noul` | Optional object with `true` and/or `false` descriptions | `noul`: the probability of yes |
30
+ | `score` | Array of 1–255 ordered levels | `score` (expected level index), `probabilities` (level index → probability), `confidence` |
31
+
32
+ Every answer also carries `mode`: `A` for a one-pass read, `B` for a read after a thought, `K` for grouped (knockout) reading of wide option sets.
33
+
34
+ Prompts longer than the context limit (131,072 tokens by default) are rejected with HTTP 422, never truncated. A malformed request or unknown effort returns HTTP 400.
35
+
36
+ ## Example: route a tool call
37
+
38
+ ```sh
39
+ curl http://localhost:8000/v1/systemone \
40
+ -H 'Content-Type: application/json' \
41
+ -d '{
42
+ "state": {
43
+ "user": "What will the weather be in Lisbon tomorrow afternoon?",
44
+ "tools_available": ["web_search", "weather_api", "calendar"]
45
+ },
46
+ "effort": "none",
47
+ "questions": {
48
+ "tool": {
49
+ "type": "choice",
50
+ "instructions": "Which tool should the agent call next?",
51
+ "criteria": {
52
+ "web_search": "General web search",
53
+ "weather_api": "Forecast for a city and time",
54
+ "calendar": "Read or create calendar events",
55
+ "none": "Answer directly without a tool"
56
+ }
57
+ }
58
+ }
59
+ }'
60
+ ```
61
+
62
+ Response shape (the numbers here are illustrative, not a measured output):
63
+
64
+ ```json
65
+ {
66
+ "model": "wald-4b",
67
+ "answers": {
68
+ "tool": {
69
+ "type": "choice",
70
+ "choice": "weather_api",
71
+ "probabilities": {"web_search": 0.04, "weather_api": 0.94, "calendar": 0.0, "none": 0.02},
72
+ "confidence": 0.92,
73
+ "mode": "A"
74
+ }
75
+ },
76
+ "usage": {"input_tokens": 142, "output_tokens": 0},
77
+ "latency_s": 0.03
78
+ }
79
+ ```
80
+
81
+ For `choice`, `confidence` rescales the top probability so that 0 means uniform and 1 means certain: `(p_top − 1/K) / (1 − 1/K)` for K options.
82
+
83
+ ## Example: decide whether to ask the user
84
+
85
+ Ask several questions about one state in a single request. Each question is read separately.
86
+
87
+ ```json
88
+ {
89
+ "state": "User: book me a table for Friday",
90
+ "effort": "medium",
91
+ "questions": {
92
+ "specific_enough": {
93
+ "type": "noul",
94
+ "instructions": "Is the request specific enough to act on without asking a follow-up question?",
95
+ "criteria": {"true": "Enough detail to act", "false": "Needs clarification first"}
96
+ },
97
+ "urgency": {
98
+ "type": "score",
99
+ "instructions": "How urgent is this request?",
100
+ "criteria": ["low", "medium", "high"]
101
+ }
102
+ }
103
+ }
104
+ ```
105
+
106
+ Response shape (illustrative numbers):
107
+
108
+ ```json
109
+ {
110
+ "model": "wald-4b",
111
+ "answers": {
112
+ "specific_enough": {"type": "noul", "noul": 0.08, "mode": "B"},
113
+ "urgency": {"type": "score", "score": 1.1, "probabilities": {"0": 0.15, "1": 0.6, "2": 0.25}, "confidence": 0.6, "mode": "A"}
114
+ },
115
+ "usage": {"input_tokens": 310, "output_tokens": 96},
116
+ "latency_s": 0.6
117
+ }
118
+ ```
119
+
120
+ A typical policy: act when `noul` is above a high threshold, ask the user when it is below a low one, and escalate the cases in between. Set both thresholds on validation data from your own task.
121
+
122
+ ## Choosing an effort
123
+
124
+ | Effort | Use it for |
125
+ |---|---|
126
+ | `none` | Lowest latency. One pass, no generated tokens. |
127
+ | `low` / `medium` | Think only when the top initial probability is below 0.5 / 0.7. |
128
+ | `high` | Think on every eligible question. The published Decision Index score uses this setting. |
129
+ | `high-k2` … `high-k8` | Several thoughts, averaged. Slowest. |
130
+
131
+ Thinking applies to questions with 2–26 options when the context has room; otherwise the one-pass answer is returned.
132
+
133
+ ## Python client
134
+
135
+ ```python
136
+ import requests
137
+
138
+ r = requests.post("http://localhost:8000/v1/systemone", json={
139
+ "state": "Refund request for order 1182; the parcel arrived damaged.",
140
+ "effort": "none",
141
+ "questions": {"route": {"type": "choice", "criteria": {
142
+ "refunds": "Refunds and returns", "shipping": "Delivery problems", "other": "Anything else"}}},
143
+ }, timeout=30)
144
+ answer = r.json()["answers"]["route"]
145
+ print(answer["choice"], answer["probabilities"])
146
+ ```
147
+
148
+ Source: [`server/src/wald_serve/wire.py`](https://huggingface.co/Harry19081/Wald-4B/blob/main/server/src/wald_serve/wire.py) (request shapes) and [`server.py`](https://huggingface.co/Harry19081/Wald-4B/blob/main/server/src/wald_serve/server.py) (endpoints).
docs/readmes/README.zh.md CHANGED
@@ -1,21 +1,56 @@
1
  <div align="center">
2
  <h1>Wald-Q4B v1.1</h1>
3
  <p><strong>直接决策,按需思考,返回概率。</strong></p>
4
- <p><a href="../../README.md">English</a> · <a href="README.zh.md">简体中文</a></p>
5
  </div>
6
 
7
- **Wald-Q4B v1.1 是面向结构化决策、可自行部署的开放权重 Jev 替代方案。** 它基于 Qwen3.5-4B-Base,提供可调思考程度和 TypeSafe 兼容的 `/v1/systemone` API,可用于工具选择、agent 路由和选项概率评分。
8
 
9
- **W**ait **A** bit, **L**ook, then **D**ecide:稍等一下,看清楚,再决定。名字也致敬序贯分析先驱 Abraham Wald:证据足够时就停止。
10
 
11
- 给 Wald 一段状态、一个问题和一组选项,它会返回每个选项的概率。可用于工具选择、请求路由、分类和判断是否需要向用户澄清。通过一个 `effort` 参数控制思考程度;无论选择哪个级别,答案都是结构化概率分布。
12
 
13
  ## 一览
14
 
15
- - **4B 参数**,基于 Qwen3.5-4B-Base,BF16 权重。
 
16
  - **可调思考程度**:`none`、`low`、`medium`、`high`,以及多次思考的 `high-k`。
17
- - **完整 Decision Index 0.2.1:54.59**,使用 `high`,150,317 个请求全部成功。
18
- - **可自行部署**:`POST /v1/systemone`;每题最多 131,072 个提示 token、255 个选项。
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
  ## 思考程度
21
 
@@ -29,7 +64,7 @@
29
  | **`high`(默认)** | 每个符合条件的问题 | 最多 512 token |
30
  | `high-k2` … `high-k8` | 多次思考,平均答案分布 | 每次最多 512 token |
31
 
32
- 思考适用于 2–26 个选项且上下文空间足够的问题。更多选项采用分组读取,再比较各组优胜项;若放不下思考文本,则保留初次答案。提高 effort 会增加计算量,但不保证每道题都更准确。**54.59 仅对应 `high`。**
33
 
34
  通过 `EFFORT=medium ./run.sh "$PWD"` 设置服务默认值,也可在单次请求中传入 `"effort": "none"` 覆盖。
35
 
@@ -37,64 +72,73 @@
37
 
38
  Wald 先从普通文本提示末尾读取选项字母的 logits,得到初始概率。如果 effort 策略触发思考,就生成一段短思考,再读取选项。返回的概率经过分桶温度校准。
39
 
40
- v1.1 结合全参数决策训练、LoRA 精修、短思考蒸馏和 RLCD。训练使用我们的自生成决策语料 **WaldGen**,并混合公开训练数据。[数据来源](../../PROVENANCE.md) · [评测说明](../../CONTAMINATION.md)
41
 
42
  ## 评测
43
 
44
- | 基准 | 配置 | 分数 |
45
- |---|---|---:|
46
- | **Decision Index 0.2.1 完整套件** | v1.1 · `high` | **54.59** |
 
47
 
48
- 150,317/150,317 个请求全部成功,包含 HLE。在单张 RTX PRO 6000 96 GB 上使用固定版本的复现工具运行。这是作者自行测量的结果;[提交 #30](https://github.com/apolinario/decision-index/pull/30) 等待维护者验证。
49
 
50
- [完整结果](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results/tree/805716601b2466be324ed6716407b4c3d9267faa/runs/wald-q4b-22d0-f7-full021) · [分项成绩](../../evaluation/benchmark-summary.json) · [复现指南](../../RUNBOOK.md)
51
 
52
  ### 速度
53
 
54
- v1.1 `high` 在 RTX PRO 6000 上的 32 请求串行预检中,**中位延迟为 821 ms**。这只是小规模预检,不代表完整套件延迟或维护者的准入测试。effort、上下文长度、选项数量和并发都会影响速度。
55
 
56
- ## 部署与调用
57
 
58
- 在装有 `uv` 的 Linux CUDA 机器上:
59
 
60
- ```sh
61
- hf download Harry19081/Wald-4B --revision v1.1 --local-dir ./Wald-Q4B
62
- cd Wald-Q4B
63
- ./run.sh "$PWD" # 默认 high
64
- # EFFORT=medium ./run.sh "$PWD" # 按置信度触发思考
65
- ```
66
 
67
- ```sh
68
- curl http://localhost:8000/v1/systemone \
69
- -H 'Content-Type: application/json' \
70
- -d '{
71
- "state": "The customer wants to return a damaged kettle.",
72
- "effort": "medium",
73
- "questions": {
74
- "route": {
75
- "type": "choice",
76
- "instructions": "Choose the support queue.",
77
- "criteria": {
78
- "returns": "Returns and refunds",
79
- "delivery": "Delivery tracking",
80
- "other": "Other enquiries"
81
- }
82
- }
83
- }
84
- }'
85
- ```
86
 
87
- `GET /health` 返回当前生效策略。服务使用 vLLM 0.30.0 和仓库内的 `wald-serve`。Docker 与精确评测配置见[运行指南](../../RUNBOOK.md)。普通文本生成接口不会复现决策 API 的读出流程。
 
 
 
 
88
 
89
- ## 使用限制
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
90
 
91
- 置信度不保证正确性,应在自己的任务上验证阈值。超过上下文限制的提示会被拒绝,不会截断。已过滤已知的严格训练重叠,但无法排除语义重叠和预训练污染;开发过程中使用了可见的基准样本。各来源文本的使用权不同,详见[评测说明](../../CONTAMINATION.md)与[来源声明](../../PROVENANCE.md)。
92
 
93
- ## Jev 与 agent 集成
94
 
95
- **可以接入 Jev 风格的客户端吗?** 内置服务在 `/v1/systemone` 接收 `state` 和带类型的 `questions`,支持选择和是非决策。将客户端指向自建端点,并按[运行指南](../../RUNBOOK.md)核对支持的格式。Wald 是独立模型,不是 Jev 权重,也不属于 TypeSafe AI 官方发布。
96
 
97
- **Agent 应选哪个 effort?** 直接读取用 `none`,按置信度触发思考用 `medium`,复现公布分数用 `high`。它们都返回选项概率,无需从生成文本中解析答案。
 
 
 
 
 
 
98
 
99
  ## 版本
100
 
@@ -103,8 +147,24 @@ curl http://localhost:8000/v1/systemone \
103
  | **v1.1 — 当前版本** | `022D0-f7` | `high` |
104
  | [v1.0 — 历史版本](https://huggingface.co/Harry19081/Wald-4B/tree/v1.0) | `021A0-f10` | `medium` |
105
 
106
- v1.0 模型卡保留其 XL、任务 LoRA 和延迟报告;这些测量属于各自注明的模型版本。
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
107
 
108
  ---
109
 
110
- [机器可读模型信息](../../model-info.json) · 权重与代码:Apache-2.0。[第三方声明](../../NOTICE) · [训练数据使用说明](../../PROVENANCE.md)
 
1
  <div align="center">
2
  <h1>Wald-Q4B v1.1</h1>
3
  <p><strong>直接决策,按需思考,返回概率。</strong></p>
4
+ <p><a href="https://huggingface.co/Harry19081/Wald-4B">English</a> · <a href="https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/readmes/README.zh.md">简体中文</a> · <a href="https://github.com/Harry19081/wald-4b">GitHub</a> · <a href="https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md">API</a></p>
5
  </div>
6
 
7
+ **Wald-Q4B v1.1 是一个开放权重的 4B 决策模型:给它一段状态和一组选项,它为每个选项返回校准过的概率。** 它面向构建 agent 和数据流水线的开发者:需要一个快速、可自行部署的组件来选择工具、路由请求、分类输入,或判断是否需要向用户澄清。和聊天模型不同,它不生成需要再解析的答案,而是一遍读出所有选项的概率(effort `none` 时单张 RTX PRO 6000 上中位延迟 33 ms),也可以先思考再回答。它提供与 Jev 兼容的 `POST /v1/systemone` API,基于 Qwen3.5-4B-Base,以 Apache-2.0 发布。
8
 
9
+ Wald-Q4B 是 TypeSafe 托管 Jev API 之外、可自行部署的独立替代方案。它不是 Jev,不含 Jev 权重,与 TypeSafe AI 没有隶属或背书关系。Hugging Face 仓库为 `Harry19081/Wald-4B`(旧名 Wald-4B)。
10
 
11
+ **W**ait **A** bit, **L**ook, then **D**ecide:稍等一下,看清楚,再决定。名字也致敬序贯分析先驱 Abraham Wald:证据足够时就停止。
12
 
13
  ## 一览
14
 
15
+ - **4B 参数**,基于 Qwen3.5-4B-Base,BF16 权重(8.4 GB)。
16
+ - **每个选项都有概率。** 题型:`choice`(1–255 个命名选项)、`noul`(是/否)、`score`(有序等级)。
17
  - **可调思考程度**:`none`、`low`、`medium`、`high`,以及多次思考的 `high-k`。
18
+ - **完整 Decision Index 0.2.1:54.59**,使用 `high`,150,317 个请求全部成功。作者自测,等待维护者验证。
19
+ - **JevBench 公开集:203/231**,使用 `none`,ECE 0.041,p50 33 ms / p95 168 ms。用 JevBench 自己的评测工具自评。
20
+ - **可自行部署的 API**:`POST /v1/systemone`,最多 131,072 个提示 token。
21
+
22
+ ## 快速开始
23
+
24
+ 在装有 NVIDIA GPU 和 [`uv`](https://docs.astral.sh/uv/) 的 Linux 机器上:
25
+
26
+ ```sh
27
+ hf download Harry19081/Wald-4B --revision v1.1 --local-dir ./Wald-Q4B
28
+ cd Wald-Q4B
29
+ EFFORT=none ./run.sh "$PWD" # 一遍读出,延迟最低
30
+ # ./run.sh "$PWD" # 默认 high,即 Decision Index 的评测配置
31
+ ```
32
+
33
+ ```sh
34
+ curl http://localhost:8000/v1/systemone \
35
+ -H 'Content-Type: application/json' \
36
+ -d '{
37
+ "state": "The customer wants to return a damaged kettle.",
38
+ "effort": "medium",
39
+ "questions": {
40
+ "route": {
41
+ "type": "choice",
42
+ "instructions": "Choose the support queue.",
43
+ "criteria": {
44
+ "returns": "Returns and refunds",
45
+ "delivery": "Delivery tracking",
46
+ "other": "Other enquiries"
47
+ }
48
+ }
49
+ }
50
+ }'
51
+ ```
52
+
53
+ `route` 的答案包含选中的键,以及 `returns`、`delivery`、`other` 各自的概率。请求与响应字段、是非题和评分题、澄清判断示例见 [API 说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md)。`GET /health` 返回当前生效策略。服务使用 vLLM 0.30.0 和仓库内的 `wald-serve`;Docker 与精确评测配置见 [RUNBOOK.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md)。普通文本生成接口不会复现决策 API 的读出流程。
54
 
55
  ## 思考程度
56
 
 
64
  | **`high`(默认)** | 每个符合条件的问题 | 最多 512 token |
65
  | `high-k2` … `high-k8` | 多次思考,平均答案分布 | 每次最多 512 token |
66
 
67
+ 思考适���于 2–26 个选项且上下文空间足够的问题。更多选项采用分组读取,再比较各组优胜项;若放不下思考文本,则保留初次答案。提高 effort 会增加计算量,但不保证每道题都更准确。**54.59 仅对应 `high`;203/231 仅对应 `none`。**
68
 
69
  通过 `EFFORT=medium ./run.sh "$PWD"` 设置服务默认值,也可在单次请求中传入 `"effort": "none"` 覆盖。
70
 
 
72
 
73
  Wald 先从普通文本提示末尾读取选项字母的 logits,得到初始概率。如果 effort 策略触发思考,就生成一段短思考,再读取选项。返回的概率经过分桶温度校准。
74
 
75
+ v1.1 结合全参数决策训练、LoRA 精修、短思考蒸馏和 RLCD。训练使用我们的自生成决策语料 **WaldGen**,并混合公开训练数据。[数据来源](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md) · [评测说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md)
76
 
77
  ## 评测
78
 
79
+ | 基准 | 配置 | 结果 | 状态 |
80
+ |---|---|---:|---|
81
+ | **Decision Index 0.2.1 完整套件** | v1.1 · `high` | **54.59** | 作者自测;[PR #30](https://github.com/apolinario/decision-index/pull/30) 等待维护者验证 |
82
+ | **JevBench 公开集(231 题)** | v1.1 · `none` | **203/231**(87.9%)· ECE 0.041 · Brier 0.188 | 自评;[issue #146](https://github.com/fstandhartinger/jevbench/issues/146) 请维护者自行测量 |
83
 
84
+ **Decision Index:** 150,317/150,317 个请求全部成功,包含 HLE。在单张 RTX PRO 6000 96 GB 上使用固定版本的复现工具运行。[完整结果](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results/tree/805716601b2466be324ed6716407b4c3d9267faa/runs/wald-q4b-22d0-f7-full021) · [分项成绩](https://huggingface.co/Harry19081/Wald-4B/blob/main/evaluation/benchmark-summary.json) · [复现指南](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md)
85
 
86
+ **JevBench:** 使用 JevBench 自己的命令行工具(`fstandhartinger/jevbench` @ `9ec6f15a`,`typesafe` 适配器),在单张 RTX PRO 6000 上通过本机回环逐条请求打包服务。easy 48/48、original 72/72、hard 83/111;不生成任何 token。`medium` 为 205/231,p95 1.80 s。公开题在开发中被用作计分板(从未作为训练数据),因此这不是留出集结果。JevBench 排行榜只在维护者自己跑过模型后才公布分数。
87
 
88
  ### 速度
89
 
90
+ 使用 `none` 时,上述 JevBench 运行的单次决策**中位延迟 33 ms、p95 168 ms**。`high` 在同一 GPU 上的 32 请求串行预检中,**中位延迟为 821 ms**;这只是小规模预检,不代表完整套件延迟或 Decision Index 维护者的准入测试。effort、上下文长度、选项数量和并发都会影响速度。
91
 
92
+ ## 相关项目与对比
93
 
94
+ 多个项目在做带校准选项概率的结构化决策。以下名称归各自所有者;Wald 与它们都没有隶属关系。
95
 
96
+ - **[Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev)** 是 TypeSafe AI 通过 `/v1/systemone` API 提供的托管决策模型,权重不公开。Wald 接受相同的请求格式,运行在你自己的 GPU 上。
97
+ - **[Kev](https://github.com/jaredpalmer/kev)** 是 Jared Palmer 的开放权重项目,在 Qwen3.5 基座(0.8B、4B、9B)上加 LoRA 和指针头。Wald 的请求解析改编自 Kev 的 Apache-2.0 代码(见 [NOTICE](https://huggingface.co/Harry19081/Wald-4B/blob/main/NOTICE));Wald 从语言模型头读取选项字母,不使用单独的头。
98
+ - **[Laya](https://huggingface.co/convaiinnovations/laya)**([代码](https://github.com/NandhaKishorM/laya))是开放权重的 421M ModernBERT-large ���码器加决策头。它比 Wald 小得多,最多读取 512 个 token。
 
 
 
99
 
100
+ **JevBench 公开集,同一批 231 题(数据集哈希相同),JevBench 命令行工具,由我们运行:**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
101
 
102
+ | 系统 | 运行方式 | 答对 |
103
+ |---|---|---:|
104
+ | Wald-Q4B v1.1 · `none` | 自行部署,RTX PRO 6000,2026-09-29 | 203/231 |
105
+ | Jev(`jev-1.13.0`) | TypeSafe 托管 API,2026-09-25 | 200/231 |
106
+ | Laya(英文 checkpoint `55cf4c4e`) | 自行部署,NVIDIA L4,2026-09-26 | 134/231 |
107
 
108
+ 231 题上 3 题的差距在多次运行和抽样的噪声范围内。公开题影响过 Wald 的开发;231 题中有 52 题的状态超过 Laya 的 512 token 窗口。
109
+
110
+ **Decision Index 0.2.1:**
111
+
112
+ | 系统 | 指数 | 来源 |
113
+ |---|---:|---|
114
+ | Jev(`jev-1.13.0`) | 57.91 | [排行榜](https://huggingface.co/spaces/multimodalart/jev-decision-index),维护者运行(2026-09-28 数据) |
115
+ | Wald-Q4B v1.1 · `high` | 54.59 | 作者自测完整套件;尚未上榜([PR #30](https://github.com/apolinario/decision-index/pull/30)) |
116
+ | Kev 9B | 38.48 | 排行榜,维护者运行(2026-09-28 数据) |
117
+ | Kev 4B | 34.64 | 排行榜,维护者运行(2026-09-28 数据) |
118
+
119
+ 排行榜各行由维护者评分;Wald 的分数是用官方工具自测的,验证后可能变化。
120
+
121
+ ## 常见问题
122
+
123
+ **有开源的 Jev 替代品吗?** Wald-Q4B 是一个开放权重的选择:Apache-2.0 的权重和服务代码,自己部署,提供与 Jev 兼容的 `/v1/systemone` API。Kev 和 Laya(见上)是其他开放项目。Wald 是独立项目,不是 TypeSafe 的发布。
124
+
125
+ **能用 Jev 客户端调用自部署模型吗?** 把客户端指向你自己的端点。内置服务在 `POST /v1/systemone` 接收 `state` 和带类型的 `questions`(`choice`、`noul`、`score`),按 TypeSafe 的答案键返回。服务不校验 API key。见 [API 说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md)。
126
+
127
+ **如何做工具路由,或判断是否需要向用户提问?** 把对话或任务作为 `state` 发送。工具路由:提一个 `choice` 问题,选项就是你的工具。是否澄清:提一个 `noul` 问题,例如“这个请求是否足够具体,可以不问就执行?”概率高就执行,概率低就提问,两个阈值都在你自己的验证数据上确定。模型只选工具,不生成工具参数。
128
 
129
+ **概率校准得怎么样?** 在 JevBench 公开集上使用 `none`,期望校准误差为 0.041(10 个分桶),Brier 分数为 0.188。温度是在我们自己开发数据的留出行上拟合的,不含 JevBench 题目,并排除了已知的 Decision Index 匹配项。置信度不是保证,请在你的任务上检查校准。
130
 
131
+ **能在单张 GPU 或笔记本上运行吗?** 内置服务需要 Linux 上的一张 NVIDIA GPU(vLLM 0.30.0);BF16 权重为 8.4 GB。v1.1 的测量来自 RTX PRO 6000 96 GB;同一 4B 架构的早期版本也曾用 vLLM 在 24 GB 的 NVIDIA L4 上以 16K 上下文上限运行。内置服务不支持 CPU、Apple Silicon 和笔记本环境,也没有测试过。
132
 
133
+ **Kev 和 Wald、Laya 和 Wald 怎么选?** 三者都开放权重。Kev 在 Qwen3.5 基座上加指针头和 LoRA;Laya 是带决策头的小型编码器;Wald 是完整训练的 4B 解码器,可选思考。我们在同一协议下的测量见上表。请根据你自己的任务、延迟预算和硬件选择。
134
 
135
+ **能针对我的任务微调吗?** 它是标准的 Transformers checkpoint,常见的 LoRA 工具都适用。v1.0 时我们为单个任务训练 LoRA,每个花费 $0.12–$1.81 的 GPU 时间;这套工具尚未公开,这些 adapter 也未在 v1.1 上验证([v1.0 说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/history/v1.0/README.md))。
136
+
137
+ **许可证是什么?** 权重与代码为 Apache-2.0。基座 Qwen3.5-4B-Base 也是 Apache-2.0。部分公开训练数据有各自的条款或没有注明许可证,列在 [PROVENANCE.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md)。模型与代码的许可证不授予这些文本的权利。
138
+
139
+ ## 使用限制
140
+
141
+ 置信度不保证正确性,应在自己的任务上验证阈值。超过上下文限制的提示会被拒绝,不会截断。已过滤已知的严格训练重叠,但无法排除语义重叠和预训练污染;开发过程中使用了可见的基准样本。各来源文本的使用权不同,详见[评测说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md)与[来源声明](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md)。
142
 
143
  ## 版本
144
 
 
147
  | **v1.1 — 当前版本** | `022D0-f7` | `high` |
148
  | [v1.0 — 历史版本](https://huggingface.co/Harry19081/Wald-4B/tree/v1.0) | `021A0-f10` | `medium` |
149
 
150
+ v1.0 模型卡保留其 XL、任务 LoRA 和延迟报告,[v1.0 讲解幻灯片](https://claude.ai/artifact/XfCHVaCuj9A5ectrpaWzV5)只描述 v1.0。这些测量属于各自注明的模型版本。
151
+
152
+ ## 引用
153
+
154
+ Wald-Q4B v1.1 (2026), an open-weight 4B decision model with calibrated option probabilities. https://huggingface.co/Harry19081/Wald-4B
155
+
156
+ ```bibtex
157
+ @misc{wald_q4b_2026,
158
+ title = {Wald-Q4B v1.1: an open-weight 4B decision model with calibrated option probabilities},
159
+ author = {{Wald-4B authors}},
160
+ year = {2026},
161
+ howpublished = {\url{https://huggingface.co/Harry19081/Wald-4B}},
162
+ note = {Revision v1.1}
163
+ }
164
+ ```
165
+
166
+ 机器可读:[CITATION.cff](https://huggingface.co/Harry19081/Wald-4B/blob/main/CITATION.cff) · [llms.txt](https://huggingface.co/Harry19081/Wald-4B/blob/main/llms.txt) · [model-info.json](https://huggingface.co/Harry19081/Wald-4B/blob/main/model-info.json)
167
 
168
  ---
169
 
170
+ [模型](https://huggingface.co/Harry19081/Wald-4B) · [GitHub](https://github.com/Harry19081/wald-4b) · [Decision Index 结果](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results) · 权重与代码:Apache-2.0。[第三方声明](https://huggingface.co/Harry19081/Wald-4B/blob/main/NOTICE) · [训练数据使用说明](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md)
llms.txt ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Wald-Q4B v1.1
2
+
3
+ > Open-weight 4B decision model. Given a state and typed questions (choice, yes/no, score), it returns a calibrated probability for every option through a Jev-compatible `POST /v1/systemone` API. Built on Qwen3.5-4B-Base; Apache-2.0 weights and serving code; self-hosted on one NVIDIA GPU with vLLM. Independent project, not affiliated with or endorsed by TypeSafe AI.
4
+
5
+ Key facts:
6
+ - Hugging Face repository: Harry19081/Wald-4B (current release tag `v1.1`; earlier name Wald-4B). GitHub: Harry19081/wald-4b.
7
+ - Uses: tool selection, agent routing, classification, deciding whether to ask the user a clarifying question.
8
+ - Effort levels: `none` (one pass, no generated tokens), `low`, `medium`, `high` (default), `high-k2`…`high-k8`.
9
+ - JevBench public set (231 items), effort `none`: 203/231, ECE 0.041, p50 33 ms / p95 168 ms on one RTX PRO 6000. Self-scored with JevBench's harness; the public items were a development scoreboard, not held out. Leaderboard row requested in fstandhartinger/jevbench issue #146.
10
+ - Decision Index 0.2.1 complete suite, effort `high`: 54.59 balanced-skill index. Author-run; submission apolinario/decision-index PR #30 awaits maintainer validation.
11
+
12
+ ## Docs
13
+
14
+ - [Model card](https://huggingface.co/Harry19081/Wald-4B): what it is, benchmarks, comparisons with Jev, Kev and Laya, FAQ, limits
15
+ - [API reference](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md): request and response JSON with examples
16
+ - [RUNBOOK.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md): serving and exact evaluation settings
17
+ - [Chinese model card](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/readmes/README.zh.md)
18
+
19
+ ## Evidence
20
+
21
+ - [Decision Index results dataset](https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results): untouched responses and scores
22
+ - [Decision Index submission PR #30](https://github.com/apolinario/decision-index/pull/30)
23
+ - [JevBench request issue #146](https://github.com/fstandhartinger/jevbench/issues/146)
24
+ - [model-info.json](https://huggingface.co/Harry19081/Wald-4B/blob/main/model-info.json): machine-readable facts
25
+
26
+ ## Optional
27
+
28
+ - [PROVENANCE.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/PROVENANCE.md): training-data sources and their terms
29
+ - [CONTAMINATION.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md): evaluation caveats
30
+ - [CITATION.cff](https://huggingface.co/Harry19081/Wald-4B/blob/main/CITATION.cff)
31
+ - [Server source](https://github.com/Harry19081/wald-4b/tree/main/server)
model-info.json CHANGED
@@ -5,7 +5,7 @@
5
  "Wald-4B"
6
  ],
7
  "checkpoint": "022D0-f7",
8
- "description": "Self-hosted open-weight decision model and independent Jev alternative with adjustable thinking effort and a TypeSafe-compatible /v1/systemone API.",
9
  "base_model": "Qwen/Qwen3.5-4B-Base",
10
  "model_url": "https://huggingface.co/Harry19081/Wald-4B",
11
  "weights_revision": "50f94ecd5d6e7e8459e9c198ef811bfaba12a3fe",
@@ -72,7 +72,36 @@
72
  "zh": "docs/readmes/README.zh.md",
73
  "serving": "RUNBOOK.md",
74
  "sources": "PROVENANCE.md",
75
- "evaluation_notes": "CONTAMINATION.md"
 
 
 
76
  },
77
- "affiliation": "Independent; not affiliated with TypeSafe AI"
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
78
  }
 
5
  "Wald-4B"
6
  ],
7
  "checkpoint": "022D0-f7",
8
+ "description": "Open-weight 4B decision model that returns a calibrated probability for every option through a Jev-compatible POST /v1/systemone API; one pass or optional thinking; self-hosted. Independent, not affiliated with TypeSafe AI.",
9
  "base_model": "Qwen/Qwen3.5-4B-Base",
10
  "model_url": "https://huggingface.co/Harry19081/Wald-4B",
11
  "weights_revision": "50f94ecd5d6e7e8459e9c198ef811bfaba12a3fe",
 
72
  "zh": "docs/readmes/README.zh.md",
73
  "serving": "RUNBOOK.md",
74
  "sources": "PROVENANCE.md",
75
+ "evaluation_notes": "CONTAMINATION.md",
76
+ "api": "docs/api.md",
77
+ "citation": "CITATION.cff",
78
+ "llms": "llms.txt"
79
  },
80
+ "affiliation": "Independent; not affiliated with TypeSafe AI",
81
+ "evaluations": [
82
+ {
83
+ "benchmark": "JevBench",
84
+ "split": "public",
85
+ "items": 231,
86
+ "effort": "none",
87
+ "correct": 203,
88
+ "accuracy": 0.8788,
89
+ "ece": 0.041,
90
+ "brier": 0.188,
91
+ "latency_p50_ms": 33,
92
+ "latency_p95_ms": 168,
93
+ "hardware": "NVIDIA RTX PRO 6000 96 GB",
94
+ "harness": "fstandhartinger/jevbench@9ec6f15a (jevbench.cli, typesafe adapter, serial, loopback)",
95
+ "status": "self-scored; public items were a development scoreboard, not held out; leaderboard measurement requested",
96
+ "request": "https://github.com/fstandhartinger/jevbench/issues/146"
97
+ }
98
+ ],
99
+ "links": {
100
+ "model": "https://huggingface.co/Harry19081/Wald-4B",
101
+ "github": "https://github.com/Harry19081/wald-4b",
102
+ "results_dataset": "https://huggingface.co/datasets/Harry19081/Wald-Q4B-decision-index-results",
103
+ "decision_index_submission": "https://github.com/apolinario/decision-index/pull/30",
104
+ "jevbench_request": "https://github.com/fstandhartinger/jevbench/issues/146",
105
+ "api": "https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/api.md"
106
+ }
107
  }