Wald-4B v1.0

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

CONTAMINATION.md

@@ -0,0 +1,253 @@
+# Contamination statement
+
+This covers the checkpoint submitted as Wald-4B and **every stage of its lineage**, back to the base model. The Decision
+Index rule is that any test item an entry was trained on counts as wrong. The list below is for the maintainers to apply
+that rule. We also count these items wrong in our own blocklist-adjusted scores.
+
+## 1. Decision Index items in our training data
+
+**[contamination/trained-on-di-ids.json](https://huggingface.co/Harry19081/Wald-4B/blob/main/contamination/trained-on-di-ids.json)** lists 363 item ids. For each id it
+gives the benchmark, the lineage stage whose data contains it, and why it is there. Please count all 363 as trained on.
+
+| benchmark | ids | scored in 0.2.1 | why they are in our data |
+|---|---:|---|---|
+| ToolRet (#2) | 67 | yes | **shared upstream source.** ToolRet aggregates the user queries of public tool-calling datasets. Our data contains the train splits of ToolACE and Glaive function-calling. |
+| BANKING77 (#4) | 49 | yes | **public train split duplicates a test sentence.** The BANKING77 train split is in our data, and these test sentences also appear verbatim in train. |
+| CLINC150+OOS (#5) | 44 | yes | **public train split duplicates a test sentence** (as above, for the CLINC-OOS train split). |
+| MMLU-Pro (#57) | 6 | yes | **shared upstream source.** MMLU-Pro includes MATH / TheoremQA problems, and the same problems are in our math data (MATH train). |
+| SATA-Bench (#33) | 4 | yes | **shared upstream source.** Reading passages from RACE, which makes up most of MMLU auxiliary_train. |
+| ANLI (#12) | 3 | yes | **shared upstream text.** The premise text also appears in other public datasets in our data. |
+| WinoGrande (#28) | 2 | yes | **public train split duplicates a test sentence.** |
+| BRIGHT (#36) | 1 | yes | **shared upstream source.** A public programming problem statement. |
+| RouterBench (#6) | 170 | no (removed in 0.2.1) | **shared upstream source.** RouterBench prompts embed GSM8K / MMLU questions, and those questions are in our math and QA training data (GSM8K train and others). |
+| MMLU (#24), ARC-Easy (#26), ARC-Challenge (#27) | 6 / 7 / 4 | display only | **public train split duplicates** (ARC train, MMLU auxiliary_train). These are the same kind of duplicates the board already records for other entries. |
+
+That is 176 ids on scored benchmarks. On our 6,948-request sample, counting them wrong moves the index by 0.3 points:
+53.91 → 53.62 for v1.0 (effort medium, `repeat_state_plain`), and 49.62 → 49.34 for its parent at high-k4.
+
+**How the items got in.** We never trained on a Decision Index test split. Every hit above reached us through a public
+train split or an upstream dataset that the benchmark itself draws from. Stages 1–3 of the lineage were built before we
+had the full blocklist (below), so those stages contain these items. Stage 4 (the task-coverage LoRA) was filtered with the full blocklist before training: 0 hits.
+
+## 2. Training data, by source
+
+This section names every public source in the lineage and the split we read. It gives no proportions: no ratios, token
+or row counts per source, or mixing weights. The training data itself is not released.
+
+### 2.1 The lineage
+
+| stage | what it is | data |
+|---|---|---|
+| 1 | full-parameter decision training of Qwen/Qwen3.5-4B-Base | converted public datasets (§2.2), code-generated decision families, and synthetic tool, document-QA and style items |
+| 2 | LoRA refinement with KL replay (merged) | a curated mix of human-labelled public sets and code-generated items, synthetic hard items, and a KL-only replay of stage-1 rows |
+| 3 | short-thought distillation (LoRA, merged) | synthetic short thoughts on questions from the stage-2 pool, one-pass rows from the same pool, and replay rows whose target is the stage-2 model's own distribution |
+| 4 | task-coverage LoRA (merged; the submitted checkpoint) | the train splits of iSarcasmEval, API-Bank, ContractNLI, VAST, NLI4CT, ACOS and RAGTruth, new Home-appliance households from the Decision Index kit's generator, and a KL-only replay of stage-2 rows |
+
+Stage 1 drew on a public decision-training mixture. Its constituent datasets are listed one by one below, and its custom
+and routing parts were left out. Stage 4 teaches the formats of eight Decision Index benchmarks from their source train
+splits and the kit's own generator. No test item is included. **Home appliance scores 1.00 (skill) for this checkpoint
+on our sample, so please weigh that benchmark with this training in mind.** The maintainers can regenerate the rows
+with the kit's generator and any new seed, and compare them with their test rows.
+
+### 2.2 Public sources
+
+Every public dataset was read through its **train split only**, unless the split column says otherwise. No test split
+of any source was used.
+
+In the stage column, **(replay)** means that the stage re-used rows from that source only as KL replay. For those rows
+the training target is the model's own earlier distribution, not the source label.
+
+The licence column gives the licence **as the source states it**. "card" means the Hugging Face dataset card, and
+"upstream" means the original release, used where the card is empty or says unknown. We have not reviewed these terms
+here.
+
+#### Constituents of the stage-1 public mixture
+
+| dataset | split(s) used | stages | licence (as stated) | note |
+|---|---|---|---|---|
+| [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) (all) | auxiliary_train | 1, 2, 3, 4 (replay) | MIT (card) | dev / validation / test never used. auxiliary_train is mostly RACE passages (see §1, SATA-Bench) |
+| [allenai/ai2_arc](https://huggingface.co/datasets/allenai/ai2_arc) (ARC-Challenge, ARC-Easy) | train | 1, 2, 3, 4 (replay) | CC-BY-SA-4.0 (card) | also in the wide replay (ARC-Easy). Some test questions duplicate train questions (§1) |
+| [tau/commonsense_qa](https://huggingface.co/datasets/tau/commonsense_qa) | train | 1, 2, 3, 4 (replay) | MIT (card) | also read directly as a human-labelled set |
+| [google/boolq](https://huggingface.co/datasets/google/boolq) | train | 1, 2, 3, 4 (replay) | CC-BY-SA-3.0 (card) | also read directly as a human-labelled set |
+| [nyu-mll/multi_nli](https://huggingface.co/datasets/nyu-mll/multi_nli) | train | 1, 2, 3, 4 (replay) | mixed: CC-BY-3.0 / CC-BY-SA-3.0 / MIT / OANC (card) | validation_matched / mismatched never used. Also read directly |
+| [stanfordnlp/snli](https://huggingface.co/datasets/stanfordnlp/snli) | train | 1, 2, 3, 4 (replay) | CC-BY-SA-4.0 (card) | |
+| [mteb/banking77](https://huggingface.co/datasets/mteb/banking77) | train | 1, 2, 3, 4 (replay) | MIT (card); upstream PolyAI release CC-BY-4.0 | some test sentences also appear verbatim in train (§1) |
+| [allenai/openbookqa](https://huggingface.co/datasets/allenai/openbookqa) (main) | train | 1, 2, 3, 4 (replay) | Apache-2.0 (upstream allenai/OpenBookQA) | also read directly as a human-labelled set |
+| [GBaker/MedQA-USMLE-4-options](https://huggingface.co/datasets/GBaker/MedQA-USMLE-4-options) | train | 1, 2, 3, 4 (replay) | CC-BY-4.0 (card); upstream MIT | |
+| [allenai/winogrande](https://huggingface.co/datasets/allenai/winogrande) (winogrande_xl) | train | 1, 2, 3, 4 (replay) | CC-BY-4.0 (upstream allenai/winogrande) | also in the wide replay. Some test sentences duplicate train (§1) |
+| [clinc/clinc_oos](https://huggingface.co/datasets/clinc/clinc_oos) (plus) | train | 1, 2, 3, 4 (replay) | CC-BY-3.0 (card) | also in the wide replay. Some test sentences duplicate train (§1) |
+| [SetFit/amazon_massive_intent_en-US](https://huggingface.co/datasets/SetFit/amazon_massive_intent_en-US) | train | 1, 2, 3, 4 (replay) | CC-BY-4.0 (upstream Amazon MASSIVE) | |
+| [bitext/Bitext-customer-support-llm-chatbot-training-dataset](https://huggingface.co/datasets/bitext/Bitext-customer-support-llm-chatbot-training-dataset) | train (the only split) | 1, 2, 3, 4 (replay) | CDLA-Sharing-1.0 (card) | |
+| [fancyzhx/dbpedia_14](https://huggingface.co/datasets/fancyzhx/dbpedia_14) | train | 1, 2, 3, 4 (replay) | CC-BY-SA-3.0 (card) | also read directly as a human-labelled set |
+| [google-research-datasets/go_emotions](https://huggingface.co/datasets/google-research-datasets/go_emotions) (simplified) | train | 1, 2, 3, 4 (replay) | Apache-2.0 (card) | |
+| [SetFit/hate_speech_offensive](https://huggingface.co/datasets/SetFit/hate_speech_offensive) | train | 1, 2, 3, 4 (replay) | MIT (upstream t-davidson/hate-speech-and-offensive-language) | |
+| [SetFit/amazon_counterfactual_en](https://huggingface.co/datasets/SetFit/amazon_counterfactual_en) | train | 1, 2, 3, 4 (replay) | CC-BY-SA-4.0 (upstream amazon-research/amazon-multilingual-counterfactual-dataset) | |
+| [google/civil_comments](https://huggingface.co/datasets/google/civil_comments) | train (a leading slice) | 1, 2, 3, 4 (replay) | CC0-1.0 (card) | also in the wide replay |
+| [ucirvine/sms_spam](https://huggingface.co/datasets/ucirvine/sms_spam) | train (the only split) | 1, 2, 3, 4 (replay) | CC-BY-4.0 (upstream UCI SMS Spam Collection) | |
+| [allenai/qasc](https://huggingface.co/datasets/allenai/qasc) | train | 1, 2, 3, 4 (replay) | CC-BY-4.0 (card) | also in the wide replay |
+| [Rowan/hellaswag](https://huggingface.co/datasets/Rowan/hellaswag) | train | 1, 2, 3, 4 (replay) | MIT (upstream rowanz/hellaswag) | validation / test never used |
+| [ybisk/piqa](https://huggingface.co/datasets/ybisk/piqa) | train | 1, 2, 3, 4 (replay) | unknown | also in the wide replay |
+| [LabHC/bias_in_bios](https://huggingface.co/datasets/LabHC/bias_in_bios) | train | 1, 2, 3, 4 (replay) | MIT (card) | |
+| [nvidia/HelpSteer2](https://huggingface.co/datasets/nvidia/HelpSteer2) | train | 2, 3 | CC-BY-4.0 (card) | |
+| [nvidia/HelpSteer3](https://huggingface.co/datasets/nvidia/HelpSteer3) (preference) | train | 1, 2, 3, 4 (replay) | CC-BY-4.0 (card) | also in the wide replay |
+| [ucberkeley-dlab/measuring-hate-speech](https://huggingface.co/datasets/ucberkeley-dlab/measuring-hate-speech) | train (the only split) | 1, 2, 3, 4 (replay) | CC-BY-4.0 (card) | |
+| [chengxuphd/liar2](https://huggingface.co/datasets/chengxuphd/liar2) | train | 1, 2, 3, 4 (replay) | Apache-2.0 (card) | |
+| [allenai/prosocial-dialog](https://huggingface.co/datasets/allenai/prosocial-dialog) | train | 1, 2, 3, 4 (replay) | CC-BY-4.0 (card) | also in the wide replay |
+| [HuggingFaceH4/ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) | train_prefs | 1, 2, 3, 4 (replay) | MIT (card) | test_prefs never used |
+| [Anthropic/hh-rlhf](https://huggingface.co/datasets/Anthropic/hh-rlhf) | train | 1, 2, 3, 4 (replay) | MIT (card) | |
+| [glaiveai/glaive-function-calling-v2](https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2) | train (the only split) | 1, 2, 3, 4 (replay) | Apache-2.0 (card) | also a seed source for the synthetic tool items. Shares user queries with ToolRet (§1) |
+| [Team-ACE/ToolACE](https://huggingface.co/datasets/Team-ACE/ToolACE) | train (the only split) | 1, 2, 3, 4 (replay) | Apache-2.0 (card) | also in the wide replay and the tool items. Shares user queries with ToolRet (§1) |
+| [copenlu/fever_gold_evidence](https://huggingface.co/datasets/copenlu/fever_gold_evidence) | train | 1, 2, 3, 4 (replay) | unknown | |
+| [openlifescienceai/medmcqa](https://huggingface.co/datasets/openlifescienceai/medmcqa) | train | 1, 2, 3, 4 (replay) | Apache-2.0 (card); upstream MIT | also in the wide replay |
+| [osunlp/Mind2Web](https://huggingface.co/datasets/osunlp/Mind2Web) | train | 1, 2, 3, 4 (replay) | CC-BY-4.0 (card) | also a seed source for the synthetic tool items |
+
+#### Wide replay of public train splits (sources not listed above)
+
+| dataset | split(s) used | stages | licence (as stated) | note |
+|---|---|---|---|---|
+| [allenai/cosmos_qa](https://huggingface.co/datasets/allenai/cosmos_qa) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-4.0 (card) | |
+| [ucinlp/drop](https://huggingface.co/datasets/ucinlp/drop) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-SA-4.0 (card) | |
+| [allenai/quoref](https://huggingface.co/datasets/allenai/quoref) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-4.0 (card) | |
+| [allenai/quartz](https://huggingface.co/datasets/allenai/quartz) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-4.0 (card) | |
+| [allenai/ropes](https://huggingface.co/datasets/allenai/ropes) (plain_text) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-4.0 (card) | |
+| [alisawuffles/WANLI](https://huggingface.co/datasets/alisawuffles/WANLI) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-4.0 (card) | crowd labels that revise model-written candidates |
+| [allenai/social_i_qa](https://huggingface.co/datasets/allenai/social_i_qa) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-4.0 (card text) | |
+| [deepmind/aqua_rat](https://huggingface.co/datasets/deepmind/aqua_rat) (raw) | train | 1, 2 (replay), 3, 4 (replay) | Apache-2.0 (card) | |
+| [ChilleD/SVAMP](https://huggingface.co/datasets/ChilleD/SVAMP) | train | 1, 2 (replay), 3, 4 (replay) | MIT (card) | |
+| [math-eval/TAL-SCQ5K](https://huggingface.co/datasets/math-eval/TAL-SCQ5K) (EN) | train | 1, 2 (replay), 3, 4 (replay) | MIT (card) | |
+| [tasksource/ruletaker](https://huggingface.co/datasets/tasksource/ruletaker) | train | 1, 2 (replay), 3, 4 (replay) | Apache-2.0 (card) | |
+| [ImperialCollegeLondon/health_fact](https://huggingface.co/datasets/ImperialCollegeLondon/health_fact) (PUBHEALTH) | train | 1, 2 (replay), 3, 4 (replay) | MIT (card) | |
+| [wenhu/tab_fact](https://huggingface.co/datasets/wenhu/tab_fact) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-4.0 (card) | |
+| [lighteval/wikitablequestions](https://huggingface.co/datasets/lighteval/wikitablequestions) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-SA-4.0 (upstream ppasupat/WikiTableQuestions) | |
+| [AmazonScience/massive](https://huggingface.co/datasets/AmazonScience/massive) (en-US) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-4.0 (card) | |
+| [google-research-datasets/schema_guided_dstc8](https://huggingface.co/datasets/google-research-datasets/schema_guided_dstc8) (SGD) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-SA-4.0 (card) | |
+| [tasksource/esci](https://huggingface.co/datasets/tasksource/esci) (us) | train | 1, 2 (replay), 3, 4 (replay) | Apache-2.0 (card) | |
+| [google-research-datasets/poem_sentiment](https://huggingface.co/datasets/google-research-datasets/poem_sentiment) | train | 1, 2 (replay), 3, 4 (replay) | CC-BY-4.0 (card) | |
+| [zeroshot/twitter-financial-news-sentiment](https://huggingface.co/datasets/zeroshot/twitter-financial-news-sentiment) | train | 1, 2 (replay), 3, 4 (replay) | MIT (card) | |
+| [NousResearch/hermes-function-calling-v1](https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1) (glaive subset) | train | 1, 2 (replay), 3, 4 (replay) | Apache-2.0 (card) | |
+| [openbmb/UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) (evol_instruct, ultrachat) | train | 1, 2 (replay), 3, 4 (replay) | MIT (card) | ratings in the source are model-written |
+| [jackhhao/jailbreak-classification](https://huggingface.co/datasets/jackhhao/jailbreak-classification) | train | 1, 2 (replay), 3, 4 (replay) | Apache-2.0 (card) | |
+| [deepset/prompt-injections](https://huggingface.co/datasets/deepset/prompt-injections) | train | 1, 2 (replay), 3, 4 (replay) | Apache-2.0 (card) | |
+| [tonytan48/TempReason](https://huggingface.co/datasets/tonytan48/TempReason) | train_l1 | 1, 2 (replay), 3, 4 (replay) | CC-BY-SA-3.0 (card) | answers recomputed by code |
+
+#### Other public decision sets
+
+| dataset | split(s) used | stages | licence (as stated) | note |
+|---|---|---|---|---|
+| [jaredpalmer/kev-suites](https://huggingface.co/datasets/jaredpalmer/kev-suites) | v7/decision-v7/train.jsonl | 2, 3 | Apache-2.0 (card); rows derive from public datasets under their own terms | Kev's public training partition. Its rows come from the BANKING77, BoolQ, DBpedia-14 and MNLI train splits |
+| [thu-coai/cold](https://huggingface.co/datasets/thu-coai/cold) (COLD, Chinese) | train | 2, 3 | Apache-2.0 (repo) | test never used for training |
+| [ZefanCai/Open-Jev](https://huggingface.co/datasets/ZefanCai/Open-Jev) | train | 1, 2, 3, 4 (replay) | CC0-1.0 (card); code MIT | labels by rules, solvers and generator latents. Its calibration / validation / test / OOD rows were never used. `customer-control-v1` excluded |
+| [ZefanCai/Open-Jev-v1.1](https://huggingface.co/datasets/ZefanCai/Open-Jev-v1.1) | train (`wanli-decisions-v1`, `community-diversity-v2`) | 2, 3 | CC0-1.0 (community-diversity-v2); CC-BY-4.0 (wanli-decisions-v1, from WANLI) | |
+
+#### Prompt sources for routing and verification items
+
+The items themselves are ours. A routing item labels a public prompt by the kind of source it came from, and a
+verification item plants an error in a human-written solution by code.
+
+| dataset | split(s) used | stages | licence (as stated) | note |
+|---|---|---|---|---|
+| [openai/gsm8k](https://huggingface.co/datasets/openai/gsm8k) (main) | train | 1, 2, 3, 4 (replay) | MIT (card) | routing prompts and the worked solutions of the verification items. Test never used. RouterBench embeds GSM8K questions (§1) |
+| [EleutherAI/hendrycks_math](https://huggingface.co/datasets/EleutherAI/hendrycks_math) | train | 1, 2, 3, 4 (replay) | MIT (card) | test never used. MMLU-Pro includes MATH problems (§1) |
+| [deepmind/code_contests](https://huggingface.co/datasets/deepmind/code_contests) | train | 1, 2, 3, 4 (replay) | CC-BY-4.0 (card) | valid / test never used |
+| [google-research-datasets/mbpp](https://huggingface.co/datasets/google-research-datasets/mbpp) (full) | train, validation, prompt | 2, 3 | CC-BY-4.0 (card) | test never used |
+| [princeton-nlp/SWE-bench](https://huggingface.co/datasets/princeton-nlp/SWE-bench) | train | 1, 2, 3, 4 (replay) | MIT (card) | test (the benchmark) never used |
+| [mandarjoshi/trivia_qa](https://huggingface.co/datasets/mandarjoshi/trivia_qa) (rc.nocontext) | train | 1, 2, 3, 4 (replay) | Apache-2.0 (card) | |
+| [abisee/cnn_dailymail](https://huggingface.co/datasets/abisee/cnn_dailymail) (3.0.0) | train | 1, 2, 3, 4 (replay) | Apache-2.0 (card) | |
+| [OpenAssistant/oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1) | train | 1, 2, 3, 4 (replay) | Apache-2.0 (card) | |
+| [HuggingFaceM4/WebSight](https://huggingface.co/datasets/HuggingFaceM4/WebSight) (v0.2) | train (the only split) | 1, 2, 3 | CC-BY-4.0 (card) | |
+
+#### Text embedded in long states
+
+| dataset | split(s) used | stages | licence (as stated) | note |
+|---|---|---|---|---|
+| [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) (sample-10BT) | train (the only split) | 1, 2, 3, 4 (replay) | ODC-BY-1.0 (card) | documents placed inside code-generated long-state items |
+| [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) (20231101.en) | train (the only split) | 1, 2, 3, 4 (replay) | CC-BY-SA-3.0 and GFDL (card) | as above |
+
+#### Seed sources of the synthetic tool items
+
+| dataset | split(s) used | stages | licence (as stated) | note |
+|---|---|---|---|---|
+| [nvidia/When2Call](https://huggingface.co/datasets/nvidia/When2Call) | train_sft | 1 | CC-BY-4.0 (card) | test never used |
+| [liminghao1630/API-Bank](https://huggingface.co/datasets/liminghao1630/API-Bank) | training-data: lv1 (stage 1); lv1 + lv2 (stage 4) | 1, 4 | MIT (card); the GitHub repo AlibabaResearch/DAMO-ConvAI (api-bank) states Apache-2.0 | test-data never used. In stage 4, 53-option tool catalogs are built the kit's way from the train tool pool |
+| [AgentGym/AgentTraj-L](https://huggingface.co/datasets/AgentGym/AgentTraj-L) | train | 1 | unknown | |
+
+#### Stage 4: Decision Index benchmark sources
+
+| dataset | split(s) used | stages | licence (as stated) | note |
+|---|---|---|---|---|
+| [iSarcasmEval](https://github.com/iabufarha/iSarcasmEval) (SemEval-2022 Task 6) | train.En.csv (tasks A-En, B-En), train.Ar.csv (task A-Ar) | 4 | MIT (repo LICENSE); the README asks for the task citation | test never used |
+| [ContractNLI](https://github.com/stanfordnlp/contract-nli) | train | 4 | CC BY 4.0 (README / project site) | train rows only. Rows that share clauses with test contracts were dropped |
+| [VAST](https://github.com/emilyallaway/zero-shot-stance) @ e7c4775 | data/VAST/vast_train.csv | 4 | unknown | test never used |
+| [NLI4CT](https://github.com/ai-systems/Task-2-SemEval-2024) (SemEval-2024 Task 2) @ 7f32fa6 | train.json, dev.json | 4 | unknown | test never used |
+| [ACOS](https://github.com/NUSTM/ACOS) @ 45d179a | data/*/*_quad_train.tsv | 4 | unknown | test never used |
+| [RAGTruth](https://github.com/ParticleMedia/RAGTruth) @ c103204 | response.jsonl split = train, source_info.jsonl | 4 | MIT (repo LICENSE); contexts: unknown | test never used |
+| [Decision Index kit](https://github.com/apolinario/decision-index) @ 19ad28e, Home-appliance generator (`decision_index/suite/build/home_appliance.py`) | new households from our own seed | 4 | MIT (repo) | code, not a dataset. 0 states are shared with the benchmark's dev + test rows |
+
+That is **84 public sources**: 83 datasets and one generator.
+
+### 2.3 Synthetic data
+
+Everything that is not a public source above falls into four categories.
+
+- **Code-generated and code-labelled items.** These are decision families whose answer a program computes: rule
+  application, dates and business days, numeric extraction, lookup, ordering, long policies with amendments and decoys,
+  temporal traces, multi-hop and buried-evidence long states, and tool-use worlds. The judge items plant an error in a
+  human-written GSM8K solution by code, so the label holds by construction. The routing items take their label from the
+  prompt's source dataset. This category also includes the rule-based renderings (JSON, described, wide and
+  single-question forms) and the rule-twin items that ship with the stage-1 mixture's public code, and the stage-4
+  Home-appliance households.
+- **Synthetic items and short thoughts.** Part of the data is synthetic: items and short thoughts written or labelled
+  by much larger frontier LLMs — far above 120B parameters where the size is published. An item is kept only when
+  checks by code or independent answers agree and it passes our schema, shortcut and overlap gates; a thought is kept
+  only if its final answer matches the gold label.
+- **Our own model's errors, used for judge data.** None of the four stages' training files contain this category. The
+  judge items of v1.0 are the code-planted errors described above, plus synthetic judge items.
+- **KL replay.** Stages 2, 3 and 4 re-show earlier rows, and the target is the model's own distribution before that
+  stage (§2.2, "(replay)"). This keeps earlier abilities while the new rows are learned. No label is added.
+
+The training data is not released.
+
+### 2.4 How Decision Index items were screened
+
+- **Stages 1–3** were built before the full blocklist existed. For stage 3, we removed the 5 rows that held the strictly
+  verified hits known at the time. Every remaining hit in the stage-1, stage-2 and stage-3 training files is listed in
+  §1: 363 ids in [contamination/trained-on-di-ids.json](https://huggingface.co/Harry19081/Wald-4B/blob/main/contamination/trained-on-di-ids.json). Please count them as
+  trained on.
+- **The blocklist.** Since 2026-09-27, every Decision Index 0.2 request we rebuilt (all but HLE) is on a permanent
+  blocklist in our training repository (`trainer/data/blocklists`). Items are stored as fingerprints, not text. The data acceptance gate fails any
+  training row that matches it (§3).
+- **Stage 4** was checked row by row before training against the full blocklist, our sample rows, the calibration
+  holdouts and our vertical test sets. It passed the gate with 0 rows that contain a Decision Index item.
+
+## 3. How we scanned
+
+- **Suite.** The Decision Index 0.2 suite was rebuilt with the maintainers' public kit (github.com/apolinario/decision-index
+  @ 19ad28e) from its pinned public sources. It has 154,877 requests; HLE (501 rows, gated) is not included.
+- **Match rule** (applied to every string of every training record):
+  - Words are lower-case `[a-z0-9]+`.
+  - Each item's state is taken after stripping the benchmark's fixed instruction line. A shared prompt prefix is
+    stripped too, so training text without the DI prompt is still recognized.
+  - A state of 4–11 words must appear verbatim.
+  - A state of 12 words or more needs ≥ 80 % of its 8-word shingles in one record. Shingles that occur in more than 20
+    states are dropped as boilerplate.
+  - Benchmarks whose options are the item's content (MMLU / ARC / MMLU-Pro / GPQA / BBH / HellaSwag / WinoGrande …)
+    also need ≥ 50 % of the option text to match.
+  - States under 8 words are recorded as "weak" and not counted.
+- **Coverage.** The scan covered every training file of every stage of this lineage, plus sibling and ancestor corpora.
+  The per-file scan outputs (line counts, file sha256, hit counts) are summarized in the `scans` field of
+  trained-on-di-ids.json.
+- **Blocklist.** Since 2026-09-27 every Decision Index item is on a permanent blocklist, stored as fingerprints, not
+  text. The data acceptance gate fails any training row that hits it.
+- **Items the scan cannot see.** Items without free text cannot be checked this way: CLINC150 short intents, POP909 and
+  cfcolor. Nothing in the pretraining data of the base model was scanned.
+
+## 4. What we did not do
+
+- We never used a Decision Index test split, the kit's suite rows or the sample rows for training, selection or
+  calibration.
+- We never tuned the prompt, the readout or the effort policy per benchmark. The knockout for more than 26 options is one
+  rule for every question.
+- We selected checkpoints on our own holdouts only. The Decision Index was read once per candidate checkpoint.
+- One disclosure: the benchmarks that stage 4 covers were chosen from our Decision Index sample reads, as the
+  benchmarks where we trailed Jev most. That choice used scores only. No items were used, and the stage has no
+  checkpoint selection.

Dockerfile

@@ -0,0 +1,20 @@
+# Wald-4B /v1/systemone server: vLLM 0.30.0 (bf16) on the weights as a loopback-only sidecar + wald-serve in front.
+#
+#   docker build -t wald-serve .
+#   docker run --gpus all -v /path/to/Wald-4B:/model:ro -p 8000:8000 wald-serve
+#   # ready when GET http://127.0.0.1:8000/health returns {"ok": true, ...}
+#
+# The weights directory holds config, tokenizer, bf16 safetensors, temperature.json and serving.json (declared policy,
+# prompt format, context limit). Extra vLLM arguments: -e WALD_VLLM_ARGS="...". Another policy: -e WALD_EFFORT=high.
+FROM python:3.12-slim
+
+RUN pip install --no-cache-dir uv==0.9.5 \
+ && uv pip install --system --no-cache "vllm==0.30.0" \
+ && python -c "import importlib.metadata as m; print('vllm', m.version('vllm'), 'torch', m.version('torch'))"
+COPY server /app/server
+RUN uv pip install --system --no-cache /app/server
+ENV HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 TOKENIZERS_PARALLELISM=false PYTHONUNBUFFERED=1 \
+    VLLM_USE_FLASHINFER_SAMPLER=0 VLLM_NO_USAGE_STATS=1 DO_NOT_TRACK=1 \
+    WALD_EFFORT="" WALD_VLLM_ARGS=""
+EXPOSE 8000
+CMD ["sh", "-c", "exec wald-serve --model /model --port 8000 ${WALD_EFFORT:+--effort $WALD_EFFORT} --vllm-args \"$WALD_VLLM_ARGS\""]

LICENSE

@@ -0,0 +1,202 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+

MANIFEST.json

@@ -0,0 +1,34 @@
+{
+ "chat_template.jinja": {
+  "bytes": 7756,
+  "sha256": "a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715"
+ },
+ "config.json": {
+  "bytes": 1979,
+  "sha256": "4ccab9e001f1b53e8213882a6e36c9aa3db09413aa2cf1e11caf18085077aabe"
+ },
+ "generation_config.json": {
+  "bytes": 116,
+  "sha256": "62153eb6c69f2e1f426beaa8002b7186437e949c7588167085df14e10e9c0a73"
+ },
+ "model.safetensors": {
+  "bytes": 8411558400,
+  "sha256": "dabeb7bc3f43bf6ea7d20ecfdac8758b3945517a991809a5772988829598e893"
+ },
+ "serving.json": {
+  "bytes": 126,
+  "sha256": "34ae0348497abc99bedee0e8f65b4f6de5bc7888f7347ca30cb926b75acf8ca6"
+ },
+ "temperature.json": {
+  "bytes": 2717,
+  "sha256": "a0f72cd2d0a653e81051e5a0c77fc1a69131552a8102b580a93a6dbe7908b2da"
+ },
+ "tokenizer.json": {
+  "bytes": 19989509,
+  "sha256": "bd53432f0de26d67a83b634040d4f043053da4ecd0c759e7f4b24ad4f8bb9a81"
+ },
+ "tokenizer_config.json": {
+  "bytes": 1127,
+  "sha256": "171ecbe7ddae98d11840698f7df2b8d5b4722139db0f0620d3bbf429bd656250"
+ }
+}

NOTICE

@@ -0,0 +1,19 @@
+Wald-4B
+Copyright 2026 the Wald-4B authors
+
+The weights and the serving code in this repository are licensed under the Apache License, Version 2.0 (see LICENSE).
+
+Base model
+  The weights are derived from Qwen/Qwen3.5-4B-Base (https://huggingface.co/Qwen/Qwen3.5-4B-Base), licensed under the
+  Apache License, Version 2.0, copyright the Qwen team, Alibaba Cloud. Its licence and notices continue to apply.
+
+Third-party code and text in server/
+  server/src/wald_serve/wire.py adapts the request models and the state / option rendering (render, option_text,
+  question_keys, to_record) of Kev (https://github.com/jaredpalmer/kev), Copyright Jared Palmer, licensed under the
+  Apache License, Version 2.0. Modified: the record carries each question's type and keys; unused fields removed.
+
+  server/src/wald_serve/prompt.py contains the sentence STATE_REPEAT, copied verbatim from simple-jev
+  (https://github.com/featherless-ai/simple-jev, commit dae340e, hf-server/hf_prompt_policies.py), licensed under the
+  Apache License, Version 2.0. It is used only by the `repeat_state_plain` prompt format.
+
+No benchmark items or training data are included in this repository.

README.md

@@ -0,0 +1,201 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3.5-4B-Base
+base_model_relation: finetune
+language:
+- en
+tags:
+- decision-model
+- calibration
+- typesafe
+- decision-index
+pipeline_tag: text-generation
+---
+
+<div align="center">
+  <h1>Wald-4B v1.0</h1>
+  <p><strong>A calibrated decision model: one forward pass turns a state and a question into a probability for every option.</strong></p>
+</div>
+
+**W**ait **A** bit, **L**ook, then **D**ecide. Sure? Decide now. Not sure? Look once more. Also named after Abraham Wald (1902–1950), who founded sequential analysis: stop as soon as the evidence is enough.
+
+---
+
+<p align="center"><a href="https://huggingface.co/Harry19081/Wald-4B">English</a> · <a href="https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/readmes/README.zh.md">简体中文</a></p>
+
+---
+
+**Wald-4B turns a decision into one forward pass.** Give it a state and a question, and it returns a calibrated
+probability for every option: which tool to call, whether to ask the user, which intent, whether an output is safe. Act
+above 0.9, ask a person below 0.6, and send the cases in between to a larger model. Under effort `medium` it thinks
+(≤ 512 tokens) only when its top probability is below 0.7, about one question in nine; the answer is always a
+distribution, never parsed text.
+
+## At a glance
+
+- **Decision Index 0.2.1: 53.91** (sample estimate), #6 of 67 on the board and the best 4B entry ([Decision Index](#decision-index-021)).
+- **XL hidden split: 63.2**, #2 behind Jev and the best 4B ([XL](#xl-hidden-split)).
+- **Your task for under $2:** one LoRA beats Jev on 9 of 12 public tasks ([Verticals](#verticals-a-quick-lora-per-task)).
+- **~50 ms per decision** at p50, about 1/5 of Jev's price per 1k decisions ([Latency and cost](#latency-and-cost)).
+
+![Decision Index vs model size](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/di-vs-size.svg)
+
+## How it works
+
+A 4B model built on Qwen/Qwen3.5-4B-Base @ `1001bb4d826a52d1f399e183466143f4da7b741b`: full-parameter decision training,
+then three merged LoRAs (refinement with KL replay, short-thought distillation, task coverage). Each question becomes one
+plain prompt with no chat template and no new tokens. The state is written twice (format `repeat_state_plain`; the
+second copy is introduced by one fixed sentence, see NOTICE), then:
+
+```
+Question: {instructions}
+(A) {option 1}
+(B) {option 2}
+Answer: (
+```
+
+The option-letter logits at the last position give the distribution. Above 26 options, options are read in ⌈n / 26⌉
+chunks and the chunk winners are read again (knockout). A temperature per (question type, option count), fitted on our own
+held-out rows, changes confidence only. The prompt format, the policy and the context limit are fixed in `serving.json`
+and are the same for every benchmark.
+
+**Training data** (not released): four stages over 84 public sources (83 datasets read through their train splits only, NLI4CT also its dev split,
+plus the Decision Index kit's MIT Home-appliance generator), code-generated and code-labelled decision items, and KL
+replay of earlier rows. Part of the data is synthetic: items and short thoughts written or labelled by much larger
+frontier LLMs — far above 120B parameters where the size is published. Every source, and how Decision Index items
+were screened: [CONTAMINATION.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md).
+
+## Benchmarks
+
+What we measured: the Decision Index 0.2.1 (the board), XL (a decision benchmark with a hidden split), task LoRAs on
+public verticals, and serving latency. All numbers are our own reads, zero-shot unless marked, with the same requests
+for every system; rows marked *board* are the official leaderboard's.
+
+### Decision Index 0.2.1
+
+Our read is a **stratified sample of 6,948 requests** (37,469 questions; the kit's own sampler and seed; HLE not
+rebuilt), scored with the 0.2.1 rules. Jev reads 57.19 on this sample against its board 57.89, so the sample reads about
+0.7 points low. Board rows are official full-suite numbers ([board](https://huggingface.co/spaces/multimodalart/jev-decision-index)).
+
+| system | size | Decision Index 0.2.1 | source |
+|---|---|---|---|
+| Jev (hosted API, jev-1.13.0) | undisclosed | 57.19 [55.29, 58.60] · board 57.89 | ours, sample · board |
+| simple-jev · Qwen3.8-27B (board #4) · Jebadiah 27B (#5) | 27B | 55.74 · 54.67 | board |
+| **Wald-4B v1.0 · effort medium** | 4B | **53.91** [51.87, 55.27] | ours, sample |
+| reflex 27B (board #6) · Decider chat · Qwen3.6-27B (#7) | 27B | 52.16 · 51.35 | board |
+| Qwen3.8-27B, untrained, our letter readout (one pass) | 27B | 47.78 [45.95, 49.17] | ours, sample |
+| Decider 35B-A3B (board #11) | 35B-A3B | 47.11 | board |
+| Decider 4B (board #15) | 4B | 40.70 | board |
+| Kev 9B (board #23) · Kev 4B (#28) | 9B · 4B | 38.48 · 34.64 | board |
+
+53.91 places #6 on the 67-entry board (between Jebadiah 27B and reflex 27B); against Jev on the same sample it is
+−3.3 [−4.9, −1.8]. On the same 4B base it is +19.3 over Kev 4B.
+
+![Decision Index by area](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/di-areas.svg)
+
+**Disclosures.** 363 Decision Index item ids reached our training data through public train splits or shared upstream
+sources (never a test split); counting them wrong gives **53.62**. Home appliance scores 1.00 (skill): the last stage
+trained on new households from the benchmark's own MIT generator (new seed, 0 shared states); with Home appliance held
+at the parent build's answers the index is 51.57 [49.56, 52.95]. Details: [CONTAMINATION.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md).
+
+**Same-request suite.** JevBench public 231 (accuracy %): Wald-4B v1.0 **88.3** (packaged server), Jev 86.6, raw
+Qwen3.5-4B-Base 67.5, Laya 421M 58.0, CLM-8B 39.0. CLM-8B and Laya have no Decision Index read.
+
+### XL (hidden split)
+
+A decision benchmark we built (13 families; ranked on a hidden split of 1,895 items, chance-corrected composite XL-Int).
+We are its authors, so no number is independent; our models score far higher on its public split than on the hidden one,
+so only the hidden split is reported.
+
+| system | XL-Int (hidden) | rank |
+|---|---:|---:|
+| Jev (API, v1.13) | 73.0 [69.8, 75.9] | #1 |
+| **Wald-4B v1.0 · effort medium** | **63.2** [60.0, 66.4] | **#2**, best 4B |
+| Cygnet (gemma-4-12B-it + shim) | 59.8 [56.7, 62.7] | #4 |
+| Laya 421M · CLM-8B | 13.4 · 12.7 | #24 · #25 |
+
+### Verticals: a quick LoRA per task
+
+![Verticals](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/verticals.svg)
+
+One LoRA on the task's labels costs $0.12–$1.81 of GPU time (< 2 GPU-hours); every other system is zero-shot on the
+same fixed test items and byte-identical requests. Differences to Jev are paired bootstrap 95 % CIs; ▲ = the CI excludes
+0. These reads use the pre-release build v0.9 (see Versioning).
+
+| task (metric) | Wald-4B v0.9 + LoRA | Jev |
+|---|---:|---:|
+| MetaTool (tool selection, accuracy) | **97.0** | 83.0 |
+| BANKING77 (77 intents, macro-F1) | **92.8** | 78.1 |
+| AndroidControl (phone-agent action, accuracy) | **85.8**¹ | 73.4 |
+| ToxicChat (toxic-class F1) | **84.9** | 80.4 |
+| COLD, Chinese offensive language (macro-F1, full 5,323-item test) | **84.1** [83.2, 85.1] | 75.3 |
+| When2Call (call / ask / refuse, accuracy) | **83.0**² | 72.6 |
+
+¹ Trained on all training steps; the figure shows the first all-labels arm (81.6). ² Best of 8 LoRA configurations read
+on test; the dev-selected one scores 82.6. COLD: paired +8.8 [7.7, 10.0]; 83.8 on the 4,823 items not used for
+temperature fitting; ties the best published 83.7.
+
+#### With few labels, starting from Wald beats LoRA on the raw base
+
+Same LoRA recipe, labels and test items; Wald-4B (v0.9) minus raw Qwen3.5-4B-Base, points, paired 95 % CIs.
+**Bold** = the CI excludes 0.
+
+| task (metric) | 0 labels (zero-shot) | 300 labels |
+|---|---:|---:|
+| When2Call (accuracy) | **+21.2** [17.0, 25.2] | +2.0 [−0.2, 4.2] |
+| BANKING77 (macro-F1) | **+10.9** [7.6, 14.7] | **+4.4** [2.0, 7.2] |
+| SGD intent (macro-F1) | +2.8 (n.s.) | −0.1 |
+
+### Latency and cost
+
+![Latency and cost](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/latency-cost.svg)
+
+One question, one pass, serial: p50 26 ms on one RTX PRO 6000 (51 ms on an H100). Batched: 115 decisions/s (fp8), about
+$0.007 per 1,000 decisions at the GPU-hour price; the Jev API lists $0.04. Under `medium` the median stays at the
+one-pass level and the p95 is about one second. Measured on v0.9, which has the same architecture and serving path.
+
+## Train your own task LoRA (CLI, releasing soon)
+
+Every vertical above was made with one CLI, which we plan to release soon. It splits a labelled dataset with fixed seeds,
+checks the test items for leaks, trains a LoRA on Wald-4B on a GPU you choose, reads the same test items with your LoRA,
+Jev and zero-shot baselines, and serves the adapter behind the same `/v1/systemone` API.
+
+![Architecture of the vertical CLI](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/lora-cli-architecture.svg)
+
+Stages, guards, costs and a command preview: [docs/lora-cli.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/lora-cli.md).
+
+## Serving
+
+```sh
+docker build -t wald-serve .
+docker run --gpus all -v /path/to/Wald-4B:/model:ro -p 8000:8000 wald-serve   # ready when GET /health is {"ok": true}
+```
+
+Without Docker: `./run.sh /path/to/Wald-4B`. The server takes `--effort` (`none | low | medium | high | high-k<k>`), and a
+request may carry `"effort"` to override it. Declared: effort `medium`, prompt format `repeat_state_plain`, 131,072
+tokens per question prompt (longer → HTTP 422, never truncated), 1–255 options. Commands, limits and runtimes:
+[RUNBOOK.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md). Weights: bf16, `model.safetensors` sha256
+`dabeb7bc3f43bf6ea7d20ecfdac8758b3945517a991809a5772988829598e893`; every file's sha256 is in `MANIFEST.json`.
+
+## Limits
+
+- Our non-regression gate against the parent build has no failing metric. Two losses exceed the tolerance but are not
+  significant after correction: XL long_policy −6.7 [−11.6, −2.0] and JevBench public 231 −2.6 [−5.2, 0.0] (plain-prompt
+  read).
+- The temperature table is the parent build's, the one the measured Decision Index read used.
+- Our Decision Index read used a 16,384-token context; prompts longer than that were counted unsupported there and are
+  answered by the packaged server.
+
+## Versioning
+
+| version | status | checkpoint | base | date | policy | prompt format |
+|---|---|---|---|---|---|---|
+| **v1.0** | current release | internal build 021A0-f10 | Qwen/Qwen3.5-4B-Base | 2026-09-27 | effort `medium` | `repeat_state_plain` |
+| v0.9 | pre-release; the vertical LoRAs and latency above | internal build 015D0-f4 | Qwen/Qwen3.5-4B-Base | 2026-09-26 | effort `medium` | plain |
+
+v1.x = serving or LoRA refreshes on the same base generation; v2.0 = a new base generation.
+
+---
+
+Apache-2.0 (weights and code), subject to the licence of the base model Qwen/Qwen3.5-4B-Base (Apache-2.0, © the Qwen
+team, Alibaba Cloud). Third-party notices: [NOTICE](https://huggingface.co/Harry19081/Wald-4B/blob/main/NOTICE).

RUNBOOK.md

@@ -0,0 +1,110 @@
+# Runbook for the Decision Index maintainers
+
+These steps serve Wald-4B on one NVIDIA RTX PRO 6000 (96 GB) so that the kit's `http` engine can run the full 0.2.1
+suite against it.
+
+## 1. Install
+
+The weights are in the Hub repo `Harry19081/Wald-4B` (bf16 safetensors, tokenizer, config, `temperature.json`,
+`serving.json`, `MANIFEST.json`). The code is in this repository, pinned by commit.
+
+```sh
+huggingface-cli download Harry19081/Wald-4B --local-dir /models/Wald-4B
+cd /models/Wald-4B && python - <<'EOF'   # optional: verify every file against MANIFEST.json
+import hashlib, json; m = json.load(open("MANIFEST.json"))
+for f, v in m.items():
+    h = hashlib.sha256(open(f, "rb").read()).hexdigest(); assert h == v["sha256"], f
+print("ok", len(m))
+EOF
+```
+
+Choose one of the following.
+
+- **Docker:**
+  ```sh
+  docker build -t wald-serve .
+  docker run --gpus '"device=0"' -v /models/Wald-4B:/model:ro -p 8000:8000 wald-serve
+  ```
+- **No Docker** (Python 3.12 and `uv`):
+  ```sh
+  ./run.sh /models/Wald-4B
+  ```
+- **By hand:**
+  ```sh
+  uv venv -p 3.12 .venv && VIRTUAL_ENV=.venv uv pip install "vllm==0.30.0" ./server
+  VLLM_USE_FLASHINFER_SAMPLER=0 .venv/bin/wald-serve --model /models/Wald-4B --port 8000
+  ```
+  `VLLM_USE_FLASHINFER_SAMPLER=0` is needed because FlashInfer's sampler refuses sm_120 in vLLM 0.30.0.
+
+In every case, `wald-serve` starts vLLM 0.30.0 on the weights: bf16, loopback only, `--max-model-len 131072`,
+`--gpu-memory-utilization 0.90`, `--max-num-seqs 256`, `--seed 0`. It then serves `POST /v1/systemone` on port 8000. The
+service is ready when `curl -s localhost:8000/health` returns `{"ok": true, ...}` with the effective configuration. The
+first start takes about 2–4 minutes (weights load and CUDA graph capture).
+
+## 2. Run the suite
+
+```sh
+python -m decision_index run --engine http --option base_url=http://127.0.0.1:8000 --option model=wald-4b \
+    --option timeout=3600 ...
+```
+
+- **Send requests concurrently.** We ran 16–32 kit processes over group-balanced shards against one server, and vLLM
+  batches them. A single sequential client would be several times slower (section 4).
+- **Use a long timeout (≥ 3,600 s).** Under load, the p95 request latency was several minutes, from the requests with
+  many long questions (BRIGHT / ToolRet chunks, ContractNLI, ACOS).
+- **A 422 means a declared capacity limit**, and the body contains `maximum context length`. Count it as unsupported,
+  not as an error.
+- **Smoke test:**
+  ```sh
+  curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{"state":"I was charged twice.","questions":{"q":{"type":"noul","instructions":"Is this a billing problem?"}}}'
+  ```
+
+## 3. Declared configuration and limits
+
+| item | value |
+|---|---|
+| declared policy | **effort `medium`**: one pass; if the top probability is below 0.7, a thought of at most 512 tokens (temperature 0.6, top-p 0.95, top-k 20, seed = hash of the request and question id), then a second read |
+| prompt format | `repeat_state_plain` (the state written twice), from `serving.json`; the same for every benchmark |
+| readout | option-letter logits at the last position; tempered with `temperature.json` (A table for one-pass answers, B512 after a thought) |
+| > 26 options | knockout: ⌈n / 26⌉ consecutive chunks, then a final over the chunk winners; one pass; up to 676 options (the wire allows 255) |
+| context | 131,072 tokens per question prompt; longer → HTTP 422 `maximum context length`, never truncated |
+| thought that does not fit | if prompt + 512 + 64 tokens exceeds the context, the one-pass answer is returned |
+| questions per request | no fixed limit; read concurrently (8 at a time per request); tested up to 64 |
+| determinism | same request → same thought seed → same answer, up to bf16 batch-order noise (argmax agreement 99.7 % between two reads of 36k questions) |
+
+Nothing in the server depends on the benchmark: there is no per-benchmark prompt, option filtering, retry or truncation.
+
+## 4. Expected runtime of the full suite on one RTX PRO 6000
+
+The full suite has about 150,500 scoreable requests, including the display-only MMLU / ARC and RouterBench (not scored
+in 0.2.1). HLE is not counted here.
+
+**How we estimated.** Our basis is the measured wall time of our 6,948-request sample reads:
+- one pass: 875 s;
+- medium: 1,702 s. Both were measured on one card shared with a second engine, with 16 kit processes each.
+- always-think: 5,520 s, on a shared card, including a k = 4 read of the knowledge and language rows.
+
+We scaled these by a full-to-sample work ratio of 12–15×. That ratio weights each benchmark's per-request latency by its
+full size; the sample over-represents long-prompt benchmarks. A dedicated card is assumed to be 1.25–1.7× faster than
+our shared runs. `repeat_state_plain` writes the state twice, which adds about 30–60 % to prefill: this is an estimate,
+not measured on the full suite. The in-request question concurrency of this server shortens the tail of requests with
+many questions, and the table does not credit it.
+
+| policy | thinks on | plain prompt | repeat_state_plain (declared) |
+|---|---|---|---|
+| none | 0 % | ≈ 2–3 h | ≈ 2.5–4.5 h |
+| low | a few % | ≈ 2.5–4 h | ≈ 3–5.5 h |
+| **medium (declared)** | ≈ 12 % of questions | **≈ 3.5–6 h** | **≈ 4.5–9 h** |
+| high | every 2–26-option question | ≈ 7–12 h | ≈ 9–16 h |
+| high-k4 | every question, 4 thoughts | ≈ 12–20 h | ≈ 15–25 h |
+
+These are estimates, and our own full-suite read with the packaged server will replace them (CHECKLIST.md). We declare
+`medium`. `high-k4` would take most of a day on one card: prefill for every question would be about 2.3× that of `high`,
+and decoding 4×. Our measured `high-k4` gain over `high` was not significant (+0.48 [−0.4, +1.5] on the parent
+checkpoint), and that measurement applied k = 4 only to the knowledge and language benchmarks.
+
+## 5. Throughput and latency (for reference)
+
+- One question, one pass, serial: p50 26 ms on one RTX PRO 6000; 51 ms on an H100.
+- `medium`, serial: p50 ≈ 52 ms, p95 ≈ 1.1 s (the questions that think).
+- Batched one-pass throughput: about 115 decisions/s on one RTX PRO 6000 (fp8). bf16 throughput was not measured separately.

chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

config.json

@@ -0,0 +1,83 @@
+{
+  "architectures": [
+    "Qwen3_5ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attn_output_gate": true,
+  "bos_token_id": null,
+  "dtype": "bfloat16",
+  "eos_token_id": 248044,
+  "full_attention_interval": 4,
+  "head_dim": 256,
+  "hidden_act": "silu",
+  "hidden_size": 2560,
+  "initializer_range": 0.02,
+  "intermediate_size": 9216,
+  "layer_types": [
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention",
+    "linear_attention",
+    "linear_attention",
+    "linear_attention",
+    "full_attention"
+  ],
+  "linear_conv_kernel_dim": 4,
+  "linear_key_head_dim": 128,
+  "linear_num_key_heads": 16,
+  "linear_num_value_heads": 32,
+  "linear_value_head_dim": 128,
+  "mamba_ssm_dtype": "float32",
+  "max_position_embeddings": 262144,
+  "mlp_only_layers": [],
+  "model_type": "qwen3_5_text",
+  "mtp_num_hidden_layers": 1,
+  "mtp_use_dedicated_embeddings": false,
+  "num_attention_heads": 16,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 4,
+  "pad_token_id": null,
+  "partial_rotary_factor": 0.25,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "mrope_interleaved": true,
+    "mrope_section": [
+      11,
+      11,
+      10
+    ],
+    "partial_rotary_factor": 0.25,
+    "rope_theta": 10000000,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.17.0",
+  "use_cache": false,
+  "vocab_size": 248320
+}

docs/lora-cli.md

@@ -0,0 +1,96 @@
+# Fine-tune Wald-4B on your task (CLI, releasing soon)
+
+> **Preview.** The CLI is not public yet; we plan to release it soon. The commands below are preview syntax and may
+> change before the release.
+
+**One dataset in, one task model out, with an honest comparison to Jev.** The `vertical` CLI takes a labelled dataset (a
+Hugging Face repo, a URL, or your own file) and trains a LoRA on Wald-4B for that one task. It then reads the task's
+fixed test items with your LoRA, with Jev and with zero-shot baselines, using byte-identical requests, and writes a report
+with paired confidence intervals and calibration. The verticals in the [README](https://huggingface.co/Harry19081/Wald-4B#verticals-a-quick-lora-per-task)
+were all made this way. One LoRA costs $0.12–$1.81 of GPU time (under 2 GPU-hours).
+
+![Architecture of the vertical CLI](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/lora-cli-architecture.svg)
+
+## The pipeline
+
+Free stages run on your machine. Paid stages run on a GPU backend you choose: any SSH GPU box, RunPod or Modal. Every
+stage is resumable. Reads are cached per item, a finished LoRA is not trained again, and each run keeps a manifest with
+the sha256 of every output.
+
+| stage | command | what it does |
+|---|---|---|
+| task spec | `init` · `inspect` · `validate` | Scaffold `task.yaml` from a dataset: the source pinned to a commit, fields, labels, question, metric and split sizes. `inspect` renders example records as the model will see them. |
+| prep | `prep` | Download the data and hash every file. Render each item once, both as a decision record and as the exact request Jev receives. Make a seeded, label-stratified, group-aware split into train · calib · dev · test, and dedupe across splits. |
+| overlap check | `prep` | Compare held-out items with the train pool, with Decision Index items and with Wald-4B's training data. The counts go in the data manifest. |
+| audit | `audit` · `approve` | Free, deterministic rule checks before and after every stage, plus a review of a sample by a cheap model. A FAIL blocks the next paid stage. |
+| augment (optional) | `augment` | Extra training rows: soft labels for unlabelled texts, synthetic items, or paraphrases. The teacher is your own coding agent or any OpenAI-compatible API. Every row is re-checked on ingest (schema, labels, dedupe, overlap with the held-out splits). |
+| baseline | `baseline` | Read calib, dev and test with Jev (cached per request, so re-runs are free) and with zero-shot Wald-4B and the raw base. Optionally add frontier LLMs, plus random and class-prior floors. |
+| train | `train` · `watch` | One LoRA per arm. An arm is a training size, for example 300 labels, 1,000 labels, or all of them. The recipe is rank 32 on every projection, lr 1e-4, 2 epochs, and KL replay toward the base's own answers, so the base keeps its other skills. `watch` tails the log and stops a bad run. |
+| eval | `eval` | One vLLM server holds Wald-4B plus every adapter. Reads use the serving reader: letter readout, knockout above 26 options, and optional thinking effort. |
+| calibrate | `calibrate` | Fit one temperature per system on the calib split, never on test. It changes confidence, never the answer. Jev is scored as served. |
+| report | `report` · `compare` | `report.md` + `report.json`: the test metric with 2,000 paired bootstrap resamples against Jev and the base, ECE, high-confidence errors, and dev numbers for picking a configuration. `compare` gives paired differences between two runs, for example Wald-4B vs the raw base. |
+| deploy | `serve` · `try` · `latency` | Serve the adapter and its temperature on Wald-4B behind the same `/v1/systemone` decision API. The adapter stays a separate file, and Wald-4B itself is never changed. `try` sends one item through Jev and your model side by side, and `latency` measures serial latency on an otherwise idle card. |
+
+## Guards
+
+A task model is only useful if its numbers are real. The CLI enforces these rules itself:
+
+- **Same bytes everywhere.** Each item is rendered once, into one request. Jev receives that request, our reader reads
+  it, and your LoRA trains on the same bytes.
+- **Leak gate.** Before training, any test item whose word 5-gram Jaccard similarity with a train or calib item is 0.8 or
+  higher stops the run. You can drop those items from this run (`--drop-near-dups`), or a person can approve keeping them.
+- **Audit gate.** A FAIL (exit code 4) blocks every paid stage. Only a person can override it: `vertical approve` asks
+  y/N on a real terminal, or you click *Approve override* in the UI. The approval is signed, and it expires when the
+  data or the audit result changes. An agent cannot sign it.
+- **Held-out discipline.** Temperatures are fitted on calib. Configurations are chosen on dev. Test is reported once per
+  configuration, and a best-of-N read on test is labelled as best-of-N.
+- **Cost gate.** Paid stages print a cost estimate and run only with `--yes`, or when the estimate fits under `--max-usd`.
+  `--dry-run` prints the plan and changes nothing.
+- **Training watch.** A NaN or infinite loss, a zero learning rate, the wrong base, or a cost above twice the estimate
+  stops the job.
+- **Secrets.** Keys are passed at run time and never reach the GPU box's disk or the logs.
+
+## Built for coding agents
+
+Every command takes `--json` (one JSON object on stdout, with a `next` hint) and has fixed exit codes: 0 ok, 2 user or
+config error, 3 remote failure, 4 audit FAIL. Spending money and overriding an audit still need a person's yes.
+
+## Cost and time
+
+- **One LoRA:** $0.12–$1.81 of GPU time on one RTX PRO 6000 at about $1 per hour (under 2 GPU-hours). Training runs at
+  about 5,000 tokens/s.
+- **A full run** has cost $0.25–$2.15 on our public tasks. That covers the baseline reads, one to three training sizes,
+  eval and the report. Jev reads are billed by Jev and cached, so re-runs do not pay for them again.
+
+## Example (preview syntax)
+
+A task is one `task.yaml`:
+
+```yaml
+name: banking77
+title: Bank customer intent routing
+source: {hf: legacy-datasets/banking77, revision: <commit>, license: CC-BY-4.0, splits: {train: train, test: test}}
+fields: {text: text, label: label}
+labels: {from_features: true}
+state_template: "Customer message: {text}"
+question: Which intent does this bank customer's message express?
+metric: macro_f1
+sizes: {calib: 300, test: 500}
+arms: [n300, n1000, all]
+```
+
+```sh
+vertical init banking77 --hf legacy-datasets/banking77   # scaffold task.yaml
+vertical inspect banking77                               # rendered records, labels, lengths
+vertical validate banking77
+vertical prep banking77                                  # download, split, dedupe, overlap checks
+vertical audit banking77 --stage data
+vertical run banking77 --dry-run                         # cost of every paid stage; nothing runs
+vertical run banking77 --max-usd 3                       # baseline → train → eval → calibrate → report
+vertical report banking77                                # report.md + report.json
+vertical serve banking77 --arm all --yes                 # the adapter behind /v1/systemone
+vertical try banking77 --arm all --text "My card still hasn't arrived"
+```
+
+Other commands: `list`, `status`, `stop`, `backends --check`, `compare`, `latency`, `note`, `import-read` (adds a read
+made outside the CLI to a run's report).

docs/observations/one-lora-all-verticals.md

@@ -0,0 +1,112 @@
+# One LoRA for all verticals? (observation, 2026-09-27)
+
+**中文摘要：** 用一个 LoRA 代替每个任务各自一个 LoRA：在 13 个 vertical 的训练数据（外加几个 Decision Index 弱项的训练数据）上混训一个 LoRA，
+1 epoch，一张 RTX PRO 6000 约 2.5 GPU 小时。在各任务固定测试集上逐题配对：读过 Jev 的 12 个任务里 **7 个显著超过 Jev**；和每个任务自己的
+LoRA 比，When2Call / Mind2Web / RewardBench 打平、SGD intent 只差 1.4、agent 轨迹护栏（hard）反而 +3.5；BANKING77 / AndroidControl /
+MetaTool / 提示注入差 4–7 分；**ToxicChat（−11）和两个路由任务（F1 ≈ 2，"永远不交给强模型"）失效**。结论：一个共享适配器能覆盖大多数意图和
+agent 决策任务，代价 1–5 分；正例稀少的护栏任务和按概率排序的路由任务还需要单独的适配器。
+
+## 1. Question
+
+Can one LoRA serve every vertical, instead of one adapter per task? A customer with a dozen decision points (intent routing,
+tool choice, guardrails, agent next-step) would rather deploy one adapter than twelve.
+
+## 2. Setup
+
+- **Model:** one LoRA on an intermediate Wald-4B build between v0.9 and v1.0 (the v1.0 base before its final
+  task-coverage LoRA was merged; see Versioning in the README). Recipe: LoRA r32 / α64 on every projection, lr 1e-4 cosine,
+  5 % warm-up, **1 epoch**, letter readout, and 30 % replay rows trained toward the base's own answer distribution (a KL
+  anchor). The LoRA is kept separate, never merged into the released weights.
+- **Data** (commercial licences only): 53,243 rows = 37,270 labelled + 15,973 replay, about 37M prompt tokens.
+  - 13 verticals, each capped at 2,500 rows after removing near-duplicates of **every** vertical's test / calibration /
+    development items. Rows per task after that: When2Call, BANKING77, SGD intent, ToxicChat, MetaTool, AndroidControl,
+    RewardBench, Mind2Web 2,500 each; model routing 1,174 (and 1,174 for a second framing of the same prompts); prompt
+    injection 780; agent-trajectory safety 52; **agent-trajectory safety (hard) 0**.
+  - Training splits of a few Decision Index benchmarks where the base trails (a home-appliance simulator, iSarcasmEval,
+    API-Bank, ContractNLI), about 16.5k rows.
+  - Uniform mixing: no per-task sampling weight, no loss re-weighting.
+- **Cost:** about 2.5 GPU-hours of training on one RTX PRO 6000 (≈ $2.4 at the GPU-hour price), about $0.8 for the 13
+  evaluation reads.
+- **Evaluation:** each task's **fixed test items**, a temperature fitted on that task's calibration split, paired on the
+  same items (1,000 item resamples; SGD intent 2,000) against:
+  - **that task's own LoRA** (all labels). **Caveat: every own LoRA sits on Wald-4B v0.9**, a different build, so this
+    comparison is paired on items, not on base;
+  - **Jev** (same request bytes);
+  - **the same base zero-shot**, where a hosted read exists (a different reader than ours).
+
+## 3. Results
+
+| Task | Metric | one LoRA | own LoRA (v0.9) | Jev | base zero-shot | one − own | one − Jev | one − zero-shot |
+|---|---|---:|---:|---:|---:|---|---|---|
+| When2Call | accuracy | 82.4 | 82.8 | 72.6 | 76.6 | −0.4 [−2.8, +2.2] | **+9.8** [+6.2, +13.2] | +5.8 [+2.2, +9.2] |
+| BANKING77 | macro-F1 | 88.2 | 92.8 | 78.1 | — | −4.6 [−7.2, −2.6] | **+10.1** [+7.3, +14.1] | — |
+| SGD intent | macro-F1 | 96.6 | 98.0 | 92.8 | — | −1.4 [−2.7, −0.2] | **+3.7** [+1.5, +6.6] | — |
+| AndroidControl | accuracy | 81.8 | 85.8 | 73.4 | — | −4.0 [−6.8, −1.0] | **+8.4** [+4.4, +12.4] | — |
+| MetaTool | accuracy | 90.2 | 97.0 | 83.0 | 81.4 | −6.8 [−9.2, −4.4] | **+7.2** [+4.4, +10.4] | +8.8 [+6.2, +11.6] |
+| Agent-trajectory safety (hard) | unsafe F1 | 75.7 | 72.2 | 47.0 | 29.3 | **+3.5** [+1.8, +5.2] | **+28.7** [+21.6, +36.3] | +46.5 [+39.5, +53.7] |
+| Agent-trajectory safety | unsafe F1 | 98.0 | — | 92.2 | 74.8 | — | **+5.8** [+3.2, +8.7] | +23.2 [+18.7, +28.1] |
+| Mind2Web | accuracy | 51.2 | 51.9 | 48.5 | — | −0.6 [−5.0, +3.3] | +2.7 [−1.5, +6.9] | — |
+| RewardBench | accuracy | 82.8 | 83.4 | 90.6 | 83.6 | −0.6 [−2.5, +1.1] | −7.8 [−9.9, −5.7] | −0.8 [−2.4, +0.8] |
+| Prompt injection | injection F1 | 75.0 | 79.4 | 82.8 | 72.7 | −4.4 [−7.9, −1.4] | −7.8 [−14.1, −0.9] | +2.3 [−1.8, +6.4] |
+| ToxicChat | toxic F1 | 73.8 | 84.9 | 80.4 | 71.8 | **−11.1** [−14.5, −7.9] | −6.6 [−10.0, −3.4] | +2.0 [−0.6, +4.6] |
+| Model routing | needs-strong F1 | 1.9 | 26.8 | 0.0 | 10.4 | **−24.9** [−34.0, −14.7] | +1.9 [+0.0, +6.2] | −8.5 [−16.9, −0.6] |
+| Model routing (difficulty framing) | hard F1 | 2.0 | — | — | 16.2 | — | — | −14.2 [−22.7, −6.0] |
+
+## 4. Three groups
+
+**One adapter is enough.**
+- When2Call (−0.4), Mind2Web (−0.6) and RewardBench (−0.6) are level with their own LoRA; SGD intent is 1.4 behind.
+- Agent-trajectory safety (hard) is **+3.5 ahead** of its own LoRA with **zero** rows of its own task in the mix: the gain is
+  transfer from the other agent and guard tasks. Its own LoRA had only 250 training rows and was often confidently wrong,
+  so the bar was low.
+
+**A small cost (−4 to −7).**
+- BANKING77 −4.6, AndroidControl −4.0, MetaTool −6.8, prompt injection −4.4. All but prompt injection still beat Jev.
+- The first three had 2,500 rows in the mix vs 8,000–30,000 for their own LoRA, so part of the gap is data volume (the own
+  LoRAs gain about one point per doubling of labels), not interference.
+
+**It breaks.**
+- **ToxicChat −11.1** vs its own LoRA, back to the zero-shot level. 2,500 rows at about 7 % toxic means about 175 positives
+  among 37k labelled rows: the rare-positive signal is drowned.
+- **Model routing collapses to "never route"** (F1 1.9 vs 26.8; 2.0 for the second framing). Routing is a probability-
+  ranking task: its value is the ordering of P(needs a strong model) across prompts, and the mixed model puts nearly all mass
+  on the majority option. It also had only 1,174 rows after near-duplicate removal (own LoRA: 8,000).
+
+## 5. Why (hypotheses, not tested)
+
+1. **Class imbalance under uniform mixing.** Rare-positive tasks (ToxicChat 7 %, routing 20 %) contribute few positive
+   gradients in a 37k-row mix, so the model learns their majority option.
+2. **Ranking / threshold tasks lose their calibrated margin.** The replay anchor and other tasks' confident targets sharpen
+   the answer head; routing needs small, well-ordered probability differences that argmax F1 at the default threshold misses.
+3. **Label-space interference.** The letter readout shares (A) / (B) across binary tasks with different meanings and base
+   rates, which can pull against each other.
+4. **Data volume and isolation losses.** The per-task cap (2,500) and the cross-task near-duplicate removal (routing
+   8,000 → 1,174; hard agent-safety 250 → 0) mean the mix is not the data the dedicated LoRAs saw.
+5. **Base difference.** The one LoRA and the dedicated LoRAs sit on different builds. On one browser task a base swap alone
+   changed nothing (−0.4 [−2.9, +2.1]), so this likely explains little, but it is not ruled out per task.
+
+## 6. What to try next
+
+| Idea | What | Rough cost |
+|---|---|---|
+| Per-task sampling weights | sample tasks ∝ n^α (α ≈ 0.3–0.5), or up-weight rare-positive and ranking tasks | ≈ $2.5 + $0.8 reads |
+| Task-balanced or focal loss | class-balanced weights within each binary task, or focal loss on rare-positive tasks | ≈ $2.5 + $0.8 |
+| Hybrid adapters | the shared adapter plus small separate adapters for ToxicChat, prompt injection and routing | ≈ $0.2–1.5 per extra adapter; multi-LoRA serving |
+| Per-task calibration | per-task temperature and decision threshold on calibration data (routing: rank-based, not argmax) | ≈ $0 (offline) |
+| Same-base comparison | dedicated and shared LoRAs on the same build | ≈ $2.5 |
+| Lighter isolation | remove near-duplicates only against each task's own evaluation items | $0 data + one re-train |
+
+## 7. Product takeaway
+
+One adapter covers most intent and agent-decision tasks at a 1–5 point cost against dedicated adapters, and still beats Jev
+on 7 of 12 tasks. Imbalanced guard tasks (ToxicChat) and probability-ranking tasks (routing) still need their own adapter,
+or a hybrid of one shared adapter plus small per-task ones.
+
+## Caveats
+
+- For 7 tasks the Jev numbers are Jev's reads from the dedicated-LoRA evaluations on the same item ids, not a fresh read.
+- Mind2Web was read with plain knockout, because the adapter was trained on knockout-shaped questions; the knockout +
+  top-10 re-read gives 49.0.
+- The dedicated LoRAs are on Wald-4B v0.9: paired on items, not on base.
+- The base zero-shot column comes from a hosted read, not our evaluation reader.
+- One seed, one epoch, one mix; no sampling-weight search.

docs/readmes/README.zh.md

@@ -0,0 +1,173 @@
+<div align="center">
+  <h1>Wald-4B v1.0</h1>
+  <p><strong>经过校准的决策模型：一次前向，把状态和问题变成每个选项的概率。</strong></p>
+</div>
+
+**W**ait **A** bit, **L**ook, then **D**ecide（等一下，看一眼，再决定）：有把握，立刻决定；没把握，再看一眼。名字也取自 Abraham Wald（1902–1950），序贯分析的创立者：证据够了就停下来做决定。
+
+---
+
+<p align="center"><a href="https://huggingface.co/Harry19081/Wald-4B">English</a> · <a href="https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/readmes/README.zh.md">简体中文</a></p>
+
+---
+
+**Wald-4B 把一次决策变成一次前向。** 给它一段状态和一个问题，它为每个选项返回一个校准过的概率：调哪个工具、要不要问用户、
+属于哪个意图、输出是否安全。概率高于 0.9 直接执行，低于 0.6 交给人，中间的交给更大的模型。在 `medium` 档位下，只有最高概率
+低于 0.7 时它才先想一想（不超过 512 token），大约九题一次；输出始终是概率分布，不解析文本。
+
+## 一览
+
+- **Decision Index 0.2.1：53.91**（样本估计），67 条榜单中排第 6，4B 中第一（见下文“基准测试”）。
+- **XL 隐藏集：63.2**，第 2，仅次于 Jev，4B 中第一。
+- **你的任务，不到 $2**：每个任务一个 LoRA，12 个公开任务中 9 个超过 Jev。
+- **每次决策约 50 ms**（p50），每千次决策的价格约为 Jev 的 1/5。
+
+![Decision Index 与模型大小](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/di-vs-size.svg)
+
+## 工作原理
+
+基于 Qwen/Qwen3.5-4B-Base @ `1001bb4d826a52d1f399e183466143f4da7b741b` 的 4B 模型：先做全参数决策训练，再合并三个 LoRA
+（带 KL 回放的精调、短思考蒸馏、任务覆盖）。每个问题变成一条纯文本提示，不用对话模板，不加新 token。状态写两遍（格式
+`repeat_state_plain`；第二遍由一句固定的话引出，见 NOTICE），然后是：
+
+```
+Question: {instructions}
+(A) {option 1}
+(B) {option 2}
+Answer: (
+```
+
+最后一个位置上选项字母的 logits 给出分布。超过 26 个选项时，按 ⌈n / 26⌉ 块分别读，再把各块的胜者读一次（淘汰赛）。
+每个（题型、选项数）一个温度，在我们自己的留出数据上拟合，只改变置信度。提示格式、策略和上下文上限都固定在 `serving.json`
+里，所有基准一律相同。
+
+**训练数据**（不公开）：四个阶段，用到 84 个公开来源（83 个数据集，只读训练切分，NLI4CT 另读了 dev 切分；外加 Decision Index 工具包里 MIT 许可的
+Home-appliance 生成器）、由代码生成并由代码标注的决策题，以及对早先数据的 KL 回放。其中一部分是合成数据：题目和短思考
+由大得多的前沿大模型编写或标注——凡公开了参数量的都远超 120B。全部来源及 Decision Index 题目的筛查方法见
+[CONTAMINATION.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md)。
+
+## 基准测试
+
+我们测了四项：Decision Index 0.2.1（榜单）、XL（带隐藏集的决策基准）、公开垂直任务上的 LoRA，以及服务延迟。除非标注
+“榜单”，所有数字都是我们自己的读数，默认零样本，所有系统收到相同的请求。
+
+### Decision Index 0.2.1
+
+我们读的是 **6,948 个请求的分层样本**（37,469 个问题；用工具包自带的采样器和种子；未重建 HLE），按 0.2.1 规则计分。
+Jev 在这个样本上是 57.19，榜单全量是 57.89，所以样本大约偏低 0.7 分。榜单行是官方全量数字
+（[榜单](https://huggingface.co/spaces/multimodalart/jev-decision-index)）。
+
+| 系统 | 规模 | Decision Index 0.2.1 | 来源 |
+|---|---|---|---|
+| Jev（托管 API，jev-1.13.0） | 未公开 | 57.19 [55.29, 58.60] · 榜单 57.89 | 我们的样本 · 榜单 |
+| simple-jev · Qwen3.8-27B（榜单第 4）· Jebadiah 27B（第 5） | 27B | 55.74 · 54.67 | 榜单 |
+| **Wald-4B v1.0 · medium** | 4B | **53.91** [51.87, 55.27] | 我们的样本 |
+| reflex 27B（榜单第 6）· Decider chat · Qwen3.6-27B（第 7） | 27B | 52.16 · 51.35 | 榜单 |
+| Qwen3.8-27B，未训练，用我们的字母读出（一遍） | 27B | 47.78 [45.95, 49.17] | 我们的样本 |
+| Decider 35B-A3B（榜单第 11） | 35B-A3B | 47.11 | 榜单 |
+| Decider 4B（榜单第 15） | 4B | 40.70 | 榜单 |
+| Kev 9B（榜单第 23）· Kev 4B（第 28） | 9B · 4B | 38.48 · 34.64 | 榜单 |
+
+53.91 在 67 条榜单中排第 6（在 Jebadiah 27B 与 reflex 27B 之间）；在同一样本上比 Jev 低 3.3 [−4.9, −1.8]。
+同样是 4B 底座，比 Kev 4B 高 19.3。
+
+![Decision Index 分大类](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/di-areas.svg)
+
+**披露。** 有 363 个 Decision Index 题目 id 通过公开训练切分或共同的上游来源进入了我们的训练数据（从未用过测试切分）；
+��它们全部记错，得分为 **53.62**。Home appliance 得 1.00（skill）：最后一个阶段用该基准自己的 MIT 生成器造了新的家庭数据来训练
+（新种子，与测试数据没有共享状态）；把 Home appliance 换回父版本的答案，指数为 51.57 [49.56, 52.95]。详见
+[CONTAMINATION.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/CONTAMINATION.md)。
+
+**同请求套件。** JevBench public 231（准确率 %）：Wald-4B v1.0 **88.3**（打包后的服务），Jev 86.6，未训练的
+Qwen3.5-4B-Base 67.5，Laya 421M 58.0，CLM-8B 39.0。CLM-8B 和 Laya 没有 Decision Index 读数。
+
+### XL（隐藏集）
+
+我们自己构建的决策基准（13 个题族；按 1,895 题的隐藏集排名，指标为机会校正后的综合分 XL-Int）。我们是它的作者，
+所以没有一个数字是独立的；我们的模型在公开集上的得分远高于隐藏集，因此只报告隐藏集。
+
+| 系统 | XL-Int（隐藏集） | 名次 |
+|---|---:|---:|
+| Jev（API，v1.13） | 73.0 [69.8, 75.9] | 第 1 |
+| **Wald-4B v1.0 · medium** | **63.2** [60.0, 66.4] | **第 2**，4B 中第一 |
+| Cygnet（gemma-4-12B-it + shim） | 59.8 [56.7, 62.7] | 第 4 |
+| Laya 421M · CLM-8B | 13.4 · 12.7 | 第 24 · 第 25 |
+
+### 垂直任务：每个任务一个快速 LoRA
+
+![垂直任务](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/verticals.svg)
+
+用任务标签训练一个 LoRA，花费 $0.12–$1.81 的 GPU 时间（不到 2 GPU 小时）；其他系统都是零样本，用同一批固定测试题和逐字节
+相同的请求。与 Jev 的差值为配对 bootstrap 95 % 置信区间；▲ 表示区间不含 0。这些读数来自预发布版本 v0.9（见“版本”）。
+
+| 任务（指标） | Wald-4B v0.9 + LoRA | Jev |
+|---|---:|---:|
+| MetaTool（工具选择，准确率） | **97.0** | 83.0 |
+| BANKING77（77 个意图，宏 F1） | **92.8** | 78.1 |
+| AndroidControl（手机智能体动作，准确率） | **85.8**¹ | 73.4 |
+| ToxicChat（有害类 F1） | **84.9** | 80.4 |
+| COLD，中文冒犯语言（宏 F1，完整 5,323 题测试集） | **84.1** [83.2, 85.1] | 75.3 |
+| When2Call（调用 / 追问 / 拒绝，准确率） | **83.0**² | 72.6 |
+
+¹ 用全部训练步训练的臂；图中是第一个全标签臂（81.6）。² 8 个 LoRA 配置中在测试集上最好的一个；按 dev 选出的配置为 82.6。
+COLD：配对差 +8.8 [7.7, 10.0]；在未用于温度拟合的 4,823 题上为 83.8；与已发表的最好结果 83.7 持平。
+
+#### 标签很少时，从 Wald 起步胜过在原始底座上训 LoRA
+
+相同的 LoRA 配方、标签和测试题；表中为 Wald-4B（v0.9）减去原始 Qwen3.5-4B-Base 的分差，配对 95 % 置信区间。
+**加粗**表示区间不含 0。
+
+| 任务（指标） | 0 个标签（零样本） | 300 个标签 |
+|---|---:|---:|
+| When2Call（准确率） | **+21.2** [17.0, 25.2] | +2.0 [−0.2, 4.2] |
+| BANKING77（宏 F1） | **+10.9** [7.6, 14.7] | **+4.4** [2.0, 7.2] |
+| SGD intent（宏 F1） | +2.8（不显著） | −0.1 |
+
+### 延迟与成本
+
+![延迟与成本](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/latency-cost.svg)
+
+单个问题、一遍、串行：一张 RTX PRO 6000 上 p50 为 26 ms（H100 上 51 ms）。批处理：115 次决策/秒（fp8），按 GPU 小时价格
+约 $0.007 / 千次决策；Jev API 标价 $0.04。`medium` 下中位延迟与一遍相同，p95 约 1 秒。测于 v0.9，架构与服务路径相同。
+
+## 训练你自己的任务 LoRA（CLI，即将发布）
+
+上面每个垂直任务都是用同一个 CLI 做的，我们计划很快发布。它用固定种子切分带标签的数据集，检查测试题泄漏，在你选的 GPU 上
+基于 Wald-4B 训练 LoRA，用你的 LoRA、Jev 和零样本基线读同一批测试题，并把适配器放在同样的 `/v1/systemone` API 后面提供服务。
+
+![垂直任务 CLI 架构](https://huggingface.co/Harry19081/Wald-4B/resolve/main/figures/lora-cli-architecture.svg)
+
+阶段、防护、成本和命令预览：[docs/lora-cli.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/docs/lora-cli.md)（英文）。
+
+## 服务
+
+```sh
+docker build -t wald-serve .
+docker run --gpus all -v /path/to/Wald-4B:/model:ro -p 8000:8000 wald-serve   # GET /health 返回 {"ok": true} 即就绪
+```
+
+不用 Docker：`./run.sh /path/to/Wald-4B`。服务端接受 `--effort`（`none | low | medium | high | high-k<k>`），请求里也可以带
+`"effort"` 覆盖。声明配置：effort `medium`，提示格式 `repeat_state_plain`，每个问题提示最多 131,072 token（更长返回 HTTP 422，
+从不截断），1–255 个选项。命令、上限和运行时间见 [RUNBOOK.md](https://huggingface.co/Harry19081/Wald-4B/blob/main/RUNBOOK.md)（英文）。权重为 bf16，
+`model.safetensors` 的 sha256 为 `dabeb7bc3f43bf6ea7d20ecfdac8758b3945517a991809a5772988829598e893`；每个文件的 sha256 见 `MANIFEST.json`。
+
+## 局限
+
+- 对父版本的不退步门槛没有失败项。���项下降超出容差，但多重校正后不显著：XL long_policy −6.7 [−11.6, −2.0]，
+  JevBench public 231 −2.6 [−5.2, 0.0]（plain 提示的读数）。
+- 温度表沿用父版本的，也就是 Decision Index 实测读数所用的那张。
+- 我们的 Decision Index 读数用的是 16,384 token 上下文；超出的提示当时记为不支持，打包后的服务会作答。
+
+## 版本
+
+| 版本 | 状态 | 检查点 | 底座 | 日期 | 策略 | 提示格式 |
+|---|---|---|---|---|---|---|
+| **v1.0** | 当前版本 | 内部版本 021A0-f10 | Qwen/Qwen3.5-4B-Base | 2026-09-27 | effort `medium` | `repeat_state_plain` |
+| v0.9 | 预发布；上文的垂直任务 LoRA 和延迟 | 内部版本 015D0-f4 | Qwen/Qwen3.5-4B-Base | 2026-09-26 | effort `medium` | plain |
+
+v1.x = 同一底座代上的服务或 LoRA 更新；v2.0 = 新的底座代。
+
+---
+
+Apache-2.0（权重和代码），同时受底座模型 Qwen/Qwen3.5-4B-Base 许可约束（Apache-2.0，© 阿里云通义千问团队）。第三方声明：
+[NOTICE](https://huggingface.co/Harry19081/Wald-4B/blob/main/NOTICE)。

figures/data.json

@@ -0,0 +1,1584 @@
+{
+ "board": {
+  "edition": "release-v2.1",
+  "generated_utc": "2026-09-27T03:42:13+00:00",
+  "jev": {
+   "index": 57.91,
+   "areas": {
+    "knowledge": 0.514,
+    "language": 0.6202,
+    "retrieval": 0.5542,
+    "tools": 0.7509,
+    "arts": 0.3766
+   }
+  },
+  "entries": [
+   {
+    "rank": 1,
+    "name": "Surogate Rune 26B-A4B v3",
+    "size_b": 26.0,
+    "base_model": "google/gemma-4-26B-A4B-it",
+    "kind": "full fine-tune",
+    "index": 57.44,
+    "areas": {
+     "knowledge": 0.4336,
+     "language": 0.6306,
+     "retrieval": 0.6352,
+     "tools": 0.7122,
+     "arts": 0.419
+    }
+   },
+   {
+    "rank": 2,
+    "name": "Decider chat \u00b7 Gemma-4-31B",
+    "size_b": 31.0,
+    "base_model": "google/gemma-4-31B",
+    "kind": "inference technique",
+    "index": 57.33,
+    "areas": {
+     "knowledge": 0.4427,
+     "language": 0.6038,
+     "retrieval": 0.6308,
+     "tools": 0.7557,
+     "arts": 0.3834
+    }
+   },
+   {
+    "rank": 3,
+    "name": "AutoJev-27B",
+    "size_b": 27.0,
+    "base_model": "Qwen/Qwen3.8-27B",
+    "kind": "full fine-tune",
+    "index": 56.4,
+    "areas": {
+     "knowledge": 0.4088,
+     "language": 0.6346,
+     "retrieval": 0.5486,
+     "tools": 0.7935,
+     "arts": 0.3939
+    }
+   },
+   {
+    "rank": 4,
+    "name": "simple-jev \u00b7 Qwen3.8-27B (featherless)",
+    "size_b": 27.0,
+    "base_model": "Qwen/Qwen3.8-27B",
+    "kind": "inference technique",
+    "index": 55.74,
+    "areas": {
+     "knowledge": 0.3656,
+     "language": 0.6209,
+     "retrieval": 0.6325,
+     "tools": 0.7621,
+     "arts": 0.3648
+    }
+   },
+   {
+    "rank": 5,
+    "name": "Jebadiah 27B",
+    "size_b": 27.0,
+    "base_model": "Qwen/Qwen3.8-27B",
+    "kind": "LoRA",
+    "index": 54.67,
+    "areas": {
+     "knowledge": 0.3884,
+     "language": 0.6068,
+     "retrieval": 0.5389,
+     "tools": 0.7813,
+     "arts": 0.3873
+    }
+   },
+   {
+    "rank": 6,
+    "name": "reflex 27B",
+    "size_b": 27.0,
+    "base_model": "Qwen/Qwen3.8-27B",
+    "kind": "inference technique",
+    "index": 52.16,
+    "areas": {
+     "knowledge": 0.3511,
+     "language": 0.5416,
+     "retrieval": 0.5781,
+     "tools": 0.7412,
+     "arts": 0.3965
+    }
+   },
+   {
+    "rank": 7,
+    "name": "Decider chat \u00b7 Qwen3.6-27B",
+    "size_b": 27.0,
+    "base_model": "Qwen/Qwen3.6-27B",
+    "kind": "inference technique",
+    "index": 51.35,
+    "areas": {
+     "knowledge": 0.3699,
+     "language": 0.5712,
+     "retrieval": 0.5223,
+     "tools": 0.7142,
+     "arts": 0.3506
+    }
+   },
+   {
+    "rank": 8,
+    "name": "Winnow-12B",
+    "size_b": 12.0,
+    "base_model": "google/gemma-4-12B",
+    "kind": "LoRA",
+    "index": 50.02,
+    "areas": {
+     "knowledge": 0.3382,
+     "language": 0.5599,
+     "retrieval": 0.5405,
+     "tools": 0.7102,
+     "arts": 0.2997
+    }
+   },
+   {
+    "rank": 9,
+    "name": "JoshuaSP diffusiongemma (open-jev)",
+    "size_b": 26.0,
+    "base_model": "google/diffusiongemma-26B-A4B-it",
+    "kind": "inference technique",
+    "index": 49.47,
+    "areas": {
+     "knowledge": 0.3272,
+     "language": 0.5351,
+     "retrieval": 0.5837,
+     "tools": 0.7016,
+     "arts": 0.2665
+    }
+   },
+   {
+    "rank": 10,
+    "name": "Jevfire",
+    "size_b": 27.0,
+    "base_model": "Qwen/Qwen3.8-27B",
+    "kind": "inference technique",
+    "index": 49.37,
+    "areas": {
+     "knowledge": 0.3048,
+     "language": 0.5334,
+     "retrieval": 0.5622,
+     "tools": 0.7233,
+     "arts": 0.3221
+    }
+   },
+   {
+    "rank": 11,
+    "name": "Decider 35B-A3B",
+    "size_b": 35.0,
+    "base_model": "Qwen/Qwen3.5-35B-A3B-Base",
+    "kind": "full fine-tune",
+    "index": 47.11,
+    "areas": {
+     "knowledge": 0.3178,
+     "language": 0.5552,
+     "retrieval": 0.5466,
+     "tools": 0.5654,
+     "arts": 0.326
+    }
+   },
+   {
+    "rank": 12,
+    "name": "JPT-9B",
+    "size_b": 9.0,
+    "base_model": "Qwen/Qwen3.5-9B-Base",
+    "kind": "LoRA",
+    "index": 46.89,
+    "areas": {
+     "knowledge": 0.317,
+     "language": 0.5672,
+     "retrieval": 0.4459,
+     "tools": 0.6702,
+     "arts": 0.2856
+    }
+   },
+   {
+    "rank": 13,
+    "name": "Decision 1.0 Lux",
+    "size_b": 9.0,
+    "base_model": "Qwen/Qwen3.5-9B-Base",
+    "kind": "head / adapter",
+    "index": 43.49,
+    "areas": {
+     "knowledge": 0.309,
+     "language": 0.4797,
+     "retrieval": 0.5002,
+     "tools": 0.5719,
+     "arts": 0.2635
+    }
+   },
+   {
+    "rank": 14,
+    "name": "Xor",
+    "size_b": 35.0,
+    "base_model": "Qwen/Qwen3.6-35B-A3B",
+    "kind": "full fine-tune",
+    "index": 41.48,
+    "areas": {
+     "knowledge": 0.3433,
+     "language": 0.5558,
+     "retrieval": 0.2296,
+     "tools": 0.5514,
+     "arts": 0.3563
+    }
+   },
+   {
+    "rank": 15,
+    "name": "Decider 4B",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "full fine-tune",
+    "index": 40.7,
+    "areas": {
+     "knowledge": 0.2574,
+     "language": 0.4597,
+     "retrieval": 0.4467,
+     "tools": 0.5862,
+     "arts": 0.2503
+    }
+   },
+   {
+    "rank": 16,
+    "name": "Jev-Omni",
+    "size_b": 12.0,
+    "base_model": "google/gemma-4-12B",
+    "kind": "LoRA + head",
+    "index": 40.53,
+    "areas": {
+     "knowledge": 0.2549,
+     "language": 0.5052,
+     "retrieval": 0.3583,
+     "tools": 0.6077,
+     "arts": 0.2607
+    }
+   },
+   {
+    "rank": 17,
+    "name": "djev",
+    "size_b": 26.0,
+    "base_model": "google/diffusiongemma-26B-A4B-it",
+    "kind": "inference technique",
+    "index": 40.28,
+    "areas": {
+     "knowledge": 0.264,
+     "language": 0.4876,
+     "retrieval": 0.4177,
+     "tools": 0.5168,
+     "arts": 0.3045
+    }
+   },
+   {
+    "rank": 18,
+    "name": "Winnow-E4B",
+    "size_b": 4.0,
+    "base_model": "google/gemma-4-E4B",
+    "kind": "LoRA",
+    "index": 39.89,
+    "areas": {
+     "knowledge": 0.2229,
+     "language": 0.4511,
+     "retrieval": 0.438,
+     "tools": 0.6251,
+     "arts": 0.2277
+    }
+   },
+   {
+    "rank": 19,
+    "name": "Hopper",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "LoRA",
+    "index": 39.67,
+    "areas": {
+     "knowledge": 0.2327,
+     "language": 0.444,
+     "retrieval": 0.4446,
+     "tools": 0.5905,
+     "arts": 0.248
+    }
+   },
+   {
+    "rank": 20,
+    "name": "Bespoke Nimble 9B v2",
+    "size_b": 9.0,
+    "base_model": "Qwen/Qwen3.5-9B-Base",
+    "kind": "LoRA",
+    "index": 39.57,
+    "areas": {
+     "knowledge": 0.2952,
+     "language": 0.4509,
+     "retrieval": 0.3493,
+     "tools": 0.5921,
+     "arts": 0.2471
+    }
+   },
+   {
+    "rank": 21,
+    "name": "JevK5",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "LoRA",
+    "index": 38.81,
+    "areas": {
+     "knowledge": 0.235,
+     "language": 0.4329,
+     "retrieval": 0.4615,
+     "tools": 0.5212,
+     "arts": 0.2781
+    }
+   },
+   {
+    "rank": 22,
+    "name": "lev",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "LoRA + head",
+    "index": 38.54,
+    "areas": {
+     "knowledge": 0.2197,
+     "language": 0.3852,
+     "retrieval": 0.4792,
+     "tools": 0.5752,
+     "arts": 0.2796
+    }
+   },
+   {
+    "rank": 23,
+    "name": "Kev 9B",
+    "size_b": 9.0,
+    "base_model": "Qwen/Qwen3.5-9B-Base",
+    "kind": "LoRA + head",
+    "index": 38.48,
+    "areas": {
+     "knowledge": 0.2616,
+     "language": 0.4167,
+     "retrieval": 0.4367,
+     "tools": 0.5448,
+     "arts": 0.2242
+    }
+   },
+   {
+    "rank": 24,
+    "name": "Intern-Decision-4B",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "full fine-tune",
+    "index": 37.81,
+    "areas": {
+     "knowledge": 0.2371,
+     "language": 0.4188,
+     "retrieval": 0.4034,
+     "tools": 0.5566,
+     "arts": 0.2608
+    }
+   },
+   {
+    "rank": 25,
+    "name": "razorback16 openjev diffusiongemma (NVFP4, vLLM)",
+    "size_b": 26.0,
+    "base_model": "google/diffusiongemma-26B-A4B-it",
+    "kind": "inference technique",
+    "index": 37.25,
+    "areas": {
+     "knowledge": 0.2618,
+     "language": 0.4486,
+     "retrieval": 0.3702,
+     "tools": 0.4719,
+     "arts": 0.285
+    }
+   },
+   {
+    "rank": 26,
+    "name": "NeoHorse-Jev-4B",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "head / adapter",
+    "index": 36.75,
+    "areas": {
+     "knowledge": 0.2259,
+     "language": 0.3936,
+     "retrieval": 0.4391,
+     "tools": 0.5888,
+     "arts": 0.1177
+    }
+   },
+   {
+    "rank": 27,
+    "name": "Solomon v1.1",
+    "size_b": 27.0,
+    "base_model": "Qwen/Qwen3.8-27B",
+    "kind": "head / adapter",
+    "index": 36.43,
+    "areas": {
+     "knowledge": 0.2213,
+     "language": 0.5728,
+     "retrieval": 0.2597,
+     "tools": 0.4697,
+     "arts": 0.2116
+    }
+   },
+   {
+    "rank": 28,
+    "name": "Kev 4B",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "LoRA + head",
+    "index": 34.64,
+    "areas": {
+     "knowledge": 0.2286,
+     "language": 0.353,
+     "retrieval": 0.4098,
+     "tools": 0.5256,
+     "arts": 0.1792
+    }
+   },
+   {
+    "rank": 29,
+    "name": "Decision 1.0 Nox",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "head / adapter",
+    "index": 34.36,
+    "areas": {
+     "knowledge": 0.2141,
+     "language": 0.3659,
+     "retrieval": 0.4431,
+     "tools": 0.4957,
+     "arts": 0.1439
+    }
+   },
+   {
+    "rank": 30,
+    "name": "Jobe",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "inference technique",
+    "index": 32.35,
+    "areas": {
+     "knowledge": 0.1773,
+     "language": 0.3454,
+     "retrieval": 0.3669,
+     "tools": 0.4881,
+     "arts": 0.2574
+    }
+   },
+   {
+    "rank": 31,
+    "name": "mmastrac diffusiongemma (vLLM PR 57250)",
+    "size_b": 26.0,
+    "base_model": "google/diffusiongemma-26B-A4B-it",
+    "kind": "inference technique",
+    "index": 32.24,
+    "areas": {
+     "knowledge": 0.249,
+     "language": 0.4593,
+     "retrieval": 0.2417,
+     "tools": 0.3399,
+     "arts": 0.2881
+    }
+   },
+   {
+    "rank": 32,
+    "name": "open-jev (pngwn)",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "LoRA",
+    "index": 29.91,
+    "areas": {
+     "knowledge": 0.1988,
+     "language": 0.3546,
+     "retrieval": 0.2912,
+     "tools": 0.4573,
+     "arts": 0.1414
+    }
+   },
+   {
+    "rank": 33,
+    "name": "Tev1-4B-experimental",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "full fine-tune",
+    "index": 29.24,
+    "areas": {
+     "knowledge": 0.2349,
+     "language": 0.3919,
+     "retrieval": 0.1548,
+     "tools": 0.4183,
+     "arts": 0.2292
+    }
+   },
+   {
+    "rank": 34,
+    "name": "Decider 2B",
+    "size_b": 2.0,
+    "base_model": "Qwen/Qwen3.5-2B-Base",
+    "kind": "full fine-tune",
+    "index": 28.97,
+    "areas": {
+     "knowledge": 0.1493,
+     "language": 0.3257,
+     "retrieval": 0.3729,
+     "tools": 0.4245,
+     "arts": 0.1464
+    }
+   },
+   {
+    "rank": 35,
+    "name": "openvons",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3-4B-Instruct-2507",
+    "kind": "inference technique",
+    "index": 28.42,
+    "areas": {
+     "knowledge": 0.2077,
+     "language": 0.301,
+     "retrieval": 0.2537,
+     "tools": 0.4465,
+     "arts": 0.2033
+    }
+   },
+   {
+    "rank": 36,
+    "name": "this-that 1.2",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 28.14,
+    "areas": {
+     "knowledge": 0.1514,
+     "language": 0.3191,
+     "retrieval": 0.3283,
+     "tools": 0.4497,
+     "arts": 0.1183
+    }
+   },
+   {
+    "rank": 37,
+    "name": "Metask-Jev-4B",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "LoRA",
+    "index": 26.89,
+    "areas": {
+     "knowledge": 0.2096,
+     "language": 0.3939,
+     "retrieval": 0.1291,
+     "tools": 0.3759,
+     "arts": 0.1828
+    }
+   },
+   {
+    "rank": 38,
+    "name": "SemIf",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3.5-4B-Base",
+    "kind": "inference technique",
+    "index": 25.94,
+    "areas": {
+     "knowledge": 0.171,
+     "language": 0.3379,
+     "retrieval": 0.1572,
+     "tools": 0.391,
+     "arts": 0.2492
+    }
+   },
+   {
+    "rank": 39,
+    "name": "Decision 1.0 Sol",
+    "size_b": 2.0,
+    "base_model": "Qwen/Qwen3.5-2B-Base",
+    "kind": "head / adapter",
+    "index": 25.32,
+    "areas": {
+     "knowledge": 0.1149,
+     "language": 0.2535,
+     "retrieval": 0.3271,
+     "tools": 0.4611,
+     "arts": 0.0818
+    }
+   },
+   {
+    "rank": 40,
+    "name": "mini-jev",
+    "size_b": 4.0,
+    "base_model": "Qwen/Qwen3-4B-Instruct-2507",
+    "kind": "inference technique",
+    "index": 20.98,
+    "areas": {
+     "knowledge": 0.1763,
+     "language": 0.2857,
+     "retrieval": 0.0866,
+     "tools": 0.2842,
+     "arts": 0.2107
+    }
+   },
+   {
+    "rank": 41,
+    "name": "Bosun v3.1 1.7B",
+    "size_b": 1.7,
+    "base_model": "Qwen/Qwen3-1.7B-Base",
+    "kind": "LoRA + head",
+    "index": 20.1,
+    "areas": {
+     "knowledge": 0.0472,
+     "language": 0.124,
+     "retrieval": 0.4282,
+     "tools": 0.3478,
+     "arts": 0.0747
+    }
+   },
+   {
+    "rank": 42,
+    "name": "Intern-Decision-2B",
+    "size_b": 2.0,
+    "base_model": "Qwen/Qwen3.5-2B-Base",
+    "kind": "full fine-tune",
+    "index": 19.38,
+    "areas": {
+     "knowledge": 0.138,
+     "language": 0.2056,
+     "retrieval": 0.1566,
+     "tools": 0.3334,
+     "arts": 0.1272
+    }
+   },
+   {
+    "rank": 43,
+    "name": "JPT-0.8B",
+    "size_b": 0.8,
+    "base_model": "Qwen/Qwen3.5-0.8B-Base",
+    "kind": "LoRA",
+    "index": 19.22,
+    "areas": {
+     "knowledge": 0.0988,
+     "language": 0.2304,
+     "retrieval": 0.164,
+     "tools": 0.3623,
+     "arts": 0.08
+    }
+   },
+   {
+    "rank": 44,
+    "name": "Decision 1.0 Eos",
+    "size_b": 0.8,
+    "base_model": "Qwen/Qwen3.5-0.8B-Base",
+    "kind": "head / adapter",
+    "index": 18.41,
+    "areas": {
+     "knowledge": 0.0737,
+     "language": 0.1459,
+     "retrieval": 0.3057,
+     "tools": 0.321,
+     "arts": 0.0747
+    }
+   },
+   {
+    "rank": 45,
+    "name": "Kev 0.8B",
+    "size_b": 0.8,
+    "base_model": "Qwen/Qwen3.5-0.8B-Base",
+    "kind": "LoRA + head",
+    "index": 14.6,
+    "areas": {
+     "knowledge": 0.0608,
+     "language": 0.1133,
+     "retrieval": 0.1914,
+     "tools": 0.315,
+     "arts": 0.0511
+    }
+   },
+   {
+    "rank": 46,
+    "name": "Bosun v3.1 0.6B",
+    "size_b": 0.6,
+    "base_model": "Qwen/Qwen3-0.6B-Base",
+    "kind": "LoRA + head",
+    "index": 14.32,
+    "areas": {
+     "knowledge": 0.0283,
+     "language": 0.0383,
+     "retrieval": 0.3415,
+     "tools": 0.2844,
+     "arts": 0.0557
+    }
+   },
+   {
+    "rank": 47,
+    "name": "Tev1-0.8B-experimental",
+    "size_b": 0.8,
+    "base_model": "Qwen/Qwen3.5-0.8B-Base",
+    "kind": "full fine-tune",
+    "index": 12.85,
+    "areas": {
+     "knowledge": 0.0729,
+     "language": 0.1405,
+     "retrieval": 0.0488,
+     "tools": 0.3197,
+     "arts": 0.0512
+    }
+   },
+   {
+    "rank": 48,
+    "name": "Intern-Decision-0.8B",
+    "size_b": 0.8,
+    "base_model": "Qwen/Qwen3.5-0.8B-Base",
+    "kind": "full fine-tune",
+    "index": 11.94,
+    "areas": {
+     "knowledge": 0.0797,
+     "language": 0.1035,
+     "retrieval": 0.0644,
+     "tools": 0.2838,
+     "arts": 0.0722
+    }
+   },
+   {
+    "rank": 49,
+    "name": "MoJev",
+    "size_b": 0.8,
+    "base_model": "Qwen/Qwen3.5-0.8B-Base",
+    "kind": "head / adapter",
+    "index": 11.69,
+    "areas": {
+     "knowledge": 0.0557,
+     "language": 0.0908,
+     "retrieval": 0.1618,
+     "tools": 0.2186,
+     "arts": 0.0672
+    }
+   },
+   {
+    "rank": 50,
+    "name": "GLiNER2.5-Decide",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 11.21,
+    "areas": {
+     "knowledge": 0.0241,
+     "language": 0.0748,
+     "retrieval": 0.2532,
+     "tools": 0.1478,
+     "arts": 0.0887
+    }
+   },
+   {
+    "rank": 51,
+    "name": "Lavoir",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 8.69,
+    "areas": {
+     "knowledge": 0.0349,
+     "language": 0.1037,
+     "retrieval": 0.12,
+     "tools": 0.1302,
+     "arts": 0.0323
+    }
+   },
+   {
+    "rank": 52,
+    "name": "jeff",
+    "size_b": null,
+    "base_model": null,
+    "kind": "inference technique",
+    "index": 8.04,
+    "areas": {
+     "knowledge": 0.0305,
+     "language": 0.104,
+     "retrieval": 0.2131,
+     "tools": 0.0119,
+     "arts": 0.0077
+    }
+   },
+   {
+    "rank": 53,
+    "name": "CLM-v0.1-8B",
+    "size_b": 8.0,
+    "base_model": "Qwen/Qwen3-8B-Base",
+    "kind": "full fine-tune",
+    "index": 7.4,
+    "areas": {
+     "knowledge": 0.0311,
+     "language": 0.0523,
+     "retrieval": 0.1126,
+     "tools": 0.1398,
+     "arts": 0.043
+    }
+   },
+   {
+    "rank": 54,
+    "name": "GLiNER 2.5 base",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 6.76,
+    "areas": {
+     "knowledge": 0.021,
+     "language": 0.0748,
+     "retrieval": 0.0966,
+     "tools": 0.1116,
+     "arts": 0.0313
+    }
+   },
+   {
+    "rank": 55,
+    "name": "LFM2.5-2.6B-RLCD",
+    "size_b": 2.6,
+    "base_model": "LiquidAI/LFM2.5-2.6B-Base",
+    "kind": "full fine-tune",
+    "index": 6.76,
+    "areas": {
+     "knowledge": 0.0742,
+     "language": 0.0542,
+     "retrieval": 0.1053,
+     "tools": 0.0443,
+     "arts": 0.0525
+    }
+   },
+   {
+    "rank": 56,
+    "name": "Decision 1.0 Kai",
+    "size_b": null,
+    "base_model": null,
+    "kind": "head / adapter",
+    "index": 6.52,
+    "areas": {
+     "knowledge": 0.0383,
+     "language": 0.0291,
+     "retrieval": 0.0932,
+     "tools": 0.1423,
+     "arts": 0.0308
+    }
+   },
+   {
+    "rank": 57,
+    "name": "Laya",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 6.04,
+    "areas": {
+     "knowledge": 0.0356,
+     "language": 0.0903,
+     "retrieval": 0.0715,
+     "tools": 0.0582,
+     "arts": 0.0289
+    }
+   },
+   {
+    "rank": 58,
+    "name": "Julia 1",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 5.54,
+    "areas": {
+     "knowledge": 0.0661,
+     "language": 0.0139,
+     "retrieval": 0.0991,
+     "tools": 0.072,
+     "arts": 0.0174
+    }
+   },
+   {
+    "rank": 59,
+    "name": "system-one-gemma",
+    "size_b": null,
+    "base_model": null,
+    "kind": "LoRA + head",
+    "index": 5.07,
+    "areas": {
+     "knowledge": 0.0129,
+     "language": 0.0108,
+     "retrieval": 0.1722,
+     "tools": 0.0331,
+     "arts": 0.0408
+    }
+   },
+   {
+    "rank": 60,
+    "name": "Decision 1.0 Lex",
+    "size_b": null,
+    "base_model": null,
+    "kind": "head / adapter",
+    "index": 4.54,
+    "areas": {
+     "knowledge": 0.0188,
+     "language": 0.0241,
+     "retrieval": 0.0625,
+     "tools": 0.1086,
+     "arts": 0.0193
+    }
+   },
+   {
+    "rank": 61,
+    "name": "GLiNER 2.5 multilingual",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 4.26,
+    "areas": {
+     "knowledge": 0.009,
+     "language": 0.0636,
+     "retrieval": 0.0531,
+     "tools": 0.0587,
+     "arts": 0.025
+    }
+   },
+   {
+    "rank": 62,
+    "name": "GLiNER 2.5 small",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 3.82,
+    "areas": {
+     "knowledge": 0.0175,
+     "language": 0.0375,
+     "retrieval": 0.043,
+     "tools": 0.0698,
+     "arts": 0.0266
+    }
+   },
+   {
+    "rank": 63,
+    "name": "Qwen-2.5-1B-RLCD",
+    "size_b": 1.5,
+    "base_model": "Qwen/Qwen2.5-1.5B",
+    "kind": "inference technique",
+    "index": 3.78,
+    "areas": {
+     "knowledge": 0.014,
+     "language": 0.0471,
+     "retrieval": 0.0094,
+     "tools": 0.07,
+     "arts": 0.0731
+    }
+   },
+   {
+    "rank": 64,
+    "name": "Lumma-Fev-0.6B",
+    "size_b": 0.6,
+    "base_model": "FrontiersMind/Lumma-0.6B-Base",
+    "kind": "LoRA",
+    "index": 2.97,
+    "areas": {
+     "knowledge": 0.0234,
+     "language": 0.0091,
+     "retrieval": 0.0888,
+     "tools": 0.0112,
+     "arts": 0.0144
+    }
+   },
+   {
+    "rank": 65,
+    "name": "Verdict",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 1.87,
+    "areas": {
+     "knowledge": 0.0036,
+     "language": 0.0145,
+     "retrieval": 0.0,
+     "tools": 0.0658,
+     "arts": 0.0203
+    }
+   },
+   {
+    "rank": 66,
+    "name": "Lumma-Fev-0.1B",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 1.78,
+    "areas": {
+     "knowledge": 0.0151,
+     "language": 0.0062,
+     "retrieval": 0.0245,
+     "tools": 0.0265,
+     "arts": 0.0257
+    }
+   },
+   {
+    "rank": 67,
+    "name": "LFM2.5-350M-RLCD",
+    "size_b": null,
+    "base_model": null,
+    "kind": "full fine-tune",
+    "index": 1.38,
+    "areas": {
+     "knowledge": 0.0047,
+     "language": 0.0098,
+     "retrieval": 0.0002,
+     "tools": 0.0384,
+     "arts": 0.0298
+    }
+   }
+  ]
+ },
+ "wald_di": [
+  {
+   "id": "v1.0",
+   "label": "Wald-4B v1.0 \u00b7 effort medium",
+   "size_b": 4.0,
+   "index": 53.91,
+   "ci95": [
+    51.87,
+    55.27
+   ],
+   "areas": {
+    "knowledge": 0.3945,
+    "language": 0.6176,
+    "retrieval": 0.5193,
+    "tools": 0.782,
+    "arts": 0.3058
+   },
+   "source": "results/decision-index/boot021-0927-021A0-f10.json#arrmed",
+   "status": "committed"
+  }
+ ],
+ "effort_di": {
+  "none": {
+   "index": 42.89,
+   "ci95": [
+    41.16,
+    44.43
+   ],
+   "areas": {
+    "knowledge": 0.2859,
+    "language": 0.448,
+    "retrieval": 0.463,
+    "tools": 0.6369,
+    "arts": 0.3011
+   },
+   "source": "results/decision-index/boot021-0927-015G0-f4.json#g4one"
+  },
+  "low": {
+   "index": 45.23,
+   "ci95": [
+    43.54,
+    46.69
+   ],
+   "areas": {
+    "knowledge": 0.3545,
+    "language": 0.4672,
+    "retrieval": 0.4676,
+    "tools": 0.6355,
+    "arts": 0.3011
+   },
+   "source": "results/decision-index/boot021-0927-015G0-f4.json#g4low"
+  },
+  "medium": {
+   "index": 47.23,
+   "ci95": [
+    45.52,
+    48.86
+   ],
+   "areas": {
+    "knowledge": 0.3998,
+    "language": 0.5023,
+    "retrieval": 0.4671,
+    "tools": 0.642,
+    "arts": 0.2819
+   },
+   "source": "results/decision-index/boot021-0927-015G0-f4.json#g4med"
+  },
+  "high": {
+   "index": 49.14,
+   "ci95": [
+    47.21,
+    50.65
+   ],
+   "areas": {
+    "knowledge": 0.4415,
+    "language": 0.5279,
+    "retrieval": 0.4698,
+    "tools": 0.6605,
+    "arts": 0.2604
+   },
+   "source": "results/decision-index/boot021-0927-015G0-f4.json#g4high"
+  },
+  "high-k4": {
+   "index": 49.62,
+   "ci95": [
+    47.95,
+    51.16
+   ],
+   "areas": {
+    "knowledge": 0.4649,
+    "language": 0.5232,
+    "retrieval": 0.4698,
+    "tools": 0.6605,
+    "arts": 0.2604
+   },
+   "source": "results/decision-index/boot021-0927-015G0-f4.json#g4k4"
+  }
+ },
+ "qwen27_paren": {
+  "index": 47.78,
+  "ci95": [
+   45.95,
+   49.17
+  ],
+  "areas": {
+   "knowledge": 0.3485,
+   "language": 0.4867,
+   "retrieval": 0.557,
+   "tools": 0.6954,
+   "arts": 0.2329
+  },
+  "source": "results/decision-index/boot021-bigbase-0927.json#q27paren"
+ },
+ "jev_sample": {
+  "index": 57.19,
+  "ci95": [
+   55.29,
+   58.6
+  ],
+  "areas": {
+   "knowledge": 0.5044,
+   "language": 0.5832,
+   "retrieval": 0.566,
+   "tools": 0.763,
+   "arts": 0.3799
+  },
+  "source": "results/decision-index/boot021-0927.json#jev"
+ },
+ "board_named": {
+  "decider_4b": {
+   "rank": 15,
+   "index": 40.7,
+   "base_model": "Qwen/Qwen3.5-4B-Base"
+  },
+  "decider_35b": {
+   "rank": 11,
+   "index": 47.11,
+   "base_model": "Qwen/Qwen3.5-35B-A3B-Base"
+  }
+ },
+ "board_4_5": [
+  {
+   "rank": 4,
+   "name": "simple-jev \u00b7 Qwen3.8-27B (featherless)",
+   "size_b": 27.0,
+   "base_model": "Qwen/Qwen3.8-27B",
+   "kind": "inference technique",
+   "index": 55.74,
+   "areas": {
+    "knowledge": 0.3656,
+    "language": 0.6209,
+    "retrieval": 0.6325,
+    "tools": 0.7621,
+    "arts": 0.3648
+   }
+  },
+  {
+   "rank": 5,
+   "name": "Jebadiah 27B",
+   "size_b": 27.0,
+   "base_model": "Qwen/Qwen3.8-27B",
+   "kind": "LoRA",
+   "index": 54.67,
+   "areas": {
+    "knowledge": 0.3884,
+    "language": 0.6068,
+    "retrieval": 0.5389,
+    "tools": 0.7813,
+    "arts": 0.3873
+   }
+  }
+ ],
+ "verticals": [
+  {
+   "laya_0shot": 35.6,
+   "laya_p50_ms": 893.8,
+   "clm_0shot": 34.8,
+   "clm_p50_ms": 1059.5,
+   "task": "When2Call",
+   "metric": "accuracy",
+   "n_test": 500,
+   "ours_lora": 82.8,
+   "ours_ci95": [
+    79.2,
+    86.0
+   ],
+   "jev": 72.6,
+   "wald_0shot": 68.6,
+   "raw_qwen_0shot": 47.4,
+   "raw_source": "verticals/when2call/runs/20260926-2141/report.json",
+   "ours_minus_jev_ci95": [
+    6.2,
+    14.2
+   ],
+   "train_usd": 1.0,
+   "beats_jev": true
+  },
+  {
+   "laya_0shot": 33.9,
+   "laya_p50_ms": 897.7,
+   "clm_0shot": 0.3,
+   "clm_p50_ms": 1057.1,
+   "task": "BANKING77",
+   "metric": "macro-F1",
+   "n_test": 500,
+   "ours_lora": 92.8,
+   "ours_ci95": [
+    89.5,
+    94.6
+   ],
+   "jev": 78.1,
+   "wald_0shot": 77.4,
+   "raw_qwen_0shot": 66.8,
+   "raw_source": "verticals/banking77/runs/20260926-2200/report.json",
+   "ours_minus_jev_ci95": [
+    12.1,
+    18.6
+   ],
+   "train_usd": 0.64,
+   "beats_jev": true
+  },
+  {
+   "laya_0shot": 62.6,
+   "laya_p50_ms": 992.6,
+   "clm_0shot": 12.7,
+   "clm_p50_ms": 1064.2,
+   "task": "SGD intent",
+   "metric": "macro-F1",
+   "n_test": 500,
+   "ours_lora": 98.0,
+   "ours_ci95": [
+    92.6,
+    99.2
+   ],
+   "jev": 92.8,
+   "wald_0shot": 82.8,
+   "raw_qwen_0shot": 79.7,
+   "raw_source": "verticals/sgd-intent/runs/20260926-2227/report.json",
+   "ours_minus_jev_ci95": [
+    2.9,
+    8.2
+   ],
+   "train_usd": 0.83,
+   "beats_jev": true
+  },
+  {
+   "laya_0shot": 14.2,
+   "laya_p50_ms": 271.4,
+   "clm_0shot": 12.9,
+   "clm_p50_ms": 88.2,
+   "task": "ToxicChat",
+   "metric": "toxic-class F1",
+   "n_test": 5029,
+   "ours_lora": 84.9,
+   "ours_ci95": [
+    81.9,
+    87.6
+   ],
+   "jev": 80.4,
+   "wald_0shot": 73.4,
+   "raw_qwen_0shot": 36.2,
+   "raw_source": "verticals/toxicchat/runs/20260927-0045/report.json",
+   "ours_minus_jev_ci95": [
+    1.5,
+    7.5
+   ],
+   "train_usd": 0.2,
+   "beats_jev": true
+  },
+  {
+   "laya_0shot": null,
+   "laya_p50_ms": null,
+   "clm_0shot": 34.0,
+   "clm_p50_ms": 1064.9,
+   "task": "MetaTool",
+   "metric": "accuracy",
+   "n_test": 500,
+   "ours_lora": 97.0,
+   "ours_ci95": [
+    95.4,
+    98.4
+   ],
+   "jev": 83.0,
+   "wald_0shot": 80.4,
+   "raw_qwen_0shot": 75.8,
+   "raw_source": "verticals/metatool/runs/20260927-0228/report.json",
+   "ours_minus_jev_ci95": [
+    10.8,
+    17.2
+   ],
+   "train_usd": 0.79,
+   "beats_jev": true
+  },
+  {
+   "laya_0shot": null,
+   "laya_p50_ms": null,
+   "clm_0shot": 61.2,
+   "clm_p50_ms": 1066.2,
+   "task": "AndroidControl",
+   "metric": "accuracy",
+   "n_test": 500,
+   "ours_lora": 81.6,
+   "ours_ci95": [
+    78.0,
+    85.0
+   ],
+   "jev": 73.4,
+   "wald_0shot": 60.6,
+   "raw_qwen_0shot": 62.0,
+   "raw_source": "verticals/androidcontrol/runs/20260927-0245/report.json",
+   "ours_minus_jev_ci95": [
+    4.4,
+    12.4
+   ],
+   "train_usd": 1.48,
+   "beats_jev": true
+  },
+  {
+   "laya_0shot": 25.6,
+   "laya_p50_ms": 992.6,
+   "clm_0shot": 0.0,
+   "clm_p50_ms": 1062.2,
+   "task": "RouteLLM routing",
+   "metric": "needs-strong F1",
+   "n_test": 500,
+   "ours_lora": 26.8,
+   "ours_ci95": [
+    18.3,
+    35.4
+   ],
+   "jev": 0.0,
+   "wald_0shot": 11.3,
+   "raw_qwen_0shot": 33.3,
+   "raw_source": "verticals/model-routing/runs/20260927-0236/report.json",
+   "ours_minus_jev_ci95": [
+    18.3,
+    35.4
+   ],
+   "train_usd": 0.63,
+   "beats_jev": true
+  },
+  {
+   "laya_0shot": 13.3,
+   "laya_p50_ms": 250.0,
+   "clm_0shot": 61.7,
+   "clm_p50_ms": 266.9,
+   "task": "Agent-trajectory safety (hard)",
+   "metric": "unsafe-class F1",
+   "n_test": 500,
+   "ours_lora": 72.2,
+   "ours_ci95": [
+    68.0,
+    75.9
+   ],
+   "jev": 47.0,
+   "wald_0shot": 32.5,
+   "raw_qwen_0shot": 70.1,
+   "raw_source": "verticals/agent-trajectory-safety-hard/runs/20260927-0259/report.json",
+   "ours_minus_jev_ci95": [
+    18.3,
+    32.2
+   ],
+   "train_usd": 0.15,
+   "beats_jev": true
+  },
+  {
+   "laya_0shot": 2.1,
+   "laya_p50_ms": 1020.9,
+   "clm_0shot": 3.8,
+   "clm_p50_ms": 1383.2,
+   "task": "Mind2Web",
+   "metric": "accuracy",
+   "n_test": 480,
+   "ours_lora": 51.9,
+   "ours_ci95": [
+    47.3,
+    56.0
+   ],
+   "jev": 48.5,
+   "wald_0shot": 32.3,
+   "raw_qwen_0shot": 9.8,
+   "raw_source": "verticals/mind2web/runs/20260927-0119/report.json",
+   "ours_minus_jev_ci95": [
+    -1.2,
+    7.9
+   ],
+   "train_usd": 1.81,
+   "beats_jev": false
+  },
+  {
+   "laya_0shot": 56.6,
+   "laya_p50_ms": 220.6,
+   "clm_0shot": 30.6,
+   "clm_p50_ms": 1021.4,
+   "task": "Prompt injection",
+   "metric": "injection-class F1",
+   "n_test": 578,
+   "ours_lora": 79.4,
+   "ours_ci95": [
+    73.5,
+    84.5
+   ],
+   "jev": 82.8,
+   "wald_0shot": 71.3,
+   "raw_qwen_0shot": 31.6,
+   "raw_source": "verticals/prompt-injection/runs/20260927-0238/report.json",
+   "ours_minus_jev_ci95": [
+    -10.1,
+    3.1
+   ],
+   "train_usd": 0.12,
+   "beats_jev": false
+  },
+  {
+   "laya_0shot": 49.2,
+   "laya_p50_ms": 547.9,
+   "clm_0shot": 50.3,
+   "clm_p50_ms": 602.6,
+   "task": "RewardBench",
+   "metric": "accuracy",
+   "n_test": 1000,
+   "ours_lora": 83.4,
+   "ours_ci95": [
+    81.0,
+    85.7
+   ],
+   "jev": 90.6,
+   "wald_0shot": 83.7,
+   "raw_qwen_0shot": 65.0,
+   "raw_source": "verticals/rewardbench/runs/20260927-0048/report.json",
+   "ours_minus_jev_ci95": [
+    -9.4,
+    -5.1
+   ],
+   "train_usd": 1.49,
+   "beats_jev": false
+  },
+  {
+   "task": "COLD (Chinese)",
+   "metric": "macro-F1",
+   "n_test": 5323,
+   "ours_lora": 84.1,
+   "ours_ci95": [
+    83.2,
+    85.1
+   ],
+   "jev": 75.3,
+   "wald_0shot": 80.1,
+   "raw_qwen_0shot": null,
+   "raw_source": null,
+   "laya_0shot": null,
+   "laya_p50_ms": null,
+   "clm_0shot": null,
+   "clm_p50_ms": null,
+   "ours_minus_jev_ci95": [
+    7.7,
+    10.0
+   ],
+   "train_usd": 1.0,
+   "beats_jev": true,
+   "note": "LoRA on all 25,382 train labels averaged with the zero-shot base; ties the best published 83.7"
+  }
+ ],
+ "suite": [
+  {
+   "system": "Wald-4B v1.0 \u00b7 medium (packaged server)",
+   "public231": 88.3,
+   "xl_hidden": 63.2,
+   "source": "public 231: packaged wald-serve smoke read of 021A0-f10 09-27 (ledger 197); XL: xl-bench leaderboard.md (hidden split)"
+  },
+  {
+   "system": "Jev (API, jev-1.13.0)",
+   "public231": 86.6,
+   "xl_hidden": 73.0,
+   "source": "results/external/jev-1.13.0; xl-bench #1"
+  },
+  {
+   "system": "Cygnet (gemma-4-12B-it + shim)",
+   "public231": null,
+   "xl_hidden": 59.8,
+   "source": "xl-bench #4"
+  },
+  {
+   "system": "Laya (zero-shot, Jev's wire request)",
+   "public231": 58.0,
+   "xl_hidden": 13.4,
+   "source": "results/external/laya; xl-bench #24"
+  },
+  {
+   "system": "CLM-8B (zero-shot, same requests, deployment verified)",
+   "public231": 39.0,
+   "xl_hidden": 12.7,
+   "source": "ledger 190 read (results/external/clm-8b); xl-bench #25"
+  },
+  {
+   "system": "raw Qwen3.5-4B-Base (zero-shot, our readout)",
+   "public231": 67.5,
+   "xl_hidden": null,
+   "source": "exported lineage summary (public one-pass 156 / 231)"
+  }
+ ],
+ "effort_serving": [
+  {
+   "effort": "none",
+   "thinks": 0.0,
+   "public231": 194,
+   "xl_dev": 72.4,
+   "processbench": 64.2,
+   "p50_s": 0.051,
+   "p95_s": 0.117,
+   "usd_per_1k": 0.0184
+  },
+  {
+   "effort": "low",
+   "thinks": 1.2,
+   "public231": 192,
+   "xl_dev": 76.0,
+   "processbench": 64.2,
+   "p50_s": 0.048,
+   "p95_s": 0.109,
+   "usd_per_1k": 0.0194
+  },
+  {
+   "effort": "medium",
+   "thinks": 11.0,
+   "public231": 202,
+   "xl_dev": 81.1,
+   "processbench": 66.7,
+   "p50_s": 0.052,
+   "p95_s": 1.08,
+   "usd_per_1k": 0.0234
+  },
+  {
+   "effort": "high",
+   "thinks": 100.0,
+   "public231": 194,
+   "xl_dev": 85.8,
+   "processbench": 83.4,
+   "p50_s": 0.62,
+   "p95_s": 1.7,
+   "usd_per_1k": 0.0366
+  }
+ ],
+ "reference_api": {
+  "name": "Jev API",
+  "p50_s": 0.4,
+  "p95_s": 0.83,
+  "usd_per_1k": 0.04
+ },
+ "serving": {
+  "rtxpro6000_fp8_usd_per_1k": 0.0073,
+  "rtxpro6000_fp8_decisions_per_s": 114.9,
+  "rtxpro6000_c1_p50_ms": 26,
+  "multitenant_c32_usd_per_1k": 0.0067,
+  "multitenant_c32_decisions_per_s": 70,
+  "vertical_when2call_p50_ms": 56,
+  "vertical_banking77_p50_ms": 176,
+  "jev_when2call_p50_ms": 541,
+  "jev_banking77_p50_ms": 702,
+  "jev_usd_per_1k": 0.04
+ }
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": 248044,
+  "transformers_version": "5.17.0",
+  "use_cache": true
+}

model-files/serving.json

@@ -0,0 +1,6 @@
+{
+ "effort": "medium",
+ "prompt_format": "repeat_state_plain",
+ "max_model_len": 131072,
+ "temperature": "temperature.json"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dabeb7bc3f43bf6ea7d20ecfdac8758b3945517a991809a5772988829598e893
+size 8411558400

run.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+# One command, no Docker: create a Python 3.12 environment with vLLM 0.30.0 and wald-serve, then serve the weights on
+# 0.0.0.0:${PORT:-8000} (POST /v1/systemone, GET /health). Needs a Linux CUDA machine and `uv`.
+#
+#   ./run.sh /path/to/Wald-4B                  # declared policy from serving.json (effort medium)
+#   EFFORT=high ./run.sh /path/to/Wald-4B      # another policy: none | low | medium | high | high-k<k>
+set -euo pipefail
+MODEL=${1:?usage: run.sh <weights dir>}
+HERE=$(cd "$(dirname "$0")" && pwd)
+VENV=${VENV:-$HERE/.venv}
+[ -x "$VENV/bin/python" ] || { uv venv -p 3.12 "$VENV" && VIRTUAL_ENV="$VENV" uv pip install "vllm==0.30.0" "$HERE/server"; }
+export VLLM_USE_FLASHINFER_SAMPLER=0 VLLM_NO_USAGE_STATS=1 TOKENIZERS_PARALLELISM=false HF_HUB_OFFLINE=1
+exec "$VENV/bin/wald-serve" --model "$MODEL" --port "${PORT:-8000}" ${EFFORT:+--effort "$EFFORT"} --vllm-args "${VLLM_ARGS:-}"

server/README.md

@@ -0,0 +1,14 @@
+# wald-serve
+
+The TypeSafe `POST /v1/systemone` server for Wald-4B: the option-letter readout over a vLLM OpenAI-compatible server,
+the effort gate (none / low / medium / high / high-k), the knockout for more than 26 options, and the temperature table.
+It needs only public packages: `pydantic`, plus `vllm==0.30.0` on the GPU machine.
+
+```sh
+pip install ".[vllm]"                              # on a Linux CUDA machine
+wald-serve --model /path/to/Wald-4B --port 8000    # starts vLLM on the weights and serves /v1/systemone
+```
+
+Tests (no model, no GPU; a fake vLLM stands in): `pip install ".[test]" && cd tests && pytest -q`.
+`tests/test_parity.py` compares prompts and answers byte for byte with the reference implementation the published
+numbers were measured with; it is skipped unless that implementation is importable (WALD_PARITY_SRC set).

server/pyproject.toml

@@ -0,0 +1,22 @@
+[project]
+name = "wald-serve"
+version = "0.1.0"
+description = "TypeSafe POST /v1/systemone server for Wald-4B: option-letter readout over vLLM, effort gate, knockout, calibration"
+readme = "README.md"
+license = { text = "Apache-2.0" }
+requires-python = ">=3.10"
+dependencies = ["pydantic>=2.5"]
+
+[project.optional-dependencies]
+vllm = ["vllm==0.30.0"]
+test = ["pytest>=8"]
+
+[project.scripts]
+wald-serve = "wald_serve.server:main"
+
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools.packages.find]
+where = ["src"]

server/src/wald_serve/__init__.py

@@ -0,0 +1,2 @@
+"""Wald-4B serving: TypeSafe `POST /v1/systemone` over vLLM (letter readout, effort gate, knockout, calibration)."""
+__version__ = "0.1.0"

server/src/wald_serve/__main__.py

@@ -0,0 +1,3 @@
+from .server import main
+
+raise SystemExit(main())

server/src/wald_serve/engine.py

@@ -0,0 +1,285 @@
+"""Reading one request: a letter read per question over a vLLM OpenAI-compatible server, the effort gate, the knockout
+for more than 26 options, and the temperature table.
+
+Per question (2..26 options):
+  1. one-pass read: the prompt's next-token distribution over the option letters, z_L = log(P("L") + P(" L")),
+     softmax over the question's letters (mode `A`);
+  2. if the effort policy's gate g > 0 and the one-pass max p < g: generate a thought after `Reasoning:` (at most
+     `budget` tokens, temperature 0.6 / top-p 0.95 / top-k 20, stop at `\\nAnswer`, seed = hash of the request and the
+     question id), then read the letters again after `\\nAnswer: (` (mode `B`). With k > 1 thoughts the answer is the
+     mean of the k post-thought distributions; thought 0 is the k = 1 thought;
+  3. the answer is tempered with the checkpoint's temperature table (bucket = question type x option count; table `A`
+     for one-pass answers, `B<budget>` for thought answers). Tempering changes confidence, never the argmax.
+
+More than 26 options (knockout, mode `K`, one pass, no thought): the options are split, in their given order, into
+ceil(n / 26) near-equal consecutive chunks read with the same prompt; the chunk winners are then read as one final
+question; P(option) = P_final(its chunk) x P_chunk(option). This is one rule for every wide question.
+
+Declared capacity limits: a prompt longer than max_model_len is refused with HTTP 422 "maximum context length" (never
+truncated); a thought that would not fit falls back to the one-pass answer; at most 26 x 26 = 676 options per question
+(the wire format allows 255).
+"""
+from __future__ import annotations
+
+import hashlib
+import json
+import math
+import time
+import urllib.error
+import urllib.request
+from concurrent.futures import ThreadPoolExecutor
+from pathlib import Path
+
+from .prompt import CLOSE, CUE, LETTERS, STOP, THINK, question_prompt
+from .wire import SystemOneRequest, to_record
+
+BUCKETS = ((2, "2"), (4, "3-4"), (8, "5-8"), (math.inf, "9+"))
+
+
+class Capacity(ValueError):
+    """A declared capacity limit: answered with HTTP 422."""
+
+
+# --- effort policies --------------------------------------------------------------------------------------------------
+# gate: think when the one-pass max p < gate (0 = never, 1.01 = always); budget: max thought tokens; k: thoughts averaged.
+POLICIES = {
+    "none": {"gate": 0.0, "budget": 512, "k": 1},
+    "low": {"gate": 0.5, "budget": 512, "k": 1},
+    "medium": {"gate": 0.7, "budget": 512, "k": 1},
+    "high": {"gate": 1.01, "budget": 512, "k": 1},
+}
+
+
+def policy(name: str) -> dict:
+    """none / low / medium / high, or high-k<k> (always think, mean of k thoughts, 2 <= k <= 8)."""
+    if name in POLICIES:
+        return dict(POLICIES[name], name=name)
+    if name.startswith("high-k") and name[6:].isdigit() and 2 <= int(name[6:]) <= 8:
+        return {"gate": 1.01, "budget": 512, "k": int(name[6:]), "name": name}
+    raise ValueError(f"unknown effort {name!r}: none, low, medium, high or high-k<2..8>")
+
+
+# --- temperature ------------------------------------------------------------------------------------------------------
+def bucket_key(qtype: str, n: int) -> str:
+    return f"{qtype}|" + next(name for upper, name in BUCKETS if n <= upper)
+
+
+def temperature(table, qtype: str, n: int) -> float:
+    if not table:
+        return 1.0
+    b = (table.get("buckets") or {}).get(bucket_key(qtype, n))
+    return float(b["temperature"]) if b else float(table.get("single", 1.0))
+
+
+def temper(p, t: float):
+    if t == 1.0:
+        return list(p)
+    z = [math.log(max(x, 1e-12)) / t for x in p]
+    m = max(z)
+    e = [math.exp(v - m) for v in z]
+    s = sum(e)
+    return [v / s for v in e]
+
+
+def load_tables(path, budget: int = 512) -> dict:
+    """-> {"A": table, "B": table, "K": table}. A plain bucket table serves every mode; a gated table's thought entry is
+    B<budget> when present, else B."""
+    if not path:
+        return {}
+    t = json.loads(Path(path).read_text())
+    if "buckets" in t or "single" in t:
+        return {"A": t, "B": t, "K": t}
+    return {"A": t.get("A"), "B": t.get(f"B{budget}") or t.get("B"), "K": t.get("A")}
+
+
+def hash_seed(s: str) -> int:
+    return int(hashlib.sha256(s.encode()).hexdigest()[:8], 16)
+
+
+# --- vLLM client (stdlib; thread-safe, no shared mutable state after __init__) ----------------------------------------
+class Client:
+    def __init__(self, endpoint: str, served: str, max_len: int, prompt_format: str = "plain", timeout: float = 900):
+        self.url, self.served, self.max_len, self.timeout = endpoint.rstrip("/"), served, max_len, timeout
+        self.prompt_format = prompt_format
+        self.letter_ids = []
+        for L in LETTERS:
+            ids = []
+            for v in (L, " " + L):
+                t = self.ids(v)
+                if len(t) == 1 and t[0] not in ids:
+                    ids.append(t[0])
+            if not ids:
+                raise RuntimeError(f"no single-token encoding of letter {L!r}")
+            self.letter_ids.append(ids)
+        self.close = self.ids(CLOSE)
+
+    def post(self, path: str, body: dict, retries: int = 4):
+        last = None
+        for a in range(retries):
+            try:
+                req = urllib.request.Request(self.url + path, data=json.dumps(body).encode(), method="POST",
+                                             headers={"content-type": "application/json"})
+                with urllib.request.urlopen(req, timeout=self.timeout) as r:
+                    return json.loads(r.read())
+            except urllib.error.HTTPError as e:
+                detail = e.read()[:400].decode("utf-8", "replace")
+                if e.code in (400, 413, 422):
+                    if "maximum context length" in detail or "max_model_len" in detail or "too long" in detail:
+                        raise Capacity(f"maximum context length: {detail[:300]}") from None
+                    raise ValueError(f"{e.code}: {detail}") from None
+                last = RuntimeError(f"{e.code}: {detail}")
+            except (urllib.error.URLError, OSError) as e:
+                last = e
+            time.sleep(2 ** a)
+        raise RuntimeError(f"{path}: {last}")
+
+    def ids(self, text: str) -> list[int]:
+        return self.post("/tokenize", {"model": self.served, "prompt": text, "add_special_tokens": False})["tokens"]
+
+    def readout(self, ids: list[int], n: int):
+        """-> (softmax over the first n letters, total letter mass)."""
+        if len(ids) + 1 > self.max_len:
+            raise Capacity(f"prompt of {len(ids)} tokens is longer than the maximum context length {self.max_len}")
+        want = [t for L in self.letter_ids[:n] for t in L]
+        body = {"model": self.served, "prompt": ids, "max_tokens": 1, "temperature": 0.0, "logprobs": 20,
+                "logprob_token_ids": want, "return_tokens_as_token_ids": True}
+        r = self.post("/v1/completions", body)
+        top = r["choices"][0]["logprobs"]["top_logprobs"][0]
+        lp = {int(k.split(":", 1)[1]): v for k, v in top.items() if k.startswith("token_id:") and v is not None and v > -9999}
+        floor = min(lp.values()) - 2.0 if lp else -30.0
+        z = [math.log(sum(math.exp(lp.get(t, floor)) for t in L)) for L in self.letter_ids[:n]]
+        m = max(z)
+        e = [math.exp(v - m) for v in z]
+        s = sum(e)
+        return [x / s for x in e], sum(math.exp(v) for v in z)
+
+    def generate(self, ids: list[int], max_tokens: int, seed: int, n: int = 1):
+        """-> (text, completion tokens) for n = 1; (list of texts, total tokens) for n > 1."""
+        body = {"model": self.served, "prompt": ids, "max_tokens": max_tokens, "temperature": 0.6, "top_p": 0.95,
+                "top_k": 20, "seed": seed, "stop": [STOP], "include_stop_str_in_output": False}
+        if n > 1:
+            body["n"] = n
+        r = self.post("/v1/completions", body)
+        ntok = (r.get("usage") or {}).get("completion_tokens", 0)
+        if n > 1:
+            return [c.get("text") or "" for c in r["choices"]], ntok
+        c = r["choices"][0]
+        return c.get("text") or "", ntok
+
+
+# --- one question -----------------------------------------------------------------------------------------------------
+def chunks(n: int, size: int = 26) -> list[list[int]]:
+    k = math.ceil(n / size)
+    base, extra = divmod(n, k)
+    out, i = [], 0
+    for c in range(k):
+        m = base + (1 if c < extra else 0)
+        out.append(list(range(i, i + m)))
+        i += m
+    return out
+
+
+def subset(q: dict, idx: list[int]) -> dict:
+    return {**q, "options": [q["options"][i] for i in idx], "option_texts": [q["option_texts"][i] for i in idx]}
+
+
+def wide_read(n: int, read):
+    """Knockout over n > 26 options, given read(option indices) -> probabilities over them."""
+    groups = chunks(n)
+    within, winners = [], []
+    for g in groups:
+        p = read(g)
+        within.append(p)
+        winners.append(g[max(range(len(g)), key=lambda j: p[j])])
+    top = read(winners)
+    out = [0.0] * n
+    for c, g in enumerate(groups):
+        for j, i in enumerate(g):
+            out[i] = top[c] * within[c][j]
+    s = sum(out)
+    return [x / s for x in out]
+
+
+def read_question(cl: Client, state: str, q: dict, pol: dict, seed_text: str, usage: dict):
+    """-> (probabilities in the question's option order, mode A | B | K)."""
+    n = len(q["options"])
+    if n <= len(LETTERS):
+        prompt = question_prompt(state, q, cl.prompt_format)
+        ida = cl.ids(prompt)
+        usage["input_tokens"] += len(ida)
+        p, _ = cl.readout(ida, n)
+        gate, budget, k = pol["gate"], pol["budget"], pol["k"]
+        if gate > 0 and n > 1 and max(p) < gate:
+            pre = cl.ids(prompt[: -len(CUE)] + THINK)
+            if len(pre) + budget + 64 <= cl.max_len:   # else the one-pass answer stands
+                text, ntok = cl.generate(pre, budget, hash_seed(seed_text))
+                texts = [text]
+                usage["output_tokens"] += ntok
+                if k > 1:
+                    more, ntok = cl.generate(pre, budget, hash_seed(seed_text + "#k"), n=k - 1)
+                    texts += more
+                    usage["output_tokens"] += ntok
+                reads = []
+                for text in texts:
+                    body = text.strip()
+                    ids = pre + (cl.ids(" " + body) if body else []) + cl.close
+                    pb, _ = cl.readout(ids, n)
+                    usage["input_tokens"] += len(ids)
+                    reads.append(pb)
+                return [sum(r[i] for r in reads) / len(reads) for i in range(n)], "B"
+        return p, "A"
+    if len(chunks(n)) > len(LETTERS):
+        raise Capacity(f"{n} options: at most {len(LETTERS) ** 2} options per choice")
+
+    def read(idx):
+        ids = cl.ids(question_prompt(state, subset(q, idx), cl.prompt_format))
+        usage["input_tokens"] += len(ids)
+        return cl.readout(ids, len(idx))[0]
+    return wide_read(n, read), "K"
+
+
+def seed_base(req: dict) -> str:
+    """The thought seed covers the TypeSafe fields only (a per-request `effort` override does not change it)."""
+    return json.dumps({k: v for k, v in req.items() if k != "effort"}, sort_keys=True)
+
+
+def answer(cl: Client, req: dict, pol: dict, tables: dict, workers: int = 8):
+    """-> (answers in TypeSafe's wire format, usage). Questions are read concurrently (each question's read depends only
+    on the state, the question and its seed, so the order of reads does not change an answer)."""
+    rec, meta = to_record(SystemOneRequest.model_validate({k: v for k, v in req.items() if k != "effort"}))
+    base = seed_base(req)
+    jobs = []
+    for rq, m in zip(rec["questions"], meta):
+        q = {"id": m["id"], "type": rq["qtype"], "instructions": rq["instr"], "options": rq["options"],
+             "option_texts": rq["options"]}
+        jobs.append((q, m))
+
+    def one(job):
+        q, m = job
+        usage = {"input_tokens": 0, "output_tokens": 0}
+        p, mode = read_question(cl, rec["state"], q, pol, base + m["id"], usage)
+        return p, mode, usage
+
+    if workers > 1 and len(jobs) > 1:
+        with ThreadPoolExecutor(max_workers=min(workers, len(jobs))) as ex:
+            results = list(ex.map(one, jobs))
+    else:
+        results = [one(j) for j in jobs]
+
+    answers, usage = {}, {"input_tokens": 0, "output_tokens": 0}
+    for (q, m), (p, mode, u) in zip(jobs, results):
+        usage["input_tokens"] += u["input_tokens"]
+        usage["output_tokens"] += u["output_tokens"]
+        keys = m["keys"]
+        p = temper(p, temperature(tables.get(mode), m["type"], len(keys)))
+        j = max(range(len(p)), key=lambda i: p[i])
+        if m["type"] == "noul":
+            answers[m["id"]] = {"type": "noul", "noul": p[keys.index("true")], "mode": mode}
+        elif m["type"] == "choice":
+            K = len(p)
+            answers[m["id"]] = {"type": "choice", "choice": keys[j], "probabilities": dict(zip(keys, p)),
+                                "confidence": 1.0 if K == 1 else (p[j] - 1 / K) / (1 - 1 / K), "mode": mode}
+        else:
+            answers[m["id"]] = {"type": "score", "score": sum(i * v for i, v in enumerate(p)),
+                                "probabilities": {str(i): v for i, v in enumerate(p)}, "confidence": p[j], "mode": mode}
+    return answers, usage

server/src/wald_serve/prompt.py

@@ -0,0 +1,97 @@
+"""The plain letter prompt the model reads, one prompt per question. No chat template, no special tokens:
+
+    State:
+    {state}
+
+    Question: {instructions}
+    (A) {option 1}
+    (B) {option 2}
+    Answer: (
+
+The answer is read at the last position from the LM head's option-letter logits (engine.Client.readout). When the model
+thinks, the final `Answer: (` is replaced by `Reasoning:`, a short thought is generated, and the letters are read again
+after `\\nAnswer: (`.
+
+Prompt formats (one general rewrite applied to every question, never chosen per benchmark):
+    plain                the layout above
+    repeat_state_plain   the state written twice, the second copy introduced by STATE_REPEAT. The sentence is taken
+                         verbatim from featherless-ai/simple-jev @ dae340e (Apache-2.0; see NOTICE).
+"""
+from __future__ import annotations
+
+import re
+
+LETTERS = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+CUE = "Answer: ("
+THINK = "Reasoning:"
+CLOSE = "\nAnswer: ("
+STOP = "\nAnswer"
+STATE_TAIL = "Question:"
+
+# featherless-ai/simple-jev @ dae340e, hf-server/hf_prompt_policies.py (Apache-2.0), verbatim.
+STATE_REPEAT = ("\n\nRead the same context again before answering. This is a repeated copy, not additional events or "
+                "independent evidence:\n")
+PROMPT_FORMATS = ("plain", "repeat_state_plain")
+
+_SPECIAL = re.compile(r"<\|([A-Za-z0-9_]+)\|>")     # caller text never forms a <|special|> token
+_POSITIONAL = re.compile(r"[a-z]|o[0-9]+")          # choice keys that only number the options
+_SLUG = re.compile(r"[a-z0-9][a-z0-9_.-]{0,39}")
+
+
+def plain(text) -> str:
+    return _SPECIAL.sub(r"<¦\1¦>", str(text))
+
+
+def choice_keys(options) -> list[str]:
+    """The options themselves when they are all distinct short slugs, else a, b, c, ... (o1, o2, ... past 26)."""
+    opts = [str(o) for o in options]
+    if opts and len(set(opts)) == len(opts) and all(isinstance(o, str) and _SLUG.fullmatch(o) for o in options):
+        return opts
+    return [chr(ord("a") + i) if len(opts) <= 26 else f"o{i + 1}" for i in range(len(opts))]
+
+
+def raw_options(qtype: str, keys, texts) -> list[str]:
+    """The option strings after their letter: a choice question whose keys are all positional (a, b, ... / o27) drops
+    the `a: ` prefix, since the letter replaces it."""
+    if qtype == "choice":
+        keys = list(keys)
+        if all(_POSITIONAL.fullmatch(k) for k in keys):
+            return [t[len(k) + 2:] if t.startswith(f"{k}: ") else t for k, t in zip(keys, texts)]
+    return list(texts)
+
+
+def state_text(state: str) -> str:
+    """`State:\\n{state}\\n\\nQuestion:` (just `Question:` for a blank state)."""
+    return (f"State:\n{plain(state)}\n\n" if str(state).strip() else "") + STATE_TAIL
+
+
+def branch_text(q: dict) -> str:
+    """The rest of one question's prompt after `Question:`. q = {"type", "instructions", "options", "option_texts"}."""
+    texts = [str(o) for o in q["option_texts"]]
+    if len(texts) > len(LETTERS):
+        raise ValueError(f"{len(texts)} options: one letter read handles at most {len(LETTERS)}")
+    keys = choice_keys(q["options"]) if q["type"] == "choice" else None
+    out = f" {plain(q['instructions'])}"
+    for j, t in enumerate(raw_options(q["type"], keys, texts)):
+        out += f"\n({LETTERS[j]}) {plain(t)}"
+    return out + "\n" + CUE
+
+
+def format_prompt(prompt: str, fmt: str) -> str:
+    """Apply a prompt format to one question's plain prompt."""
+    if fmt == "plain":
+        return prompt
+    if not prompt.endswith(CUE):
+        raise ValueError("prompt does not end with the answer cue")
+    if fmt == "repeat_state_plain":
+        if prompt.startswith("State:\n") and "\n\nQuestion:" in prompt:
+            i = prompt.index("\n\nQuestion:")
+            state = prompt[len("State:\n"):i]
+            return "State:\n" + state + STATE_REPEAT + state + prompt[i:]
+        return prompt   # blank state: nothing to repeat
+    raise ValueError(f"unknown prompt format {fmt!r}; one of {PROMPT_FORMATS}")
+
+
+def question_prompt(state: str, q: dict, fmt: str = "plain") -> str:
+    """The one-pass prompt of one question under a rendered state."""
+    return format_prompt(state_text(state) + branch_text(q), fmt)

server/src/wald_serve/server.py

@@ -0,0 +1,152 @@
+"""`wald-serve`: the TypeSafe `POST /v1/systemone` server for Wald-4B.
+
+One command starts vLLM on the weights (a loopback-only sidecar) and this front-end in front of it:
+
+    wald-serve --model /path/to/Wald-4B --port 8000
+
+or attaches to a vLLM server that is already running:
+
+    wald-serve --vllm http://127.0.0.1:8011 --served wald --temperature /path/to/temperature.json --port 8000
+
+Defaults come from `<model>/serving.json` when present (`effort`, `prompt_format`, `max_model_len`), else the built-in
+ones below; command-line flags override both. The response carries TypeSafe's answer keys plus `mode` (A = one pass,
+B = after a thought, K = knockout) and `usage`.
+
+Endpoints: POST /v1/systemone (also POST /), GET /health, GET /v1/models.
+"""
+from __future__ import annotations
+
+import argparse
+import atexit
+import json
+import os
+import shlex
+import signal
+import subprocess
+import sys
+import time
+import urllib.request
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+
+from . import __version__
+from .engine import Capacity, Client, answer, load_tables, policy
+from .prompt import PROMPT_FORMATS
+
+DEFAULTS = {"effort": "medium", "prompt_format": "plain", "max_model_len": 131072}
+
+
+def read_serving(model_dir) -> dict:
+    p = Path(model_dir) / "serving.json" if model_dir else None
+    if p and p.is_file():
+        return {k: v for k, v in json.loads(p.read_text()).items() if k in DEFAULTS or k == "temperature"}
+    return {}
+
+
+def launch_vllm(model: str, served: str, port: int, max_len: int, mem: float, extra: str) -> subprocess.Popen:
+    cmd = [sys.executable, "-m", "vllm.entrypoints.openai.api_server", "--model", model, "--served-model-name", served,
+           "--host", "127.0.0.1", "--port", str(port), "--max-model-len", str(max_len), "--gpu-memory-utilization",
+           str(mem), "--max-num-seqs", "256", "--seed", "0", *shlex.split(extra)]
+    print(json.dumps({"launching": cmd}), flush=True)
+    proc = subprocess.Popen(cmd, start_new_session=True)
+
+    def stop():
+        if proc.poll() is None:
+            os.killpg(proc.pid, signal.SIGTERM)
+    atexit.register(stop)
+    signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
+    url = f"http://127.0.0.1:{port}/health"
+    for _ in range(1200):
+        if proc.poll() is not None:
+            raise SystemExit(f"vLLM exited with code {proc.returncode}")
+        try:
+            with urllib.request.urlopen(url, timeout=5):
+                return proc
+        except OSError:
+            time.sleep(2)
+    raise SystemExit("vLLM did not become healthy within 40 minutes")
+
+
+def make_handler(cl: Client, pol: dict, tables: dict, info: dict, workers: int):
+    class H(BaseHTTPRequestHandler):
+        protocol_version = "HTTP/1.1"
+
+        def log_message(self, *args):
+            pass
+
+        def send(self, code, obj):
+            b = json.dumps(obj).encode()
+            self.send_response(code)
+            self.send_header("content-type", "application/json")
+            self.send_header("content-length", str(len(b)))
+            self.end_headers()
+            self.wfile.write(b)
+
+        def do_GET(self):
+            if self.path.startswith("/v1/models"):
+                return self.send(200, {"object": "list", "data": [{"id": info["model"], "object": "model"}]})
+            self.send(200, {"ok": True, **info})
+
+        def do_POST(self):
+            try:
+                req = json.loads(self.rfile.read(int(self.headers.get("content-length", 0))))
+                p = policy(req["effort"]) if isinstance(req, dict) and req.get("effort") else pol
+            except Exception as e:   # noqa: BLE001
+                return self.send(400, {"error": f"{type(e).__name__}: {e}"[:400]})
+            t0 = time.perf_counter()
+            try:
+                ans, usage = answer(cl, req, p, tables, workers)
+            except Capacity as e:
+                return self.send(422, {"error": str(e)[:400]})
+            except Exception as e:   # noqa: BLE001
+                return self.send(400 if isinstance(e, ValueError) else 500, {"error": f"{type(e).__name__}: {e}"[:400]})
+            self.send(200, {"model": info["model"], "answers": ans, "usage": usage, "latency_s": time.perf_counter() - t0})
+    return H
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(prog="wald-serve", description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    src = ap.add_mutually_exclusive_group(required=True)
+    src.add_argument("--model", help="weights directory: start vLLM on it (serving.json / temperature.json read from it)")
+    src.add_argument("--vllm", help="URL of a running vLLM OpenAI-compatible server")
+    ap.add_argument("--served", default="wald", help="vLLM served model name")
+    ap.add_argument("--model-name", default="wald-4b", help="name reported in responses")
+    ap.add_argument("--effort", default=None, help="none | low | medium | high | high-k<k> (default: serving.json, else medium)")
+    ap.add_argument("--prompt-format", choices=PROMPT_FORMATS, default=None, help="default: serving.json, else plain")
+    ap.add_argument("--temperature", default=None, help="temperature table (default: <model>/temperature.json)")
+    ap.add_argument("--max-model-len", type=int, default=None, help="context limit in tokens (default: serving.json, else 131072)")
+    ap.add_argument("--question-workers", type=int, default=8, help="questions of one request read concurrently (1 = serial)")
+    ap.add_argument("--vllm-port", type=int, default=8011)
+    ap.add_argument("--gpu-memory-utilization", type=float, default=0.90)
+    ap.add_argument("--vllm-args", default="", help="extra `vllm serve` arguments, e.g. \"--quantization fp8\"")
+    ap.add_argument("--host", default="0.0.0.0")
+    ap.add_argument("--port", type=int, default=8000)
+    a = ap.parse_args(argv)
+
+    cfg = {**DEFAULTS, **read_serving(a.model)}
+    effort = a.effort or cfg["effort"]
+    fmt = a.prompt_format or cfg["prompt_format"]
+    max_len = a.max_model_len or int(cfg["max_model_len"])
+    pol = policy(effort)
+    temps = a.temperature or (str(Path(a.model) / cfg.get("temperature", "temperature.json")) if a.model else None)
+    if temps and not Path(temps).is_file():
+        raise SystemExit(f"temperature table not found: {temps}")
+    tables = load_tables(temps, pol["budget"])
+
+    if a.model:
+        launch_vllm(a.model, a.served, a.vllm_port, max_len, a.gpu_memory_utilization, a.vllm_args)
+        endpoint = f"http://127.0.0.1:{a.vllm_port}"
+    else:
+        endpoint = a.vllm
+    cl = Client(endpoint, a.served, max_len, prompt_format=fmt)
+    info = {"model": a.model_name, "version": __version__, "effort": pol["name"], "gate": pol["gate"],
+            "budget": pol["budget"], "think_k": pol["k"], "prompt_format": fmt, "max_model_len": max_len,
+            "wide": "knockout", "max_options": 676, "temperature": bool(tables)}
+    ThreadingHTTPServer.daemon_threads = True
+    httpd = ThreadingHTTPServer((a.host, a.port), make_handler(cl, pol, tables, info, a.question_workers))
+    print(json.dumps({"serving": info, "port": a.port}), flush=True)
+    httpd.serve_forever()
+
+
+if __name__ == "__main__":
+    sys.exit(main())

server/src/wald_serve/wire.py

@@ -0,0 +1,96 @@
+"""The TypeSafe `POST /v1/systemone` request shape and how a request becomes the text the model reads.
+
+A request is a `state` (string, object, array or null) plus `questions` (id -> noul / choice / score). Every question is
+answered with a probability for each of its options:
+
+    noul    options [no, yes]                      answer {"type": "noul", "noul": p(yes)}
+    choice  one option per criteria key            answer {"type": "choice", "choice": key, "probabilities": {...}}
+    score   one option per ordered level            answer {"type": "score", "score": E[level], "probabilities": {...}}
+
+`render` and `to_record` are adapted from Kev (github.com/jaredpalmer/kev, Apache-2.0; see NOTICE): the state and every
+option are flattened to text exactly as the model saw them in training.
+"""
+from __future__ import annotations
+
+from typing import Any, Literal, Union
+
+from pydantic import BaseModel, Field, model_validator
+
+JSONContent = Union[str, dict, list, int, float, bool, None]
+MAX_OPTIONS = 255
+
+
+class Noul(BaseModel):
+    type: Literal["noul"]
+    instructions: JSONContent = None
+    criteria: dict[str, JSONContent] | None = None
+
+
+class Choice(BaseModel):
+    type: Literal["choice"]
+    instructions: JSONContent = None
+    criteria: dict[str, JSONContent]
+
+    @model_validator(mode="after")
+    def _check(self):
+        if not 1 <= len(self.criteria) <= MAX_OPTIONS:
+            raise ValueError(f"criteria must have 1..{MAX_OPTIONS} options")
+        return self
+
+
+class Score(BaseModel):
+    type: Literal["score"]
+    instructions: JSONContent = None
+    criteria: list[JSONContent] = Field(min_length=1, max_length=MAX_OPTIONS)
+
+
+Question = Union[Noul, Choice, Score]
+
+
+class SystemOneRequest(BaseModel):
+    state: JSONContent
+    model: str = "wald"
+    questions: dict[str, Question] = Field(min_length=1)
+
+
+def render(v: JSONContent, indent: int = 0) -> str:
+    """Flatten str | object | array into text. Field names are kept as labels."""
+    pad = "  " * indent
+    if v is None:
+        return ""
+    if isinstance(v, (str, int, float, bool)):
+        return str(v)
+    if isinstance(v, list):
+        return "\n".join(f"{pad}- {render(x, indent + 1).lstrip()}" for x in v)
+    return "\n".join(f"{pad}{k}:\n{render(x, indent + 1)}" if isinstance(x, (dict, list)) else f"{pad}{k}: {render(x)}"
+                     for k, x in v.items())
+
+
+def option_text(name: str, desc: JSONContent) -> str:
+    return name if desc is None or desc == "" else f"{name}: {render(desc)}"
+
+
+def question_keys(qtype: str, criteria) -> list[str]:
+    """The keys a question's probabilities are reported under, in option order."""
+    if qtype == "choice":
+        return list(criteria)
+    if qtype == "noul":
+        return ["false", "true"]
+    return [str(i) for i in range(len(criteria))]
+
+
+def to_record(req: SystemOneRequest) -> tuple[dict[str, Any], list[dict[str, Any]]]:
+    """-> ({"state": text, "questions": [{"instr", "options", "qtype", "keys"}]}, [{"id", "type", "keys"}])."""
+    qs, meta = [], []
+    for qid, q in req.questions.items():
+        m = {"id": qid, "type": q.type, "keys": question_keys(q.type, q.criteria)}
+        if q.type == "noul":
+            c = q.criteria or {}
+            opts = [option_text("no", c.get("false")), option_text("yes", c.get("true"))]
+        elif q.type == "choice":
+            opts = [option_text(k, v) for k, v in q.criteria.items()]
+        else:
+            opts = [render(x) for x in q.criteria]
+        qs.append({"instr": render(q.instructions), "options": opts, "qtype": q.type, "keys": m["keys"]})
+        meta.append(m)
+    return {"state": render(req.state), "questions": qs}, meta

server/tests/fakevllm.py

@@ -0,0 +1,66 @@
+"""A stand-in for vLLM's OpenAI-compatible server, for tests without a model or a GPU.
+
+Tokenizer: one token per character (so "A" is one token and " A" is two). Completions: the logprob of every requested
+token id is a deterministic function of the prompt ids, so a read is reproducible; generation returns a short fixed
+thought per seed. A prompt longer than `max_len` answers 400 "maximum context length", as vLLM does.
+"""
+from __future__ import annotations
+
+import hashlib
+import json
+import threading
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+
+def _lp(prompt, tid):
+    h = hashlib.sha256(json.dumps([prompt[-64:], len(prompt), tid]).encode()).digest()
+    return -(int.from_bytes(h[:4], "big") % 4000) / 1000.0
+
+
+class FakeVLLM:
+    def __init__(self, max_len=100_000):
+        self.max_len = max_len
+        self.calls = {"tokenize": 0, "read": 0, "generate": 0}
+        fake = self
+
+        class H(BaseHTTPRequestHandler):
+            protocol_version = "HTTP/1.1"
+
+            def log_message(self, *a):
+                pass
+
+            def send(self, code, obj):
+                b = json.dumps(obj).encode()
+                self.send_response(code)
+                self.send_header("content-type", "application/json")
+                self.send_header("content-length", str(len(b)))
+                self.end_headers()
+                self.wfile.write(b)
+
+            def do_GET(self):
+                self.send(200, {"ok": True})
+
+            def do_POST(self):
+                body = json.loads(self.rfile.read(int(self.headers["content-length"])))
+                if self.path == "/tokenize":
+                    fake.calls["tokenize"] += 1
+                    return self.send(200, {"tokens": [ord(c) for c in body["prompt"]]})
+                prompt = body["prompt"]
+                if len(prompt) + body["max_tokens"] > fake.max_len:
+                    return self.send(400, {"message": f"This model's maximum context length is {fake.max_len} tokens."})
+                if body["max_tokens"] == 1:
+                    fake.calls["read"] += 1
+                    top = {f"token_id:{t}": _lp(prompt, t) for t in body["logprob_token_ids"]}
+                    return self.send(200, {"choices": [{"logprobs": {"top_logprobs": [top]}}]})
+                fake.calls["generate"] += 1
+                n = body.get("n", 1)
+                texts = [f" thought {body['seed'] % 997} #{i}: the evidence points one way." for i in range(n)]
+                return self.send(200, {"choices": [{"text": t} for t in texts], "usage": {"completion_tokens": 9 * n}})
+
+        self.httpd = ThreadingHTTPServer(("127.0.0.1", 0), H)
+        self.httpd.daemon_threads = True
+        self.url = f"http://127.0.0.1:{self.httpd.server_address[1]}"
+        threading.Thread(target=self.httpd.serve_forever, daemon=True).start()
+
+    def stop(self):
+        self.httpd.shutdown()

server/tests/test_parity.py

@@ -0,0 +1,102 @@
+"""Parity with the reference implementation the Decision Index numbers were measured with.
+
+Skipped unless WALD_PARITY_SRC points at a checkout that provides `eval.systemone_vllm` (with its `kev` and `midtrain`
+imports on PYTHONPATH). Checks, on generated requests, that this package builds byte-identical prompts and returns
+identical answers against the same (fake) vLLM for every effort policy and both prompt formats.
+"""
+from __future__ import annotations
+
+import os
+import random
+
+import pytest
+
+from fakevllm import FakeVLLM
+from wald_serve import engine
+from wald_serve.prompt import question_prompt
+from wald_serve.wire import SystemOneRequest, to_record
+
+pytestmark = pytest.mark.skipif(not os.environ.get("WALD_PARITY_SRC"), reason="reference implementation not available")
+
+WORDS = "the a order refund card payment late missing charged twice store policy item box user tool call ask".split()
+
+
+def text(rng, n):
+    return " ".join(rng.choice(WORDS) for _ in range(n))
+
+
+def gen_request(rng):
+    kind = rng.random()
+    if kind < 0.25:
+        state = ""
+    elif kind < 0.6:
+        state = text(rng, rng.randint(3, 60))
+    else:
+        state = {"ticket": text(rng, 20), "meta": {"n": rng.randint(1, 9), "tags": [text(rng, 2), text(rng, 3)]}}
+    qs = {}
+    for i in range(rng.randint(1, 5)):
+        t = rng.choice(["choice", "choice", "noul", "score"])
+        if t == "noul":
+            qs[f"q{i}"] = {"type": "noul", "instructions": text(rng, 6)}
+        elif t == "score":
+            qs[f"q{i}"] = {"type": "score", "instructions": text(rng, 5), "criteria": [text(rng, 2) for _ in range(rng.randint(2, 7))]}
+        else:
+            n = rng.choice([2, 3, 4, 9, 26, 27, 53, 77])
+            style = rng.random()
+            if style < 0.33:
+                crit = {chr(97 + j) if n <= 26 else f"o{j + 1}": text(rng, 3) for j in range(n)}
+            elif style < 0.66:
+                crit = {f"label_{j}": None for j in range(n)}
+            else:
+                crit = {f"Option {j}": text(rng, 4) for j in range(n)}
+            qs[f"q{i}"] = {"type": "choice", "instructions": text(rng, 7), "criteria": crit}
+    return {"state": state, "questions": qs}
+
+
+@pytest.fixture(scope="module")
+def fake():
+    f = FakeVLLM()
+    yield f
+    f.stop()
+
+
+def test_prompts_identical():
+    from eval.systemone_vllm import format_prompt as ref_format
+    from kev.api import SystemOneRequest as RefReq, to_record as ref_record
+    from midtrain.letter import question_prompt as ref_prompt
+    rng = random.Random(7)
+    n = 0
+    for _ in range(300):
+        req = gen_request(rng)
+        rec, _ = to_record(SystemOneRequest.model_validate(req))
+        rrec, _ = ref_record(RefReq.model_validate(req))
+        assert rec["state"] == rrec["state"]
+        for rq, rrq in zip(rec["questions"], rrec["questions"]):
+            q = {"type": rq["qtype"], "instructions": rq["instr"], "options": rq["options"], "option_texts": rq["options"]}
+            if len(q["options"]) > 26:
+                continue
+            for fmt in ("plain", "repeat_state_plain"):
+                assert question_prompt(rec["state"], q, fmt) == ref_format(ref_prompt(rrec["state"], q, "paren", "letter"), fmt)
+                n += 1
+    assert n > 500
+
+
+@pytest.mark.parametrize("effort,gate,k", [("none", 0.0, 1), ("low", 0.5, 1), ("medium", 0.7, 1), ("high", 1.01, 1),
+                                           ("high-k4", 1.01, 4)])
+@pytest.mark.parametrize("fmt", ["plain", "repeat_state_plain"])
+def test_answers_identical(fake, effort, gate, k, fmt):
+    from eval import systemone_vllm as ref
+    table = {"A": {"single": 1.4, "buckets": {"choice|2": {"temperature": 1.1}, "noul|2": {"temperature": 1.3}}},
+             "B512": {"single": 1.8, "buckets": {"choice|3-4": {"temperature": 2.2}}}}
+    tables = {"A": table["A"], "B": table["B512"], "K": table["A"]}
+    rcl = ref.Client(fake.url, "wald", 100_000)
+    rcl.wide, rcl.template, rcl.prompt_format, rcl.topk = "knockout", "paren", fmt, 0
+    cl = engine.Client(fake.url, "wald", 100_000, prompt_format=fmt)
+    rng = random.Random(11)
+    for _ in range(25):
+        req = gen_request(rng)
+        want, wu = ref.answer(rcl, req, gate, 512, tables, k)
+        got, gu = engine.answer(cl, req, engine.policy(effort), tables)
+        for a in want.values():
+            a.pop("one_pass", None)
+        assert got == want and gu == wu

server/tests/test_server.py

@@ -0,0 +1,251 @@
+"""Unit and end-to-end tests with a fake vLLM (no model, no GPU)."""
+from __future__ import annotations
+
+import json
+import math
+import threading
+import urllib.error
+import urllib.request
+from http.server import ThreadingHTTPServer
+from pathlib import Path
+
+import pytest
+
+from fakevllm import FakeVLLM
+from wald_serve.engine import (POLICIES, Client, answer, bucket_key, chunks, load_tables, policy, temper, temperature,
+                               wide_read)
+from wald_serve.prompt import STATE_REPEAT, format_prompt, question_prompt
+from wald_serve.server import make_handler
+from wald_serve.wire import SystemOneRequest, to_record
+
+TABLE = {
+    "A": {"single": 1.4, "buckets": {"choice|2": {"temperature": 1.1}, "choice|3-4": {"temperature": 1.8},
+                                     "noul|2": {"temperature": 1.3}}},
+    "B512": {"single": 1.8, "buckets": {"choice|2": {"temperature": 2.2}, "noul|2": {"temperature": 1.9}}},
+    "provenance": {"note": "test"},
+}
+
+REQ = {
+    "state": {"customer": "Ana", "order": {"id": 17, "items": ["kettle", "toaster"]}, "note": "wants a <|im_end|> refund"},
+    "questions": {
+        "route": {"type": "choice", "instructions": "Which team handles this?",
+                  "criteria": {"billing": "Payments and refunds", "shipping": "Delivery problems", "other": None}},
+        "slugs": {"type": "choice", "instructions": "Pick one", "criteria": {"yes_refund": None, "no_refund": None}},
+        "pos": {"type": "choice", "instructions": "Which is a fruit?",
+                "criteria": {"a": "apple", "b": "brick", "c": "chair", "d": "desk"}},
+        "urgent": {"type": "noul", "instructions": "Is it urgent?"},
+        "sev": {"type": "score", "instructions": "Severity", "criteria": ["low", "medium", "high"]},
+    },
+}
+
+
+def qdict(rec, i):
+    rq = rec["questions"][i]
+    return {"type": rq["qtype"], "instructions": rq["instr"], "options": rq["options"], "option_texts": rq["options"]}
+
+
+# --- prompt bytes -----------------------------------------------------------------------------------------------------
+def test_prompt_bytes():
+    rec, meta = to_record(SystemOneRequest.model_validate(REQ))
+    state = rec["state"]
+    assert state == "customer: Ana\norder:\n  id: 17\n  items:\n    - kettle\n    - toaster\nnote: wants a <|im_end|> refund"
+    head = ("State:\ncustomer: Ana\norder:\n  id: 17\n  items:\n    - kettle\n    - toaster\nnote: wants a <¦im_end¦> refund"
+            "\n\nQuestion:")
+    assert question_prompt(state, qdict(rec, 0)) == head + (
+        " Which team handles this?\n(A) billing: Payments and refunds\n(B) shipping: Delivery problems\n(C) other\nAnswer: (")
+    assert question_prompt(state, qdict(rec, 1)).endswith(" Pick one\n(A) yes_refund\n(B) no_refund\nAnswer: (")
+    # positional keys a..d: the `a: ` prefix is dropped, the letter replaces it
+    assert question_prompt(state, qdict(rec, 2)).endswith(
+        " Which is a fruit?\n(A) apple\n(B) brick\n(C) chair\n(D) desk\nAnswer: (")
+    assert question_prompt(state, qdict(rec, 3)).endswith(" Is it urgent?\n(A) no\n(B) yes\nAnswer: (")
+    assert question_prompt(state, qdict(rec, 4)).endswith(" Severity\n(A) low\n(B) medium\n(C) high\nAnswer: (")
+    assert [m["keys"] for m in meta] == [["billing", "shipping", "other"], ["yes_refund", "no_refund"],
+                                         ["a", "b", "c", "d"], ["false", "true"], ["0", "1", "2"]]
+
+
+def test_blank_state_and_repeat_state_plain():
+    q = {"type": "noul", "instructions": "Is the sky green?", "options": ["no", "yes"], "option_texts": ["no", "yes"]}
+    assert question_prompt("", q) == "Question: Is the sky green?\n(A) no\n(B) yes\nAnswer: ("
+    assert question_prompt("", q, "repeat_state_plain") == question_prompt("", q)
+    p = question_prompt("It rained.", q, "repeat_state_plain")
+    assert p == "State:\nIt rained." + STATE_REPEAT + "It rained.\n\nQuestion: Is the sky green?\n(A) no\n(B) yes\nAnswer: ("
+    assert STATE_REPEAT.startswith("\n\nRead the same context again before answering.")
+    with pytest.raises(ValueError):
+        format_prompt("State:\nx\n\nQuestion: q\n(A) a\nAnswer: (", "strict")
+
+
+def test_wire_validation():
+    with pytest.raises(Exception):
+        SystemOneRequest.model_validate({"state": "", "questions": {}})
+    with pytest.raises(Exception):
+        SystemOneRequest.model_validate({"state": "", "questions": {"q": {"type": "choice", "criteria": {}}}})
+    big = {f"o{i}": None for i in range(256)}
+    with pytest.raises(Exception):
+        SystemOneRequest.model_validate({"state": "", "questions": {"q": {"type": "choice", "criteria": big}}})
+
+
+# --- numerics ---------------------------------------------------------------------------------------------------------
+def test_chunks_and_knockout():
+    assert chunks(26) == [list(range(26))]
+    assert [len(c) for c in chunks(77)] == [26, 26, 25]
+    assert [len(c) for c in chunks(151)] == [26, 25, 25, 25, 25, 25]
+    assert sum(len(c) for c in chunks(255)) == 255
+
+    def read(idx):   # a fixed preference for lower indices
+        z = [math.exp(-i / 10) for i in idx]
+        s = sum(z)
+        return [v / s for v in z]
+    p = wide_read(77, read)
+    assert len(p) == 77 and abs(sum(p) - 1) < 1e-12 and max(range(77), key=lambda i: p[i]) == 0
+
+
+def test_temperature():
+    tables = json.loads(json.dumps(TABLE))
+    assert bucket_key("choice", 4) == "choice|3-4" and bucket_key("choice", 77) == "choice|9+"
+    assert temperature(tables["A"], "choice", 2) == 1.1 and temperature(tables["A"], "score", 7) == 1.4
+    p = [0.6, 0.3, 0.1]
+    q = temper(p, 1.8)
+    assert abs(sum(q) - 1) < 1e-12 and q.index(max(q)) == 0 and max(q) < 0.6
+
+
+def test_policies():
+    assert policy("medium") == {"gate": 0.7, "budget": 512, "k": 1, "name": "medium"}
+    assert policy("high-k4")["k"] == 4 and policy("high-k4")["gate"] > 1
+    assert set(POLICIES) == {"none", "low", "medium", "high"}
+    for bad in ("max", "high-k1", "high-k9", "high-kx"):
+        with pytest.raises(ValueError):
+            policy(bad)
+
+
+# --- end to end over HTTP with the fake vLLM ---------------------------------------------------------------------------
+@pytest.fixture(scope="module")
+def fake():
+    f = FakeVLLM(max_len=100_000)
+    yield f
+    f.stop()
+
+
+def serve(cl, effort="medium", tables=None, workers=8):
+    info = {"model": "wald-4b", "effort": effort}
+    httpd = ThreadingHTTPServer(("127.0.0.1", 0), make_handler(cl, policy(effort), tables or {}, info, workers))
+    httpd.daemon_threads = True
+    threading.Thread(target=httpd.serve_forever, daemon=True).start()
+    return httpd, f"http://127.0.0.1:{httpd.server_address[1]}"
+
+
+def post(url, body):
+    req = urllib.request.Request(url + "/v1/systemone", data=json.dumps(body).encode(), method="POST",
+                                 headers={"content-type": "application/json"})
+    try:
+        with urllib.request.urlopen(req, timeout=60) as r:
+            return r.status, json.loads(r.read())
+    except urllib.error.HTTPError as e:
+        return e.code, json.loads(e.read())
+
+
+def wide_request(n=77):
+    crit = {f"intent_{i:03d}": f"customer intent number {i}" for i in range(n)}
+    return {"state": "I was charged twice for one card payment.",
+            "questions": {"intent": {"type": "choice", "instructions": "Classify the banking intent.", "criteria": crit}}}
+
+
+def check_wire(req, resp):
+    """The Decision Index kit's validation: every question answered, a finite probability per option, sum 1 +- 0.01,
+    the choice among the options."""
+    assert set(resp["answers"]) == set(req["questions"])
+    for qid, q in req["questions"].items():
+        a = resp["answers"][qid]
+        assert a["type"] == q["type"]
+        if q["type"] == "noul":
+            assert 0.0 <= a["noul"] <= 1.0
+            continue
+        keys = list(q["criteria"]) if q["type"] == "choice" else [str(i) for i in range(len(q["criteria"]))]
+        assert list(a["probabilities"]) == keys
+        assert all(math.isfinite(v) and v >= 0 for v in a["probabilities"].values())
+        assert abs(sum(a["probabilities"].values()) - 1) < 0.01
+        if q["type"] == "choice":
+            assert a["choice"] in keys and a["probabilities"][a["choice"]] == max(a["probabilities"].values())
+
+
+def test_http_end_to_end(fake, tmp_path):
+    Path(tmp_path / "t.json").write_text(json.dumps(TABLE))
+    tables = load_tables(tmp_path / "t.json", 512)
+    cl = Client(fake.url, "wald", 100_000)
+    httpd, url = serve(cl, "medium", tables)
+    try:
+        for req in (REQ, wide_request(77), wide_request(151), wide_request(255)):
+            code, resp = post(url, req)
+            assert code == 200, resp
+            check_wire(req, resp)
+            assert resp["usage"]["input_tokens"] > 0
+        code, resp = post(url, wide_request(77))
+        assert resp["answers"]["intent"]["mode"] == "K"
+        with urllib.request.urlopen(url + "/health") as r:
+            assert json.loads(r.read())["ok"] is True
+    finally:
+        httpd.shutdown()
+
+
+def test_gate_modes(fake):
+    cl = Client(fake.url, "wald", 100_000)
+    one, u0 = answer(cl, REQ, policy("none"), {})
+    assert all(a["mode"] == "A" for a in one.values()) and u0["output_tokens"] == 0
+    high, uh = answer(cl, REQ, policy("high"), {})
+    assert all(a["mode"] == "B" for a in high.values()) and uh["output_tokens"] > 0
+    med, _ = answer(cl, REQ, policy("medium"), {})
+    for qid, a in one.items():
+        pmax = a["noul"] if a["type"] == "noul" else max(a["probabilities"].values())
+        if a["type"] == "noul":
+            pmax = max(pmax, 1 - pmax)
+        assert med[qid]["mode"] == ("B" if pmax < 0.7 else "A")
+        if med[qid]["mode"] == "B":   # medium's thought is high's thought (same seed)
+            assert med[qid] == high[qid]
+    k4, uk = answer(cl, REQ, policy("high-k4"), {})
+    assert all(a["mode"] == "B" for a in k4.values()) and uk["output_tokens"] == 4 * uh["output_tokens"]
+
+
+def test_workers_do_not_change_answers(fake):
+    cl = Client(fake.url, "wald", 100_000)
+    a1, u1 = answer(cl, REQ, policy("high"), {}, workers=1)
+    a8, u8 = answer(cl, REQ, policy("high"), {}, workers=8)
+    assert a1 == a8 and u1 == u8
+
+
+def test_effort_override_keeps_seed(fake):
+    cl = Client(fake.url, "wald", 100_000)
+    a, _ = answer(cl, REQ, policy("high"), {})
+    b, _ = answer(cl, {**REQ, "effort": "high"}, policy("high"), {})
+    assert a == b
+
+
+def test_prompt_format_changes_prompt_only(fake):
+    plain = Client(fake.url, "wald", 100_000)
+    rsp = Client(fake.url, "wald", 100_000, prompt_format="repeat_state_plain")
+    a, ua = answer(plain, REQ, policy("none"), {})
+    b, ub = answer(rsp, REQ, policy("none"), {})
+    assert ub["input_tokens"] > ua["input_tokens"]
+    check_wire(REQ, {"answers": b})
+
+
+def test_capacity_is_422(fake):
+    cl = Client(fake.url, "wald", 300)   # declared limit 300 "tokens" (characters in the fake)
+    httpd, url = serve(cl, "none")
+    try:
+        code, resp = post(url, {"state": "x " * 400, "questions": {"q": {"type": "noul", "instructions": "ok?"}}})
+        assert code == 422 and "maximum context length" in resp["error"]
+        code, resp = post(url, {"state": "short", "questions": {"q": {"type": "noul", "instructions": "ok?"}}})
+        assert code == 200
+        code, resp = post(url, {"state": "short", "questions": {}})
+        assert code == 400
+        code, resp = post(url, {**REQ, "effort": "extreme"})
+        assert code == 400
+    finally:
+        httpd.shutdown()
+
+
+def test_thought_that_does_not_fit_falls_back(fake):
+    q = {"state": "y " * 60, "questions": {"q": {"type": "choice", "instructions": "pick",
+                                                 "criteria": {"a": "one", "b": "two"}}}}
+    cl = Client(fake.url, "wald", 400)   # the prompt fits, prompt + 512-token budget does not
+    a, u = answer(cl, q, policy("high"), {})
+    assert a["q"]["mode"] == "A" and u["output_tokens"] == 0

serving.json

@@ -0,0 +1,6 @@
+{
+ "effort": "medium",
+ "prompt_format": "repeat_state_plain",
+ "max_model_len": 131072,
+ "temperature": "temperature.json"
+}

temperature.json

@@ -0,0 +1,154 @@
+{
+ "A": {
+  "single": 1.6817928305074288,
+  "buckets": {
+   "choice|2": {
+    "temperature": 1.5157165665103978,
+    "n": 37,
+    "fitted": true
+   },
+   "choice|3-4": {
+    "temperature": 1.9318726578496908,
+    "n": 69,
+    "fitted": true
+   },
+   "choice|5-8": {
+    "temperature": 1.6817928305074288,
+    "n": 17,
+    "fitted": false
+   },
+   "choice|9+": {
+    "temperature": 1.3660402567543957,
+    "n": 27,
+    "fitted": true
+   },
+   "noul|2": {
+    "temperature": 1.9999999999999998,
+    "n": 107,
+    "fitted": true
+   },
+   "score|5-8": {
+    "temperature": 1.6817928305074288,
+    "n": 18,
+    "fitted": false
+   }
+  },
+  "min_rows": 20,
+  "rows": 275,
+  "readout": "A"
+ },
+ "B256": {
+  "single": 4.756828460010884,
+  "buckets": {
+   "choice|2": {
+    "temperature": 6.498019170849887,
+    "n": 37,
+    "fitted": true
+   },
+   "choice|3-4": {
+    "temperature": 4.0,
+    "n": 69,
+    "fitted": true
+   },
+   "choice|5-8": {
+    "temperature": 4.756828460010884,
+    "n": 17,
+    "fitted": false
+   },
+   "choice|9+": {
+    "temperature": 3.863745315699382,
+    "n": 27,
+    "fitted": true
+   },
+   "noul|2": {
+    "temperature": 5.856342783782502,
+    "n": 107,
+    "fitted": true
+   },
+   "score|5-8": {
+    "temperature": 4.756828460010884,
+    "n": 18,
+    "fitted": false
+   }
+  },
+  "min_rows": 20,
+  "rows": 275,
+  "readout": "B256"
+ },
+ "B512": {
+  "single": 4.924577653379665,
+  "buckets": {
+   "choice|2": {
+    "temperature": 6.727171322029713,
+    "n": 37,
+    "fitted": true
+   },
+   "choice|3-4": {
+    "temperature": 4.43827788827138,
+    "n": 69,
+    "fitted": true
+   },
+   "choice|5-8": {
+    "temperature": 4.924577653379665,
+    "n": 17,
+    "fitted": false
+   },
+   "choice|9+": {
+    "temperature": 3.863745315699382,
+    "n": 27,
+    "fitted": true
+   },
+   "noul|2": {
+    "temperature": 5.856342783782502,
+    "n": 107,
+    "fitted": true
+   },
+   "score|5-8": {
+    "temperature": 4.924577653379665,
+    "n": 18,
+    "fitted": false
+   }
+  },
+  "min_rows": 20,
+  "rows": 275,
+  "readout": "B512"
+ },
+ "B": {
+  "single": 4.924577653379665,
+  "buckets": {
+   "choice|2": {
+    "temperature": 6.727171322029713,
+    "n": 37,
+    "fitted": true
+   },
+   "choice|3-4": {
+    "temperature": 4.43827788827138,
+    "n": 69,
+    "fitted": true
+   },
+   "choice|5-8": {
+    "temperature": 4.924577653379665,
+    "n": 17,
+    "fitted": false
+   },
+   "choice|9+": {
+    "temperature": 3.863745315699382,
+    "n": 27,
+    "fitted": true
+   },
+   "noul|2": {
+    "temperature": 5.856342783782502,
+    "n": 107,
+    "fitted": true
+   },
+   "score|5-8": {
+    "temperature": 4.924577653379665,
+    "n": 18,
+    "fitted": false
+   }
+  },
+  "min_rows": 20,
+  "rows": 275,
+  "readout": "B512"
+ }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd53432f0de26d67a83b634040d4f043053da4ecd0c759e7f4b24ad4f8bb9a81
+size 19989509

tokenizer_config.json

@@ -0,0 +1,32 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

tools/export_contamination.py

@@ -0,0 +1,90 @@
+"""Export contamination/trained-on-di-ids.json: every Decision Index item id found in the training data of the
+submitted checkpoint's whole lineage, with the training component(s) it was found in and why it is there.
+
+    python tools/export_contamination.py --src <research checkout>
+
+Inputs (relative to --src; committed scan outputs, ids and counts only, no item text):
+  results/decision-index/contamination-verified.json      strict-verified hits of the first scan (stage-1 corpus)
+  trainer/data/blocklists/filters/<dataset>.di-scan.json  full-blocklist scans of each training dataset
+"""
+from __future__ import annotations
+
+import argparse
+import collections
+import json
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+
+# lineage component -> the scan that covers it (the stage-1 corpus scan covers the verified stage-1 hits as well)
+COMPONENTS = {
+    "stage 1 (full-parameter decision training corpus)": "corpus-c11-train",
+    "stage 2 (LoRA refinement data)": "finetune-f3-stage2",
+    "stage 3 (short-thought distillation data)": "think-paren-v1",
+}
+VERIFIED_STAGE1 = "c9"
+# Stage 4 (the DI-targeted LoRA) and the RL arm's data were filtered with the full blocklist before training: 0 hits.
+
+WHY = {
+    "4": ("BANKING77", "scored", "train-split duplicate: the benchmark's public train split contains the same sentence as the test item"),
+    "5": ("CLINC150+OOS", "scored", "train-split duplicate: the public train split contains the same sentence as the test item"),
+    "28": ("WinoGrande", "scored", "train-split duplicate: the public train split contains the same sentence"),
+    "26": ("ARC-Easy", "display only", "train-split duplicate (ARC train / MMLU auxiliary_train), the board's known duplicates"),
+    "27": ("ARC-Challenge", "display only", "train-split duplicate (ARC train / MMLU auxiliary_train), the board's known duplicates"),
+    "24": ("MMLU", "display only", "question also present in MMLU auxiliary_train / other public QA sets"),
+    "2": ("ToolRet", "scored", "shared upstream source: ToolRet aggregates user queries of public tool-calling train sets (ToolACE, Glaive) that are in our data"),
+    "33": ("SATA-Bench", "scored", "shared upstream source: reading passage from RACE, which is part of MMLU auxiliary_train"),
+    "36": ("BRIGHT", "scored", "shared upstream source: a public programming problem statement (LeetCode)"),
+    "57": ("MMLU-Pro", "scored", "shared upstream source: MMLU-Pro includes problems from MATH / TheoremQA; the same problem is in our math data"),
+    "12": ("ANLI", "scored", "shared upstream text: the premise text also appears in another public dataset in our data"),
+    "6": ("RouterBench", "not scored in 0.2.1", "shared upstream source: RouterBench prompts embed GSM8K / MMLU questions that are in our math and QA data"),
+}
+
+
+def bench(i: str) -> str:
+    return i.split(":")[1] if i.startswith("candidates-v3:") else i.split(":")[0]
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--src", required=True)
+    a = ap.parse_args()
+    src = Path(a.src)
+    where = collections.defaultdict(set)
+    ver = json.loads((src / "results/decision-index/contamination-verified.json").read_text())
+    for ids in ver["run_ids"].get(VERIFIED_STAGE1, {}).values():
+        for i in ids:
+            where[i].add("stage 1 (full-parameter decision training corpus)")
+    scans = {}
+    for comp, name in COMPONENTS.items():
+        d = json.loads((src / f"trainer/data/blocklists/filters/{name}.di-scan.json").read_text())
+        scans[comp] = {"strict_hits_in_trained_files": d["strict_total"], "blocklist_sha256": d["blocklist"]["strict_sha256"],
+                       "suite_fingerprints_sha256": d["blocklist"]["suite_sha256"],
+                       "files": {f: {"lines": v["lines"], "sha256": v["sha256"], "strict": v["strict"]} for f, v in d["files"].items()}}
+        for i in d["di_items"]:
+            where[i].add(comp)
+    items = []
+    for i in sorted(where, key=lambda x: (int(bench(x)), x)):
+        b = bench(i)
+        name, scored, why = WHY[b]
+        items.append({"id": i, "benchmark_id": int(b), "benchmark": name, "scored_in_0_2_1": scored, "why": why,
+                      "found_in": sorted(where[i])})
+    per = collections.Counter((it["benchmark_id"], it["benchmark"], it["scored_in_0_2_1"]) for it in items)
+    out = {"schema": "wald/di-trained-on/1",
+           "what": "Decision Index 0.2 / 0.2.1 item ids present in the training data of the submitted checkpoint's lineage "
+                   "(strict match: the whole state in one training record, plus >= 50 % of option text where options are the "
+                   "item's content). Count every one of them as trained on.",
+           "total": len(items),
+           "per_benchmark": [{"benchmark_id": k[0], "benchmark": k[1], "scored_in_0_2_1": k[2], "items": n} for k, n in sorted(per.items())],
+           "scans": scans,
+           "stage_4_targeted_lora": "stage-4 data (Home-appliance generator rows; iSarcasmEval / API-Bank / ContractNLI / VAST / "
+                                    "NLI4CT / ACOS / RAGTruth train splits; replay) was checked against the full blocklist and the "
+                                    "sample rows before training: 0 hits after filtering",
+           "items": items}
+    (ROOT / "contamination").mkdir(exist_ok=True)
+    (ROOT / "contamination/trained-on-di-ids.json").write_text(json.dumps(out, indent=1) + "\n")
+    print(json.dumps({"total": len(items), "per_benchmark": out["per_benchmark"]}))
+
+
+if __name__ == "__main__":
+    main()

tools/export_figure_data.py

@@ -0,0 +1,166 @@
+"""Export the numbers behind figures/*.svg and the comparison table into figures/data.json.
+
+    python tools/export_figure_data.py --src <research checkout> --summary <exported summary.json>
+
+Reads only committed result files of the research checkout (paths below are relative to --src):
+  results/decision-index/board-live.json            the live Decision Index 0.2.1 board (official full-suite numbers)
+  results/decision-index/boot021-*.json             our paired-bootstrap reads on the 6,948-request stratified sample
+  verticals/<task>/runs/*/report.json               raw Qwen3.5-4B-Base zero-shot on each vertical's fixed test set
+and the exported summary (verticals: ours + LoRA / Jev / Wald zero-shot; effort table; serving cost and latency).
+
+A read that is not committed yet is taken from PENDING (a number relayed from the run, marked "reported") and is
+replaced automatically once its bootstrap file exists. Nothing here is recomputed: the script copies and labels.
+"""
+from __future__ import annotations
+
+import argparse
+import glob
+import json
+import re
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+
+# Our Decision Index rows. boot = (bootstrap file, model key) once committed; reported = relayed number until then.
+WALD_ROWS = [  # one row: v1.0 = 021A0-f10 at the declared policy (medium, prompt format repeat_state_plain)
+    {"id": "v1.0", "label": "Wald-4B v1.0 · effort medium", "boot": ("boot021-0927-021A0-f10.json", "arrmed"),
+     "reported": {"index": 53.91, "blocklist_adjusted": 53.62, "board_rank": 6},
+     "reported_areas_file": "figures/reported-areas-v1.0.json"},
+]
+EFFORT_KEYS = {"none": "g4one", "low": "g4low", "medium": "g4med", "high": "g4high", "high-k4": "g4k4"}
+QWEN27 = ("boot021-bigbase-0927.json", "q27paren")
+JEV_SAMPLE = ("boot021-0927.json", "jev")
+BOARD_NAMES = {"decider_4b": "Decider 4B", "decider_35b": "Decider 35B-A3B"}
+SUITE = [  # JevBench public 231 (accuracy %) and XL hidden split (XL-Int), zero-shot, same requests
+    {"system": "Wald-4B v1.0 · medium (packaged server)", "public231": round(100 * 204 / 231, 1), "xl_hidden": 63.2,
+     "source": "public 231: packaged wald-serve smoke read of 021A0-f10 09-27 (ledger 197); XL: xl-bench leaderboard.md (hidden split)"},
+    {"system": "Jev (API, jev-1.13.0)", "public231": 86.6, "xl_hidden": 73.0, "source": "results/external/jev-1.13.0; xl-bench #1"},
+    {"system": "Cygnet (gemma-4-12B-it + shim)", "public231": None, "xl_hidden": 59.8, "source": "xl-bench #4"},
+    {"system": "Laya (zero-shot, Jev's wire request)", "public231": 58.0, "xl_hidden": 13.4, "source": "results/external/laya; xl-bench #24"},
+    {"system": "CLM-8B (zero-shot, same requests, deployment verified)", "public231": 39.0, "xl_hidden": 12.7,
+     "source": "ledger 190 read (results/external/clm-8b); xl-bench #25"},
+    {"system": "raw Qwen3.5-4B-Base (zero-shot, our readout)", "public231": round(100 * 156 / 231, 1), "xl_hidden": None,
+     "source": "exported lineage summary (public one-pass 156 / 231)"},
+]
+NC_TASKS = {"WebLINX intent"}
+COLD_PAIRED_CI = [7.7, 10.0]   # paired bootstrap, docs: buyer-demo COLD v3 full test (+8.8)
+COLD_USD = 1.0                 # three COLD arms shared about 1 GPU-hour (about $1) on one RTX PRO 6000
+VERTICAL_TASKS = {  # summary task name -> vertical directory
+    "When2Call": "when2call", "BANKING77": "banking77", "SGD intent": "sgd-intent", "ToxicChat": "toxicchat",
+    "MetaTool": "metatool", "AndroidControl": "androidcontrol", "WebLINX intent": "weblinx-intent",
+    "RouteLLM routing": "model-routing", "Agent-trajectory safety (hard)": "agent-trajectory-safety-hard",
+    "Mind2Web": "mind2web", "Prompt injection": "prompt-injection", "RewardBench": "rewardbench",
+}
+
+
+def size_b(base_model: str | None):
+    """Total parameters in billions read from a base model name (35B-A3B -> 35, E4B -> 4); None when absent."""
+    if not base_model:
+        return None
+    m = re.search(r"(?<![A-Za-z0-9.])E?(\d+(?:\.\d+)?)B(?:-A\d+B)?", base_model.split("/")[-1])
+    return float(m.group(1)) if m else None
+
+
+def boot(src: Path, f: str, key: str):
+    p = src / "results/decision-index" / f
+    if not p.is_file():
+        return None
+    m = json.loads(p.read_text())["models"].get(key)
+    if not m:
+        return None
+    return {"index": m["balanced_skill"], "ci95": m["ci"]["balanced_skill"], "areas": m["areas"],
+            "source": f"results/decision-index/{f}#{key}"}
+
+
+def external_vertical(src: Path, task_dir: str, prefix: str):
+    """A zero-shot external system's read on a vertical (system id starting with `prefix`), with its p50 latency."""
+    for rep in sorted(glob.glob(str(src / "verticals" / task_dir / "runs/*/report.json"))):
+        for s in json.loads(Path(rep).read_text())["systems"]:
+            if s["id"] == prefix or (s["id"].startswith(prefix) and "multilingual" not in s["id"]):
+                return {"value": round(100 * s["metric"], 1), "p50_ms": s.get("p50_ms"), "source": str(Path(rep).relative_to(src))}
+    return None
+
+
+def raw_qwen_vertical(src: Path, task_dir: str):
+    for rep in sorted(glob.glob(str(src / "verticals" / task_dir / "runs/*/report.json"))):
+        for s in json.loads(Path(rep).read_text())["systems"]:
+            if s["id"] == "hosted-qwen35-4b-raw" or (s["kind"] == "base" and "(raw)" in s["label"] and "chat" not in s["label"]):
+                return {"value": round(100 * s["metric"], 1), "source": str(Path(rep).relative_to(src))}
+    return None
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--src", required=True)
+    ap.add_argument("--summary", required=True)
+    a = ap.parse_args()
+    src = Path(a.src)
+    summ = json.loads(Path(a.summary).read_text())
+    board = json.loads((src / "results/decision-index/board-live.json").read_text())
+
+    entries = sorted(board["entries"], key=lambda e: -e["index"])
+    out = {"board": {"edition": board["edition"], "generated_utc": board["generated_utc"],
+                     "jev": {"index": board["jev"]["index"], "areas": board["jev"]["areas"]},
+                     "entries": [{"rank": i + 1, "name": e["name"], "size_b": size_b(e["base_model"]),
+                                  "base_model": e["base_model"] if size_b(e["base_model"]) else None,
+                                  "kind": e["kind"], "index": e["index"], "areas": e["areas"]} for i, e in enumerate(entries)]}}
+    wald = []
+    for r in WALD_ROWS:
+        got = boot(src, *r["boot"])
+        row = {"id": r["id"], "label": r["label"], "size_b": 4.0}
+        if got:
+            row.update(got, status="committed")
+        elif r.get("reported"):
+            row.update(r["reported"], status="reported (bootstrap not committed yet)")
+            ra = ROOT / r["reported_areas_file"]
+            if ra.is_file():
+                row["areas"] = json.loads(ra.read_text())["areas"]
+        else:
+            continue
+        wald.append(row)
+    out["wald_di"] = wald
+    out["effort_di"] = {k: boot(src, "boot021-0927-015G0-f4.json", v) for k, v in EFFORT_KEYS.items()}
+    out["qwen27_paren"] = boot(src, *QWEN27)
+    out["jev_sample"] = boot(src, *JEV_SAMPLE)
+    out["board_named"] = {k: next(({"rank": e["rank"], "index": e["index"], "base_model": e["base_model"]}
+                                   for e in out["board"]["entries"] if e["name"] == n), None) for k, n in BOARD_NAMES.items()}
+    out["board_4_5"] = [out["board"]["entries"][3], out["board"]["entries"][4]]
+
+    verts = []
+    for v in summ["verticals"]:
+        d = VERTICAL_TASKS.get(v["task"])
+        raw = raw_qwen_vertical(src, d) if d else None
+        laya = external_vertical(src, d, "hosted-laya") if d else None
+        clm = external_vertical(src, d, "hosted-clm") if d else None
+        verts.append({"laya_0shot": laya and laya["value"], "laya_p50_ms": laya and laya["p50_ms"],
+                      "clm_0shot": clm and clm["value"], "clm_p50_ms": clm and clm["p50_ms"],
+                      "task": v["task"], "metric": v["metric"], "n_test": v["n_test"], "ours_lora": v["lora_all"],
+                      "ours_ci95": v.get("lora_all_ci95"), "jev": v["jev"], "wald_0shot": v["base"],
+                      "raw_qwen_0shot": raw["value"] if raw else None, "raw_source": raw["source"] if raw else None,
+                      "ours_minus_jev_ci95": v.get("all_minus_jev_ci95"), "train_usd": v.get("train_usd"),
+                      "beats_jev": v.get("beats_jev")})
+    # NC-licensed task data: internal only, not on the public card
+    verts = [v for v in verts if v["task"] not in NC_TASKS]
+    # COLD (Chinese offensive language), full test set, from the committed buyer-demo read
+    cold = json.loads((src / "results/buyer/cold-v3/full-test/summary.json").read_text())["systems"]
+    f1 = lambda k: round(100 * cold[k]["full5323"]["macro_f1"]["value"], 1)
+    ci = [round(100 * x, 1) for x in cold["01900-f8+base"]["full5323"]["macro_f1"]["ci95"]]
+    verts.append({"task": "COLD (Chinese)", "metric": "macro-F1", "n_test": 5323, "ours_lora": f1("01900-f8+base"),
+                  "ours_ci95": ci, "jev": f1("jev"), "wald_0shot": f1("015D0-f4"), "raw_qwen_0shot": None, "raw_source": None,
+                  "laya_0shot": None, "laya_p50_ms": None, "clm_0shot": None, "clm_p50_ms": None,
+                  "ours_minus_jev_ci95": COLD_PAIRED_CI, "train_usd": COLD_USD, "beats_jev": True,
+                  "note": "LoRA on all 25,382 train labels averaged with the zero-shot base; ties the best published 83.7"})
+    out["verticals"] = verts
+    # same-request suite reads (JevBench public 231 accuracy %, XL-Intelligence) from committed reports / notes
+    out["suite"] = SUITE
+    out["effort_serving"] = summ["effort"]
+    out["reference_api"] = summ["reference_api"]
+    out["serving"] = summ["serving"]
+    (ROOT / "figures").mkdir(exist_ok=True)
+    (ROOT / "figures/data.json").write_text(json.dumps(out, indent=1) + "\n")
+    print(json.dumps({"wald_di": [(w["id"], w["index"], w["status"]) for w in wald],
+                      "verticals_with_raw": sum(v["raw_qwen_0shot"] is not None for v in verts)}))
+
+
+if __name__ == "__main__":
+    main()

tools/make_figures.py

@@ -0,0 +1,299 @@
+"""Render figures/*.svg from figures/data.json (written by tools/export_figure_data.py). Standard library only:
+
+    python tools/make_figures.py
+
+Every SVG carries light and dark colors (prefers-color-scheme), a <title> tooltip on each mark and direct labels, so
+identity never depends on color alone. This script only draws the numbers in data.json.
+"""
+from __future__ import annotations
+
+import json
+import math
+from html import escape
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+D = json.loads((ROOT / "figures/data.json").read_text())
+OUT = ROOT / "figures"
+FONT = "system-ui, -apple-system, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif"
+AREAS = [("knowledge", "Knowledge"), ("language", "Language"), ("retrieval", "Retrieval"), ("tools", "Tools"), ("arts", "Arts")]
+
+STYLE = """
+<style>
+  svg { --surface:#fcfcfb; --ink:#0b0b0b; --ink2:#52514e; --muted:#8a8984; --grid:#e6e5e1; --line:#c9c8c2;
+        --s1:#2a78d6; --s2:#eb6834; --s3:#1baf7a; --s4:#8b6fd6; --laya:#c2369b; --clm:#9a7b2f; --dot:#b9b8b2; --hl:#e8f0fb; }
+  @media (prefers-color-scheme: dark) {
+    svg { --surface:#1a1a19; --ink:#ffffff; --ink2:#c3c2b7; --muted:#8f8e86; --grid:#2e2e2b; --line:#4a4a45;
+          --s1:#3987e5; --s2:#d95926; --s3:#199e70; --s4:#9d86e8; --laya:#e05cb8; --clm:#c9a24a; --dot:#5c5b56; --hl:#1d2a3b; }
+  }
+  text { font-family: FONT; fill: var(--ink); }
+  .t2 { fill: var(--ink2); } .mu { fill: var(--muted); }
+  .h1 { font-size: 16px; font-weight: 600; } .sm { font-size: 11px; } .xs { font-size: 10px; } .b { font-weight: 600; }
+  .grid { stroke: var(--grid); stroke-width: 1; } .axis { stroke: var(--line); stroke-width: 1; }
+</style>""".replace("FONT", FONT)
+
+
+def svg(w, h, body, title, desc):
+    return (f'<svg xmlns="http://www.w3.org/2000/svg" width="{w}" height="{h}" viewBox="0 0 {w} {h}" role="img" '
+            f'aria-labelledby="t d">\n<title id="t">{escape(title)}</title>\n<desc id="d">{escape(desc)}</desc>\n'
+            f'{STYLE}\n<rect width="{w}" height="{h}" fill="var(--surface)"/>\n' + "\n".join(body) + "\n</svg>\n")
+
+
+def text(x, y, s, cls="sm", anchor="start"):
+    return f'<text x="{x:.1f}" y="{y:.1f}" class="{cls}" text-anchor="{anchor}">{escape(str(s))}</text>'
+
+
+FAINT = ("--laya", "--clm")   # weak external baselines: shown, but pale so they do not draw the eye
+
+
+def op(var):
+    return ' fill-opacity="0.38"' if var in FAINT else ""
+
+
+def legend(body, x, y, items, step=190):
+    for name, var in items:
+        body.append(f'<rect x="{x}" y="{y - 10}" width="12" height="12" rx="2" fill="var({var})"{op(var)}/>')
+        body.append(text(x + 18, y, name, "sm mu" if var in FAINT else "sm"))
+        x += step
+
+
+# --- 1. Decision Index vs model size ----------------------------------------------------------------------------------
+def di_vs_size():
+    W, H, L, T, PW, PH = 1000, 520, 70, 90, 640, 360
+    xmin, xmax, ymin, ymax = math.log10(1.5), math.log10(45), 20, 60
+    sx = lambda b: L + PW * (math.log10(b) - xmin) / (xmax - xmin)
+    sy = lambda v: T + PH - PH * (v - ymin) / (ymax - ymin)
+    body = [text(24, 30, "Decision Index 0.2.1 vs model size: a 4B model among 12–35B entries", "h1"),
+            text(24, 50, "Grey: every board entry with a parameter count in its base model name (official full-suite scores). "
+                 "Blue: Wald-4B v1.0 on our 6,948-request stratified sample", "xs t2"),
+            text(24, 63, "(Jev scores 57.19 on that sample vs 57.89 on the board, so the sample reads about 0.7 low). "
+                 "Hollow = reported from a run whose result file is not committed yet.", "xs t2")]
+    for v in range(ymin, ymax + 1, 10):
+        body.append(f'<line x1="{L}" y1="{sy(v):.1f}" x2="{L + PW}" y2="{sy(v):.1f}" class="grid"/>')
+        body.append(text(L - 8, sy(v) + 4, v, "xs mu", "end"))
+    for b in (2, 4, 8, 12, 27, 35):
+        body.append(f'<line x1="{sx(b):.1f}" y1="{T}" x2="{sx(b):.1f}" y2="{T + PH}" class="grid"/>')
+        body.append(text(sx(b), T + PH + 16, f"{b}B", "xs mu", "middle"))
+    body.append(text(L + PW / 2, T + PH + 34, "parameters of the base model (log scale; MoE = total)", "xs mu", "middle"))
+    body.append(text(L - 50, T - 10, "index", "xs mu"))
+    jev = D["board"]["jev"]["index"]
+    body.append(f'<g><title>Jev (hosted API, size undisclosed): {jev}</title><line x1="{L}" y1="{sy(jev):.1f}" x2="{L + PW}" '
+                f'y2="{sy(jev):.1f}" stroke="var(--s2)" stroke-width="1.5" stroke-dasharray="6 4"/></g>')
+    body.append(text(L + 6, sy(jev) - 5, f"Jev {jev} (hosted API, size undisclosed)", "xs"))
+    named = {e["name"] for e in D["board"]["entries"][:5]} | {"Decider 4B", "Decider 35B-A3B", "Hopper", "JevK5"}
+    labels = []
+    for e in D["board"]["entries"]:
+        if not e["size_b"]:
+            continue
+        x, y = sx(e["size_b"]) + (sum(map(ord, e["name"])) % 7 - 3), sy(e["index"])
+        body.append(f'<g><title>#{e["rank"]} {escape(e["name"])} ({escape(e["base_model"] or "")}): {e["index"]}</title>'
+                    f'<circle cx="{x:.1f}" cy="{y:.1f}" r="4" fill="var(--dot)"/></g>')
+        if e["name"] in named:
+            labels.append([y, x, f"#{e['rank']} {e['name']} {e['index']}"])
+    labels.sort()
+    for i in range(1, len(labels)):   # keep labels in the same column at least 12 px apart
+        if abs(labels[i][1] - labels[i - 1][1]) < 80:
+            labels[i][0] = max(labels[i][0], labels[i - 1][0] + 12)
+    for y, x, s_ in labels:
+        body.append(text(x + 8, y + 3, s_, "xs t2"))
+    for i, w in enumerate(D["wald_di"]):
+        x, y = sx(4) + 14, sy(w["index"])
+        filled = w["status"] == "committed"
+        body.append(f'<g><title>{escape(w["label"])}: {w["index"]} ({w["status"]})</title><circle cx="{x:.1f}" cy="{y:.1f}" r="6" '
+                    f'fill="{"var(--s1)" if filled else "var(--surface)"}" stroke="var(--s1)" stroke-width="2.5"/></g>')
+        body.append(text(x + 10, y + 4, f"{w['label']} {w['index']}", "sm b" if i == 0 else "xs"))
+    body.append(f'<line x1="{L}" y1="{T}" x2="{L}" y2="{T + PH}" class="axis"/>')
+    return svg(W, H, body, "Decision Index vs model size", "Wald-4B against every sized board entry; Jev as a dashed line.")
+
+
+# --- 2. Decision Index areas --------------------------------------------------------------------------------------------
+def di_areas():
+    w = D["wald_di"][0]
+    rows = [("Jev (board)", D["board"]["jev"]["areas"], "--s2")]
+    if w.get("areas"):
+        rows.insert(0, (w["label"] + ("" if w["status"] == "committed" else " (reported)"), w["areas"], "--s1"))
+    for e, var in zip(D["board_4_5"], ("--s3", "--s4")):
+        rows.append((f"#{e['rank']} {e['name']} (board)", e["areas"], var))
+    W, L, PW, GH, BAR = 900, 110, 600, 78, 13
+    T = 146
+    H = T + len(AREAS) * GH + 40
+    sx = lambda v: L + PW * v
+    body = [text(24, 30, "Decision Index 0.2.1 by area (chance-corrected skill, 0–1)", "h1"),
+            text(24, 50, "Wald-4B v1.0 (4B) vs Jev and the board's #4 / #5 entries (27B). Board rows are official full-suite "
+                 "numbers; the Wald row is our stratified-sample read." if w.get("areas") else
+                 "Wald-4B v1.0 areas: pending its committed read. Board rows are official full-suite numbers.", "xs t2")]
+    for j, r in enumerate(rows):
+        body.append(f'<rect x="24" y="{66 + j * 16}" width="12" height="12" rx="2" fill="var({r[2]})"/>')
+        body.append(text(42, 76 + j * 16, r[0]))
+    for v in (0, 0.2, 0.4, 0.6, 0.8):
+        body.append(f'<line x1="{sx(v):.1f}" y1="{T - 6}" x2="{sx(v):.1f}" y2="{H - 30}" class="grid"/>')
+        body.append(text(sx(v), H - 16, f"{v:.1f}", "xs mu", "middle"))
+    for i, (aid, aname) in enumerate(AREAS):
+        y = T + i * GH
+        body.append(text(L - 10, y + 30, aname, "sm", "end"))
+        for j, (name, areas, var) in enumerate(rows):
+            v = areas[aid]
+            yy = y + j * (BAR + 3)
+            body.append(f'<g><title>{escape(name)} · {aname}: {v:.3f}</title><rect x="{L}" y="{yy}" width="{max(sx(v) - L, 1):.1f}" '
+                        f'height="{BAR}" rx="2" fill="var({var})"/></g>')
+            body.append(text(sx(v) + 4, yy + 10, f"{v:.2f}", "xs"))
+    body.append(f'<line x1="{L}" y1="{T - 6}" x2="{L}" y2="{H - 30}" class="axis"/>')
+    return svg(W, H, body, "Decision Index by area", "Five area skills for Wald-4B, Jev and the board's #4 and #5.")
+
+
+# --- 3. verticals --------------------------------------------------------------------------------------------------------
+def verticals():
+    rows = sorted(D["verticals"], key=lambda r: -(r["ours_lora"] - r["jev"]))
+    L, PW, GH, BAR = 230, 440, 80, 10
+    W, T = L + PW + 330, 132
+    H = T + len(rows) * GH + 50
+    sx = lambda v: L + PW * v / 100
+    wins = sum(bool(r["beats_jev"]) for r in rows)
+    lo_c, hi_c = min(r["train_usd"] for r in rows), max(r["train_usd"] for r in rows)
+    body = [text(24, 30, f"Quick LoRA (< $2, < 2 GPU-h per task) on Wald-4B beats Jev on {wins} of {len(rows)} tasks", "h1"),
+            text(24, 50, f"Same fixed test items and byte-identical requests for every system. Ours = one LoRA on the task's "
+                 f"labels, ${lo_c:.2f}–${hi_c:.2f} of GPU time each; the others are zero-shot. Jev cannot be fine-tuned.", "xs t2")]
+    series = [("ours_lora", "Wald-4B v0.9 + LoRA (< $2)", "--s1"), ("jev", "Jev (API, zero-shot)", "--s2"),
+              ("wald_0shot", "Wald-4B v0.9 zero-shot", "--s3"), ("raw_qwen_0shot", "raw Qwen3.5-4B-Base zero-shot", "--dot"),
+              ("laya_0shot", "Laya 421M (zero-shot, Jev's wire request)", "--laya"),
+              ("clm_0shot", "CLM-8B (zero-shot, deployment verified)", "--clm")]
+    legend(body, 24, 76, [(n, v) for _, n, v in series[:3]], step=330)
+    legend(body, 24, 94, [(n, v) for _, n, v in series[3:]], step=330)
+    missing = [n.split(" (")[0] for k, n, _ in series if all(r[k] is None for r in rows)]
+    if missing:
+        body.append(text(24, 112, f"Not shown (no committed read yet): {', '.join(missing)}. A task without a bar for a system "
+                         "has no read of it.", "xs mu"))
+    for v in range(0, 101, 20):
+        body.append(f'<line x1="{sx(v):.1f}" y1="{T - 8}" x2="{sx(v):.1f}" y2="{H - 40}" class="grid"/>')
+        body.append(text(sx(v), H - 26, v, "xs mu", "middle"))
+    body.append(text(L + PW / 2, H - 10, "task metric (%)", "xs mu", "middle"))
+    body.append(text(W - 270, T - 14, "ours − Jev [95 % CI] · LoRA cost", "xs t2"))
+    for i, r in enumerate(rows):
+        y = T + i * GH
+        body.append(text(L - 10, y + 16, r["task"], "sm", "end"))
+        body.append(text(L - 10, y + 30, f"{r['metric']} · n = {r['n_test']:,}", "xs mu", "end"))
+        for j, (key, name, var) in enumerate(series):
+            v = r[key]
+            if v is None:
+                continue
+            yy = y + j * (BAR + 2)
+            body.append(f'<g><title>{escape(r["task"])}: {escape(name)} {v:.1f}</title><rect x="{L}" y="{yy}" '
+                        f'width="{max(sx(v) - L, 1.5):.1f}" height="{BAR}" rx="2" fill="var({var})"{op(var)}/></g>')
+            if j == 0:
+                body.append(text(sx(v) + 4, yy + 9, f"{v:.1f}", "xs"))
+        lo, hi = r["ours_minus_jev_ci95"]
+        d = r["ours_lora"] - r["jev"]
+        mark = "▲" if r["beats_jev"] else ("▼" if hi < 0 else "n.s.")
+        body.append(text(W - 270, y + 18, f"{d:+.1f} [{lo:+.1f}, {hi:+.1f}] {mark} · ${r['train_usd']:.2f}", "sm"))
+    body.append(f'<line x1="{L}" y1="{T - 8}" x2="{L}" y2="{H - 40}" class="axis"/>')
+    return svg(W, H, body, "Verticals: quick LoRA vs Jev vs zero-shot", "Grouped bars for 12 tasks with paired differences to Jev.")
+
+
+# --- 4. effort -----------------------------------------------------------------------------------------------------------
+def effort():
+    E, DI, api = D["effort_serving"], D["effort_di"], D["reference_api"]
+    W, H = 1060, 420
+    body = [text(24, 30, "Effort: think only when unsure", "h1"),
+            text(24, 50, "Left: Decision Index 0.2.1 (stratified sample, 95 % CI) of one always-think read re-scored per "
+                 "policy (the next-generation base). Right: accuracy on our eval banks and serial latency per decision "
+                 "(Wald-4B, one H100).", "xs t2")]
+    L, T, PW, PH = 70, 90, 360, 250
+    keys = list(DI)
+    xs = {k: L + 30 + i * (PW - 60) / (len(keys) - 1) for i, k in enumerate(keys)}
+    lo, hi = 38, 54
+    sy = lambda v: T + PH - PH * (v - lo) / (hi - lo)
+    for v in range(lo, hi + 1, 4):
+        body.append(f'<line x1="{L}" y1="{sy(v):.1f}" x2="{L + PW}" y2="{sy(v):.1f}" class="grid"/>')
+        body.append(text(L - 8, sy(v) + 4, v, "xs mu", "end"))
+    body.append(text(L - 50, T - 10, "Decision Index", "xs mu"))
+    pts = []
+    for k in keys:
+        r = DI[k]
+        if not r:
+            continue
+        x, (a, b) = xs[k], r["ci95"]
+        body.append(f'<line x1="{x:.1f}" y1="{sy(a):.1f}" x2="{x:.1f}" y2="{sy(b):.1f}" stroke="var(--s1)" stroke-width="2"/>')
+        body.append(f'<g><title>effort {k}: {r["index"]} [{a}, {b}]</title><circle cx="{x:.1f}" cy="{sy(r["index"]):.1f}" r="5" '
+                    f'fill="{"var(--s1)" if k != "medium" else "var(--s2)"}" stroke="var(--surface)" stroke-width="2"/></g>')
+        body.append(text(x + 8, sy(r["index"]) + 4, f"{r['index']:.1f}", "xs"))
+        body.append(text(x, T + PH + 18, k, "sm b" if k == "medium" else "sm", "middle"))
+        pts.append((x, sy(r["index"])))
+    body.append(f'<polyline points="{" ".join(f"{x:.1f},{y:.1f}" for x, y in pts)}" fill="none" stroke="var(--s1)" '
+                f'stroke-width="1" stroke-dasharray="3 3"/>')
+    body.append(text(L, T + PH + 36, "declared policy: medium (orange) · high-k4 = mean of 4 thoughts", "xs mu"))
+    # right panel: accuracy (XL dev) vs latency, one point per effort
+    R0, RT, RW, RH = 580, 90, 380, 250
+    lmin, lmax = math.log10(0.03), math.log10(3.0)
+    lx = lambda s: R0 + RW * (math.log10(s) - lmin) / (lmax - lmin)
+    alo, ahi = 70, 90
+    ay = lambda v: RT + RH - RH * (v - alo) / (ahi - alo)
+    for s in (0.03, 0.1, 0.3, 1, 3):
+        body.append(f'<line x1="{lx(s):.1f}" y1="{RT}" x2="{lx(s):.1f}" y2="{RT + RH}" class="grid"/>')
+        body.append(text(lx(s), RT + RH + 16, f"{s:g} s", "xs mu", "middle"))
+    for v in range(alo, ahi + 1, 5):
+        body.append(f'<line x1="{R0}" y1="{ay(v):.1f}" x2="{R0 + RW}" y2="{ay(v):.1f}" class="grid"/>')
+        body.append(text(R0 - 8, ay(v) + 4, v, "xs mu", "end"))
+    body.append(text(R0 - 40, RT - 10, "XL dev accuracy %", "xs mu"))
+    body.append(text(R0 + RW / 2, RT + RH + 34, "serial latency per decision, p50 (dot) to p95 (ring), log scale", "xs mu", "middle"))
+    for e in E:
+        y, x1, x2 = ay(e["xl_dev"]), lx(e["p50_s"]), lx(e["p95_s"])
+        body.append(f'<g><title>effort {e["effort"]}: XL dev {e["xl_dev"]}, p50 {e["p50_s"]} s, p95 {e["p95_s"]} s, thinks on '
+                    f'{e["thinks"]} %</title><line x1="{x1:.1f}" y1="{y:.1f}" x2="{x2:.1f}" y2="{y:.1f}" stroke="var(--s1)" stroke-width="2"/>'
+                    f'<circle cx="{x1:.1f}" cy="{y:.1f}" r="5" fill="var(--s1)" stroke="var(--surface)" stroke-width="2"/>'
+                    f'<circle cx="{x2:.1f}" cy="{y:.1f}" r="5" fill="var(--surface)" stroke="var(--s1)" stroke-width="2"/></g>')
+        body.append(text(x1 - 8, y + 4, f"{e['effort']} ({e['thinks']:g} % think)", "xs", "end"))
+    xa = lx(api["p50_s"])
+    body.append(f'<line x1="{xa:.1f}" y1="{RT}" x2="{xa:.1f}" y2="{RT + RH}" stroke="var(--s2)" stroke-width="1.5" stroke-dasharray="5 4"/>')
+    body.append(text(xa + 5, RT + 12, f"Jev API p50 {api['p50_s']:g} s", "xs t2"))
+    return svg(W, H, body, "Effort vs accuracy vs latency", "Index rises with effort; median latency stays near one pass up to medium.")
+
+
+# --- 5. latency and cost vs Jev ----------------------------------------------------------------------------------------
+def latency_cost():
+    S, api = D["serving"], D["reference_api"]
+    W, H = 1000, 380
+    body = [text(24, 30, "Latency and cost per decision vs Jev", "h1"),
+            text(24, 50, "Left: median serial latency (ms). Right: $ per 1,000 decisions. Wald-4B on one RTX PRO 6000 "
+                 "(self-hosted GPU-hour cost at measured throughput); Jev at API list price.", "xs t2")]
+    vt = {r["task"]: r for r in D["verticals"]}
+    lat = [("one question, one pass (c = 1)", S["rtxpro6000_c1_p50_ms"], None),
+           ("When2Call request", S["vertical_when2call_p50_ms"], S["jev_when2call_p50_ms"]),
+           ("BANKING77 request (77 options)", S["vertical_banking77_p50_ms"], S["jev_banking77_p50_ms"])]
+    L, T, PW = 220, 100, 250
+    mx = 1100
+    for i, (name, w, j) in enumerate(lat):
+        y = T + i * 70
+        body.append(text(L - 10, y + 14, name, "sm", "end"))
+        key = "When2Call" if "When2Call" in name else ("BANKING77" if "BANKING77" in name else None)
+        laya = vt[key]["laya_p50_ms"] if key else None
+        clm = vt[key]["clm_p50_ms"] if key else None
+        for k, (v, var, who) in enumerate(((w, "--s1", "Wald-4B"), (j, "--s2", "Jev"), (laya and round(laya), "--laya", "Laya"),
+                                           (clm and round(clm), "--clm", "CLM-8B"))):
+            if v is None:
+                continue
+            yy = y + k * 16
+            body.append(f'<g><title>{escape(name)}: {escape(str(who))} {v} ms</title><rect x="{L}" y="{yy}" width="{max(PW * v / mx, 1.5):.1f}" '
+                        f'height="13" rx="2" fill="var({var})"{op(var)}/></g>')
+            body.append(text(L + PW * v / mx + 4, yy + 10, f"{who} {v:g} ms", "xs mu" if var in FAINT else "xs"))
+    R0 = 740
+    costs = [("Wald-4B fp8, batched", S["rtxpro6000_fp8_usd_per_1k"], "--s1"),
+             ("Wald-4B multi-tenant, c = 32", S["multitenant_c32_usd_per_1k"], "--s1"),
+             ("Jev API", api["usd_per_1k"], "--s2")]
+    for i, (name, v, var) in enumerate(costs):
+        y = T + i * 40
+        body.append(text(R0 - 10, y + 11, name, "sm", "end"))
+        body.append(f'<g><title>{escape(name)}: ${v} per 1,000 decisions</title><rect x="{R0}" y="{y}" width="{200 * v / 0.045:.1f}" '
+                    f'height="14" rx="2" fill="var({var})"/></g>')
+        body.append(text(R0 + 200 * v / 0.045 + 4, y + 11, f"${v:.4f}", "xs"))
+    ratio = api["usd_per_1k"] / S["rtxpro6000_fp8_usd_per_1k"]
+    body.append(text(R0 - 150, T + 140, f"Jev costs about {ratio:.1f}× more per decision.", "sm b"))
+    body.append(text(24, H - 14, "Laya: its own server, as measured in the same vertical runs. CLM-8B: latency and cost not "
+                     "committed yet. Not every system was measured on the same GPU; see the model card.", "xs mu"))
+    return svg(W, H, body, "Latency and cost vs Jev", "Median latency and dollars per 1,000 decisions, Wald-4B vs Jev.")
+
+
+if __name__ == "__main__":
+    for name, fn in (("di-vs-size", di_vs_size), ("di-areas", di_areas), ("verticals", verticals),
+                     ("latency-cost", latency_cost)):
+        (OUT / f"{name}.svg").write_text(fn())
+        print("wrote", OUT / f"{name}.svg")

tools/prepare_model_dir.py

@@ -0,0 +1,31 @@
+"""Prepare a weights directory for release: copy the checkpoint's own temperature table without its provenance field,
+write serving.json (declared policy and prompt format), and write MANIFEST.json with the sha256 of every file.
+
+    python tools/prepare_model_dir.py WEIGHTS_DIR --temperature path/to/temperature.json --prompt-format plain
+"""
+import argparse, hashlib, json
+from pathlib import Path
+
+ap = argparse.ArgumentParser()
+ap.add_argument("weights"); ap.add_argument("--temperature", required=True)
+ap.add_argument("--prompt-format", choices=("plain", "repeat_state_plain"), required=True)
+ap.add_argument("--effort", default="medium"); ap.add_argument("--max-model-len", type=int, default=131072)
+a = ap.parse_args()
+w = Path(a.weights)
+t = json.loads(Path(a.temperature).read_text())
+t = {k: v for k, v in t.items() if k in ("A", "B", "B256", "B512", "buckets", "single")}
+for k, v in t.items():   # keep only the fields the server reads
+    if isinstance(v, dict):
+        t[k] = {kk: vv for kk, vv in v.items() if kk in ("single", "buckets", "min_rows", "rows", "readout")}
+(w / "temperature.json").write_text(json.dumps(t, indent=1) + "\n")
+(w / "serving.json").write_text(json.dumps({"effort": a.effort, "prompt_format": a.prompt_format,
+                                            "max_model_len": a.max_model_len, "temperature": "temperature.json"}, indent=1) + "\n")
+man = {}
+for f in sorted(w.rglob("*")):
+    if f.is_file() and f.name != "MANIFEST.json" and ".cache" not in f.parts:
+        h = hashlib.sha256()
+        with open(f, "rb") as fh:
+            for b in iter(lambda: fh.read(1 << 24), b""): h.update(b)
+        man[str(f.relative_to(w))] = {"bytes": f.stat().st_size, "sha256": h.hexdigest()}
+(w / "MANIFEST.json").write_text(json.dumps(man, indent=1) + "\n")
+print(json.dumps({"files": len(man), "serving": json.loads((w / "serving.json").read_text())}))

tools/secret_scan.py

@@ -0,0 +1,28 @@
+"""Scan this repository for secrets and private infrastructure before any push. Prints variable NAMES only, never values.
+
+    python tools/secret_scan.py --env /path/to/.env [--extra-pattern REGEX ...]
+Exit code 1 on any hit."""
+import argparse, os, re, sys
+
+ap = argparse.ArgumentParser(); ap.add_argument("--env", required=True); ap.add_argument("--extra-pattern", action="append", default=[])
+a = ap.parse_args()
+root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+pairs = []
+for line in open(a.env):
+    m = re.match(r'\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)$', line)
+    if m: pairs.append((m.group(1), m.group(2).strip().strip('"\'')))
+files = [os.path.join(d, f) for d, ds, fs in os.walk(root) for f in fs
+         if not any(p in d.split(os.sep) for p in (".git", "__pycache__", ".venv", ".pytest_cache"))
+         and os.path.abspath(os.path.join(d, f)) != os.path.abspath(__file__)]
+pat = re.compile("|".join([r"/Users/", r"/root/", r"/private/tmp", r"autodl", r"hf_[A-Za-z0-9]{30,}", r"sk-[A-Za-z0-9]{20,}",
+                           r"ghp_[A-Za-z0-9]{20,}", r"AKIA[0-9A-Z]{16}", r"BEGIN [A-Z ]*PRIVATE KEY"] + a.extra_pattern), re.I)
+hits = 0
+for f in files:
+    t = open(f, errors="ignore").read(); rel = os.path.relpath(f, root)
+    for name, val in pairs:
+        if name in t: print(f"env NAME {name} in {rel}"); hits += 1
+        if len(val) >= 8 and val in t: print(f"env VALUE of {name} in {rel}"); hits += 1
+    for i, line in enumerate(t.splitlines(), 1):
+        for m in pat.finditer(line): print(f"pattern {m.group(0)!r} at {rel}:{i}"); hits += 1
+print(f"{len(pairs)} env variables, {len(files)} files, {hits} hits")
+sys.exit(1 if hits else 0)