news: add tasksource-decider-nano (trained)
Adds one trained News entry for tasksource-decider-nano, a 150M joint cross-encoder (fine-tuned from cross-encoder/ettin-reranker-150m-v1). Only news.html changes: one item at the top of ITEMS; node check.js passes.
Decision Index 0.3 public run: 29.05 (raw 46.41), complete: 140,171 ok, 7 unsupported (pairs above 8,192 tokens), no errors, no truncation. Median 73 ms per request on one A10, one request at a time.
- Model (Apache-2.0): tasksource/tasksource-decider-nano @ v3
- Results (compact, re-scorable): decision-index-results/runs/tasksource-decider-nano
- Evaluation PR: https://github.com/apolinario/decision-index/pull/120
Reproduce (kit 9eb2dbe; the engine ships in the model repo, PyTorch + transformers only):
hf download tasksource/tasksource-decider-nano --revision v3 --local-dir decider && cd decider
python -m decision_index pipeline --engine decision_index_engine:DeciderEngine --option repo=. --edition 0.3 --out runs/tasksource-decider-nano
Training data: tasksource-jev-typed-decisions, synthetic-typed-decisions, and train splits of public datasets (no evaluation splits); every training request passed an exact and 6-gram firewall against the 0.3 suite. Replaces the closed #43 and discussion #48.
cc @multimodalart , thanks for running the board!
Updated to v4, the same recipe trained for 48,000 steps on 480,000 requests: 30.90 public on 0.3 (complete, 140,171 ok, 7 unsupported, no truncation), median 52 ms on one A10.
- Model: tasksource/tasksource-decider-nano @ v4
- Results: runs/tasksource-decider-nano
Reproduce with --revision v4 in the commands above.
Updated to v5, trained for 64,000 steps on an updated mix: 33.51 on the 0.3 public suite, which 0.4 keeps (v4: 30.90). The run is complete: 140,171 ok, 7 unsupported, no truncation. Median 23 ms on one shared A30.
- Model: tasksource/tasksource-decider-nano @ v5
- Board submission: https://github.com/apolinario/decision-index/pull/263
The entry is rebuilt on current main (after the 0.4 note), so the diff is still one item in news.html; node check.js passes.