tasksource-decider-nano submission: 0.3 public 29.05 (replaces Tasksource-JEV-Nano-v1)
#48
by sileod - opened
Single 149M checkpoint (packdrop-s400, isolated S+Qi inference), submitted via apolinario/decision-index#70.
- Model: https://huggingface.co/tasksource/tasksource-jev-nano-v1
- 0.2.1 run (skill 10.46 / raw 32.09): https://huggingface.co/datasets/tasksource/decision-index-results/tree/main/runs/official_full_packdrop_s400_isolated
- 0.3 run (skill 10.84 / raw 33.12): https://huggingface.co/datasets/tasksource/decision-index-results/tree/main/runs/official_03_packdrop_s400
- scores.json says complete:true in both run dirs; runs will be re-scored on review.
sileod changed discussion title from Tasksource-JEV-Nano-v1 submission: 0.2.1 skill 10.46, 0.3 skill 10.84 to Tasksource-JEV-Nano-v1 submission:, 0.3 skill 10.84 0.2.1 skill 10.46 (ranked #1)
sileod changed discussion title from Tasksource-JEV-Nano-v1 submission:, 0.3 skill 10.84 0.2.1 skill 10.46 (ranked #1) to Tasksource-JEV-Nano-v1 submission:, 0.3 skill 10.84 0.2.1 skill 10.46
Update: Tasksource-JEV-Nano-v1 is replaced by tasksource-decider-nano, submitted via apolinario/decision-index#120 (#70 and #76 are closed).
- Model: https://huggingface.co/tasksource/tasksource-decider-nano (tag
v3, revision6fa4fce). The same repository, renamed; the old model stays at tagv1. - 0.3 public run: 29.05 (raw 46.41), complete, 140,171
ok, 7unsupported, no truncation: https://huggingface.co/datasets/tasksource/decision-index-results/tree/5853dd01c0cebaf96794b6f8c491f277e743e9c1/runs/tasksource-decider-nano - 150M joint cross-encoder fine-tuned from
cross-encoder/ettin-reranker-150m-v1; the engine ships in the model repo (decision_index_engine:DeciderEngine).
sileod changed discussion title from Tasksource-JEV-Nano-v1 submission:, 0.3 skill 10.84 0.2.1 skill 10.46 to tasksource-decider-nano submission: 0.3 public 29.05 (replaces Tasksource-JEV-Nano-v1)
sileod changed discussion status to closed