Hugging Face's logo

tasksource
/
tasksource-decider-nano

Feature Extraction
Transformers
Safetensors
modernbert
typed-decisions
decision-index
decision-model
cross-encoder
text-embeddings-inference
Model card Files
xet
 Community
sileod commited on
Commit
03461b5
·
verified ·
1 Parent(s): 0b42846

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +11 -11
README.md CHANGED
@@ -19,9 +19,9 @@ tags:
19
  **SOTA High-Throughput Native Decision Model (\<200M Parameters)**
20
  *Decoupled Multi-Vector Late Interaction for Fast, Calibrated System-One Decisions*
21
 
22
- [![Decision Index](https://img.shields.io/badge/Decision_Index_Raw-32.03-blue)](https://github.com/sileod/decision-models) [![Decision Index Skill](https://img.shields.io/badge/Decision_Index_Skill-10.32-green)](https://github.com/sileod/decision-models) [![Parameters](https://img.shields.io/badge/Parameters-149M-orange.svg)](https://huggingface.co/answerdotai/ModernBERT-base) [![Context Length](https://img.shields.io/badge/Context-8192-purple.svg)](https://huggingface.co/answerdotai/ModernBERT-base)
23
 
24
- Single 149M-parameter checkpoint (no soup): plain grouped multi-question exposure on the v0 lineage. Previous weights scored Raw 30.65 / Skill 9.58; current weights score Raw 32.03 / Skill 10.32 on the same full official suite.
25
 
26
  Built on [`answerdotai/ModernBERT-base`](https://huggingface.co/answerdotai/ModernBERT-base) and [`lightonai/LateOn`](https://huggingface.co/lightonai/LateOn). Handles choice (K=2 to 150+), NOUL, and score decisions.
27
 
@@ -29,24 +29,24 @@ Built on [`answerdotai/ModernBERT-base`](https://huggingface.co/answerdotai/Mode
29
 
30
  ### Official Decision Index 0.2.1 (Full 100% Suite: 150,759 Requests)
31
 
32
- - **Balanced Raw Index**: **32.03 / 100**
33
- - **Balanced Skill Index (Chance-Corrected)**: **10.32 / 100**
34
  - **Breadth Skill Index**: 9.01 / 100
35
  - **Suite Coverage**: 100.0% (150,759 / 150,759 requests, 1 error)
36
 
37
  #### Area Breakdown (Official 0.2.1 Weights)
38
 
39
- - **Retrieval & Intent Classification**: 43.02% (Skill: 23.09%)
40
- - **Arts & Human Taste**: 36.90% (Skill: 13.36%)
41
- - **Language Understanding**: 35.07% (Skill: 5.22%)
42
- - **Tools & Automation**: 24.92% (Skill: 13.50%)
43
- - **Knowledge & Reasoning**: 23.63% (Skill: 2.12%)
44
 
45
  ### 🏆 Sub-200M Peer Comparison
46
 
47
  | Model | Size | Decision Index Raw | Decision Index Skill |
48
  | --- | --- | --- | --- |
49
- | **Tasksource-JEV-Nano-v1** | **149M** | **32.03** 🏆 | **10.32** 🏆 |
50
  | Tasksource-JEV-Nano-v0 | 149M | 30.65 | 9.58 |
51
  | GLiNER 2.5 base | 194M | 28.71 | 6.76 |
52
  | Julia 1 | 141M | 28.01 | 5.54 |
@@ -82,7 +82,7 @@ options = [
82
  ctx = model.encode(context, is_query=False, convert_to_tensor=True)[0]
83
  opts = model.encode(options, is_query=True, convert_to_tensor=True)
84
  scores = torch.stack([(o @ ctx.T).max(dim=1).values.sum() for o in opts])
85
- temperature = 0.3199 # learned model temperature
86
  probs = torch.softmax(scores / temperature, dim=0)
87
 
88
  for opt, p in zip(options, probs):
 
19
  **SOTA High-Throughput Native Decision Model (\<200M Parameters)**
20
  *Decoupled Multi-Vector Late Interaction for Fast, Calibrated System-One Decisions*
21
 
22
+ [![Decision Index](https://img.shields.io/badge/Decision_Index_Raw-32.09-blue)](https://github.com/sileod/decision-models) [![Decision Index Skill](https://img.shields.io/badge/Decision_Index_Skill-10.46-green)](https://github.com/sileod/decision-models) [![Parameters](https://img.shields.io/badge/Parameters-149M-orange.svg)](https://huggingface.co/answerdotai/ModernBERT-base) [![Context Length](https://img.shields.io/badge/Context-8192-purple.svg)](https://huggingface.co/answerdotai/ModernBERT-base)
23
 
24
+ Single 149M-parameter checkpoint (no soup): plain grouped multi-question exposure on the v0 lineage. Weights updated 2026-10-06 to the packdrop-s400 single checkpoint (grouped training with 50% packing dropout). Previous: Raw 32.03 / Skill 10.32 (step-4000), v0: Raw 30.65 / Skill 9.58 — same full official suite.
25
 
26
  Built on [`answerdotai/ModernBERT-base`](https://huggingface.co/answerdotai/ModernBERT-base) and [`lightonai/LateOn`](https://huggingface.co/lightonai/LateOn). Handles choice (K=2 to 150+), NOUL, and score decisions.
27
 
 
29
 
30
  ### Official Decision Index 0.2.1 (Full 100% Suite: 150,759 Requests)
31
 
32
+ - **Balanced Raw Index**: **32.09 / 100**
33
+ - **Balanced Skill Index (Chance-Corrected)**: **10.46 / 100**
34
  - **Breadth Skill Index**: 9.01 / 100
35
  - **Suite Coverage**: 100.0% (150,759 / 150,759 requests, 1 error)
36
 
37
  #### Area Breakdown (Official 0.2.1 Weights)
38
 
39
+ - **Retrieval & Intent Classification**: 43.32% (Skill: 23.49%)
40
+ - **Arts & Human Taste**: 37.66% (Skill: 14.76%)
41
+ - **Language Understanding**: 34.71% (Skill: 4.79%)
42
+ - **Tools & Automation**: 24.99% (Skill: 13.60%)
43
+ - **Knowledge & Reasoning**: 23.63% (Skill: 2.13%)
44
 
45
  ### 🏆 Sub-200M Peer Comparison
46
 
47
  | Model | Size | Decision Index Raw | Decision Index Skill |
48
  | --- | --- | --- | --- |
49
+ | **Tasksource-JEV-Nano-v1** | **149M** | **32.09** 🏆 | **10.46** 🏆 |
50
  | Tasksource-JEV-Nano-v0 | 149M | 30.65 | 9.58 |
51
  | GLiNER 2.5 base | 194M | 28.71 | 6.76 |
52
  | Julia 1 | 141M | 28.01 | 5.54 |
 
82
  ctx = model.encode(context, is_query=False, convert_to_tensor=True)[0]
83
  opts = model.encode(options, is_query=True, convert_to_tensor=True)
84
  scores = torch.stack([(o @ ctx.T).max(dim=1).values.sum() for o in opts])
85
+ temperature = 0.3168 # learned model temperature
86
  probs = torch.softmax(scores / temperature, dim=0)
87
 
88
  for opt, p in zip(options, probs):