CommonLID
Common Crawl's language identification benchmark, sampled from real-world web text and human-validated across hundreds of language varieties.
Reference • License: common-crawl-tou • Main score: macro_f1
commonlid — sorted by Macro F1
Model | Macro F1 | Micro F1 | Mean FPR (%) | Languages | Samples/s |
|---|---|---|---|---|---|
OpenLID-v2 | 60.4 | 70.4 | 0.07 | 1456 | 60383.4 |
- Model — Identifier of the language identification model.
- Macro F1 — Unweighted mean of per-language F1 (x100), averaged over languages with at least one gold sample in this dataset (paper / gold-only definition). Higher is better. This is the headline ranking column.
- Micro F1 — Sample-weighted F1 (x100): pooled correct / pooled predictions across all gold samples. Less affected by rare languages than macro F1. Higher is better.
- Mean FPR (%) — Mean per-language false-positive rate (paper-style): how often the model labels a non-target sentence as the target language. Lower is better.
- Languages — Number of distinct languages the model emitted on this dataset (
set(gold) | set(pred)). Reflects the model's output vocabulary on this test, not the gold language count. - Samples/s — Throughput during evaluation (samples processed per second). Hardware-dependent; useful for relative comparison only.
Click a row to load per-language metrics.
Language | F1 | Precision | Recall | FPR (%) | GT | Predictions | Correct |
|---|---|---|---|---|---|---|---|
- Language — ISO 639-3 code of the gold and/or predicted language.
- F1 — Per-language F1 score (x100). Harmonic mean of precision and recall.
- Precision — Per-language precision (x100) = correct / predictions for this language. How often the model is right when it predicts this language.
- Recall — Per-language recall (x100) = correct / gold-count for this language. How much of this language's gold set the model recovers.
- FPR (%) — Paper-style false-positive rate: FP / (FP + TN_correct_other). Counts how often samples in other languages are misclassified as this one.
- GT — Gold-truth sample count for this language.
- Predictions — Number of times the model predicted this language.
- Correct — Predictions that match the gold label.
CommonLID (nano)
Common Crawl's language identification benchmark, sampled from real-world web text and human-validated across hundreds of language varieties. Nano slice — stratified sample (max 1000 + min 5 per language) of the parent benchmark, with the schema normalised to (index, text, language_iso639_3).
Reference • License: common-crawl-tou • Main score: macro_f1
commonlid_nano — sorted by Macro F1
Model | Macro F1 | Micro F1 | Mean FPR (%) | Languages | Samples/s |
|---|---|---|---|---|---|
GPT-4o-mini | 72.5 | 77.3 | 0.24 | 130 | 358120.3 |
- Model — Identifier of the language identification model.
- Macro F1 — Unweighted mean of per-language F1 (x100), averaged over languages with at least one gold sample in this dataset (paper / gold-only definition). Higher is better. This is the headline ranking column.
- Micro F1 — Sample-weighted F1 (x100): pooled correct / pooled predictions across all gold samples. Less affected by rare languages than macro F1. Higher is better.
- Mean FPR (%) — Mean per-language false-positive rate (paper-style): how often the model labels a non-target sentence as the target language. Lower is better.
- Languages — Number of distinct languages the model emitted on this dataset (
set(gold) | set(pred)). Reflects the model's output vocabulary on this test, not the gold language count. - Samples/s — Throughput during evaluation (samples processed per second). Hardware-dependent; useful for relative comparison only.
Click a row to load per-language metrics.
Language | F1 | Precision | Recall | FPR (%) | GT | Predictions | Correct |
|---|---|---|---|---|---|---|---|
- Language — ISO 639-3 code of the gold and/or predicted language.
- F1 — Per-language F1 score (x100). Harmonic mean of precision and recall.
- Precision — Per-language precision (x100) = correct / predictions for this language. How often the model is right when it predicts this language.
- Recall — Per-language recall (x100) = correct / gold-count for this language. How much of this language's gold set the model recovers.
- FPR (%) — Paper-style false-positive rate: FP / (FP + TN_correct_other). Counts how often samples in other languages are misclassified as this one.
- GT — Gold-truth sample count for this language.
- Predictions — Number of times the model predicted this language.
- Correct — Predictions that match the gold label.
Source: commoncrawl/commonlid-results @ HEAD.