Commit History
-
Upload LICENSE.txt
2f21748 -
add openLID license info
4f197c1 -
Rename original/openLID/lid201-model.bin to openLID/lid201-model.bin
043214f -
Rename original/lid201-model.bin to original/openLID/lid201-model.bin
c3b4362
-
Upload 3 files
add sha384 hashes for each model.4f8b035 -
Update README.md
7c79061 -
Upload lid.176.ftz
https://fasttext.cc/docs/en/language-identification.html lid.176.ftz, which is the compressed version of the model, with a file size of 917kB license : Creative Commons Attribution-Share-Alike License 3.0 sha384 : 574c310c8d82540a8765d28140bf3e6cd01ceb7d164c9ec205c518254bfa0333ae1b80aab7ceff56e7159eced98a3d37 filesize : 938013adb716a
-
Update README.md
4b235de -
Create lid201-model.bin
"An Open Dataset and Model for Language Identification" (Burchell et al., ACL 2023). The OpenLID fasttext LID model can be found at https://data.statmt.org/lid/lid201-model.bin.gz. To use it, install FastText, decompress the model, then run fasttext predict lid201-model.bin $DATA > output.fasttext. The model is licensed under the GNU General Public License v3.0. This is the unquantized, original model. The quantized model is available from this repository under quantized/b442fa7 -
Create lid218e.ftz
Original model available in https://huggingface.co/facebook/fasttext-language-identification This LID (Language IDentification) model is used to predict the language of the input text, and the hosted version (lid218e) was released as part of the NLLB project and can detect 217 languages. You can find older versions (ones that can identify 157 languages) on the official fastText website. licensed cc-by-nc-4.0 This is the quantized version of the above model.ba92a11 -
Create lid201-model.ftz
originally lid201-model.bin available from https://github.com/laurieburchell/open-lid-dataset The original model is licensed under the GNU General Public License v3.0811ae3f -
Update README.md
2211481 -
initial commit
92e7094