Tsengel-mn/nllb-mongolian

🇲🇳 NLLB-600M Mongolian v2.0 (Major Release)

A state-of-the-art bidirectional translation model for Mongolian (Cyrillic) ↔ English, fine-tuned from Meta's NLLB-200 (600M Distilled).

Version 2.0 is a complete overhaul trained on over 711,800 parallel sentence pairs with dedicated native idiomatic and technical anchor injection. It completely eliminates the Russian loanword bias, broken literal calques, and hallucinated grammar common in older or baseline models.

---

🚀 Key Improvements in v2.0

English Input	Baseline / Argos v1.1	NLLB-600M Mongolian v2.0	Linguistic Note
"Where is the hotel?"	Хотель хаана байна?	Зочид буудал хаана байна вэ?	Russian loanword eliminated
"Good morning, how are you today?"	Сайн өглөө, өнөөдөр яаж байна?	Өглөөний мэнд, та сайн байна уу?	Authentic greeting idiom
"The server ran out of memory and crashed."	Сервер дурсамжгүй болж сүйрсэн	Санах ой дууссан тул сервер гацсан.	Correct IT terminology (RAM = санах ой)
"We need to fix the bugs in the application."	...алдаанууд засах шаардлагатай	Бид програмын алдааг засах хэрэгтэй.	Natural syntax and modal auxiliary
"Can you help me assemble the robotic arm?"	Надад туслах уу угсрах...	Робот гарыг угсрахад надад туслахгүй юу?	Polite Mongolian request suffix (-хгүй юу?)
"Database migration completed successfully."	Өгөгдлийн сангийн шилжилт...	Мэдээллийн сангийн нүүлгэн шилжилт амжилттай дууссан.	Industry-standard tech term

---

💻 Quick Start (Python / Transformers)

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_id = "Tsengel-mn/nllb-mongolian"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id).to("cuda" if torch.cuda.is_available() else "cpu")

def translate(text: str, direction: str = "en-mn") -> str:
    if direction == "en-mn":
        src, tgt = "eng_Latn", "khk_Cyrl"
    else:
        src, tgt = "khk_Cyrl", "eng_Latn"
        
    tokenizer.src_lang = src
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    
    tgt_id = tokenizer.convert_tokens_to_ids(tgt)
    with torch.no_grad():
        tokens = model.generate(
            **inputs,
            forced_bos_token_id=tgt_id,
            max_length=128,
            num_beams=2
        )
    return tokenizer.decode(tokens[0], skip_special_tokens=True)

# English -> Mongolian
print(translate("The database migration completed successfully.", "en-mn"))
# Output: Мэдээллийн сангийн нүүлгэн шилжилт амжилттай дууссан.

# Mongolian -> English
print(translate("Өглөөний мэнд, та сайн байна уу?", "mn-en"))
# Output: Good morning, how are you today?

---

🛠️ Training Specifications

• Base Architecture: Meta NLLB-200 Distilled (600M parameters, Seq2Seq Encoder-Decoder).
• Training Corpus: 711,821 pairs:
  • 703,221 verified clean parallel sentences.
  • Curated idiomatic, conversational, and technical Mongolian anchor sets oversampled 200x.
• Hardware: NVIDIA GeForce RTX 4060 (8GB VRAM) on AMD Ryzen 9 7900.
• Fine-Tuning Method: PEFT LoRA (Rank 64, Alpha 128 across all attention projections and MLP feed-forwards: q_proj, k_proj, v_proj, out_proj, fc1, fc2).
• Optimization: AdamW 8-bit (adamw_bnb_8bit), Cosine Learning Rate schedule with 300 warmup steps, BF16 mixed precision.
• Training Duration: 8,000 steps (~2.25 hours).
• Final Validation Loss: 0.9372.
• Export Format: Standalone fused FP16/BF16 weights (model.safetensors).

---

🌐 Language Codes (NLLB Flores-200)

When calling the tokenizer, use standard NLLB language identifiers:

• Mongolian (Cyrillic): khk_Cyrl
• English: eng_Latn

---

📜 License & Citation

This model is licensed under Meta's Creative Commons Attribution-NonCommercial 4.0 International (CC-BY-NC 4.0) license in alignment with the underlying NLLB-200 weights.

Maintained by Tsengel-mn for the Mongolian open-source AI language ecosystem.