---
library_name: transformers
tags:
  - code
  - coding-assistant
  - indian-languages
  - hindi
  - multilingual
  - qwen2
  - lora
  - fine-tuned
  - indian-history
  - indic-nlp
license: apache-2.0
language:
  - en
  - hi
  - bn
  - ta
  - te
  - mr
  - gu
  - kn
  - ml
  - pa
  - or
base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
pipeline_tag: text-generation
---

# 🇮🇳 Indra — Indian Language AI Assistant

**Indra** is a fine-tuned LLM built on **Qwen2.5-Coder-1.5B-Instruct**, trained on Indian language and history datasets. It can converse in **10 Indian languages**, answer questions about **Indian history & culture**, and still write code.

## ✨ Highlights

- 🗣️ **10 Indian languages** — Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia
- 📜 **Indian history & culture** — From Indus Valley to modern India
- 🔄 **Bilingual conversations** — Hinglish, code-switching, vernacular queries
- 💻 **Coding preserved** — Still writes Python, JavaScript, and full-stack code
- 📦 **Lightweight** — 1.5B parameters, runs on consumer GPUs

## Quick Start

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("RockySinghRajput/Indra", torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("RockySinghRajput/Indra")

messages = [
    {"role": "system", "content": "आप Indra हैं, एक बुद्धिमान AI सहायक।"},
    {"role": "user", "content": "भारत के स्वतंत्रता संग्राम के बारे में बताइए।"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

output = model.generate(inputs.input_ids, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tokenizer.decode(output[0][len(inputs.input_ids[0]):], skip_special_tokens=True))
```

### Hinglish Example

```python
messages = [
    {"role": "system", "content": "You are Indra, a helpful AI that understands Hindi and English."},
    {"role": "user", "content": "Mujhe Python mein ek calculator banana hai, kaise karoon?"}
]
```

## Model Details

| Property | Value |
|----------|-------|
| **Base Model** | [Qwen/Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct) |
| **Parameters** | 1.5B |
| **Type** | Causal Language Model (merged LoRA fine-tune) |
| **Languages** | English, Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia |
| **License** | Apache 2.0 |
| **Developed by** | [RockySinghRajput](https://huggingface.co/RockySinghRajput) |
| **Related Model** | [IndraCoder](https://huggingface.co/RockySinghRajput/Indracoder) (coding-focused version) |

## Supported Languages

| Language | Script | Code |
|----------|--------|------|
| Hindi | देवनागरी | hi |
| Bengali | বাংলা | bn |
| Tamil | தமிழ் | ta |
| Telugu | తెలుగు | te |
| Marathi | मराठी | mr |
| Gujarati | ગુજરાતી | gu |
| Kannada | ಕನ್ನಡ | kn |
| Malayalam | മലയാളം | ml |
| Punjabi | ਪੰਜਾਬੀ | pa |
| Odia | ଓଡ଼ିଆ | or |
| English | Latin | en |

## Training Details

### Training Data

Fine-tuned on **6 Indian language & culture datasets**:

| Dataset | Purpose | Content |
|---------|---------|---------|
| [CohereForAI/aya_dataset](https://huggingface.co/datasets/CohereForAI/aya_dataset) | Multilingual Indian conversations | 10 Indian languages |
| [Cognitive-Lab/Hindi-Instruct](https://huggingface.co/datasets/Cognitive-Lab/Hindi-Instruct) | Hindi instruction following | Hindi Q&A |
| [sarvamai/samvaad-hi-en-instruct-v2](https://huggingface.co/datasets/sarvamai/samvaad-hi-en-instruct-v2) | Bilingual conversations | Hindi-English |
| CohereForAI/aya_collection (India-filtered) | Indian history & culture | History, heritage, knowledge |
| CohereForAI/aya_collection (Hindi WikiQA) | Hindi knowledge base | Wikipedia-sourced Hindi QA |
| ai4bharat/IndicSentiment | Hindi language understanding | Sentiment analysis |

### Indian History Coverage

The model has been trained on Indian history spanning:

- **Ancient India** — Indus Valley Civilization, Vedic period, Maurya & Gupta Empires
- **Medieval India** — Delhi Sultanate, Mughal Empire, Vijayanagara, Maratha Empire, Bhakti & Sufi movements
- **Modern India** — British Raj, Freedom struggle, Independence, Republic
- **Indian Constitution** — Fundamental rights, governance, democracy
- **Culture & Heritage** — Art, architecture, literature, philosophy, classical music, Ayurveda, Yoga

### Training Procedure

- **Method**: LoRA (Low-Rank Adaptation) → merged into base model
- **LoRA Config**: r=16, alpha=16, dropout=0.05
- **Target Modules**: q_proj, k_proj, v_proj, o_proj
- **Epochs**: 2
- **Learning Rate**: 2e-5 (lower to preserve base capabilities)
- **Optimizer**: paged_adamw_8bit
- **Sequence Length**: 512 tokens
- **Precision**: FP16 mixed precision
- **Quantization**: 4-bit NF4 (QLoRA) during training

### Compute Infrastructure

- **Hardware**: NVIDIA T4 GPU
- **Training Time**: ~2 hours

## Capabilities

### ✅ What Indra Can Do

- **Converse in Indian languages** — Answer questions in Hindi, Bengali, Tamil, Telugu, and more
- **Indian history & culture** — Detailed knowledge of Indian civilization
- **Hinglish/bilingual** — Handle mixed Hindi-English naturally
- **General knowledge** — Science, geography, current affairs with Indian context
- **Coding** — Write code in Python, JavaScript, and other languages
- **Sentiment analysis** — Understand sentiment in Hindi text

### ⚠️ Limitations

- **1.5B model** — Smaller than commercial LLMs; may produce shorter or less nuanced responses
- **Script limitations** — Stronger in Hindi/Devanagari; other Indian scripts may have lower quality
- **Not a translator** — Optimized for conversation, not professional translation
- **May hallucinate** — Always verify historical facts and generated content
- **English-centric base** — Indian language abilities are fine-tuned on top of an English-dominant base

### ❌ Out-of-Scope Use

- Professional translation services
- Legal or medical advice
- Factual source of record for academic research
- Generating harmful or culturally insensitive content

## Evaluation

| Test | Language | Task | Result |
|------|----------|------|--------|
| Hindi Chat | Hindi | Gandhi's role in freedom struggle | ✅ Detailed response |
| Indian History | English | Gupta Empire Golden Age | ✅ Accurate overview |
| Hinglish | Mixed | "Python mein calculator banana hai" | ✅ Code + Hindi explanation |
| Coding | English | Binary search implementation | ✅ Working code |
| Indian Culture | Hindi | Classical music ragas | ✅ Cultural knowledge |

## Model Family

| Model | Focus | Repo |
|-------|-------|------|
| **Indra** (this model) | Indian languages + history | [RockySinghRajput/Indra](https://huggingface.co/RockySinghRajput/Indra) |
| **IndraCoder** | Coding + debugging | [RockySinghRajput/Indracoder](https://huggingface.co/RockySinghRajput/Indracoder) |
| **IndraCoder-7B** | Advanced coding + chat (coming soon) | [RockySinghRajput/IndraCoder-7B](https://huggingface.co/RockySinghRajput/IndraCoder-7B) |

## Citation

```bibtex
@misc{indra2025,
  title={Indra: An Indian Language AI Assistant},
  author={RockySinghRajput},
  year={2025},
  publisher={HuggingFace},
  url={https://huggingface.co/RockySinghRajput/Indra}
}
```

## Contact

- **HuggingFace**: [RockySinghRajput](https://huggingface.co/RockySinghRajput)