Instructions to use mohamedrayyan/gemma-e4b-dhivehi-lora-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mohamedrayyan/gemma-e4b-dhivehi-lora-v1 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("google/gemma-4-E4B-it")
model = PeftModel.from_pretrained(base_model, "mohamedrayyan/gemma-e4b-dhivehi-lora-v1")

Transformers

How to use mohamedrayyan/gemma-e4b-dhivehi-lora-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mohamedrayyan/gemma-e4b-dhivehi-lora-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mohamedrayyan/gemma-e4b-dhivehi-lora-v1", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mohamedrayyan/gemma-e4b-dhivehi-lora-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mohamedrayyan/gemma-e4b-dhivehi-lora-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mohamedrayyan/gemma-e4b-dhivehi-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mohamedrayyan/gemma-e4b-dhivehi-lora-v1

SGLang

How to use mohamedrayyan/gemma-e4b-dhivehi-lora-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mohamedrayyan/gemma-e4b-dhivehi-lora-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mohamedrayyan/gemma-e4b-dhivehi-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mohamedrayyan/gemma-e4b-dhivehi-lora-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mohamedrayyan/gemma-e4b-dhivehi-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use mohamedrayyan/gemma-e4b-dhivehi-lora-v1 with Docker Model Runner:
```
docker model run hf.co/mohamedrayyan/gemma-e4b-dhivehi-lora-v1
```

Gemma E4B Dhivehi LoRA Pilot

This is a private pilot LoRA adapter for google/gemma-4-E4B-it, trained for early Dhivehi instruction-following experiments. It is not a production model.

Intended Scope

The adapter is intended for small experiments around:

Dhivehi chat in Thaana script.
Polite rewriting.
Grammar/spelling correction.
Short summarization.
Simple translation checks.

It should not be used for legal, medical, financial, emergency, official, or high-stakes advice.

Training Data

Training data is synthetic and stored in the private dataset repo mohamedrayyan/dhivehi-synthetic-v1.

Current adapter run used:

Train file: scale-combined-v1/train.jsonl
Held-out file: scale-combined-v1/heldout.jsonl
Train records: 64,679
Held-out records: 1,420
Approx train tokens: 10,039,638

scale-combined-v1 combines the earlier pilot split with the first larger controlled synthetic scale run. It includes Dhivehi rewrite, grammar correction, summary, chat, and English-to-Dhivehi translation rows.

Training

The current adapter was trained on Modal H100 with PEFT LoRA.

Important implementation detail: Gemma E4B is multimodal. LoRA target modules must be restricted to exact model.language_model modules. A previous smoke run accidentally targeted audio/vision tower internals; that adapter was superseded.

Corrected adapter properties:

LoRA target modules: 258 exact model.language_model linear modules.
Adapter parameters: 34,881,536.
Audio tower LoRA tensors: 0.
Vision tower LoRA tensors: 0.
Adapter size: about 140 MB.

Current 500-step scale-combined training:

Metric	Value
Train loss	0.1696
Final trainer eval loss	0.1593
Eval token accuracy	0.9442
Train runtime	6,270.09 seconds
Epoch fraction	0.1237

Preserved tags:

quality-fixed-smoke-20260531: corrected quality-only adapter.
controlled-batch-1-sft-20260531: controlled-only adapter, useful for comparison but overfit to narrow templates.
pilot-combined-v1-sft-20260531: earlier combined pilot adapter.
scale-combined-v1-sft-20260602: current scale adapter.

Evaluation

Base-vs-adapter eval artifacts are uploaded in this model repo under:

eval/scale-combined-v1-sft-eval-1780422776/summary.json
eval/scale-combined-v1-sft-eval-1780422776/outputs.jsonl
eval/pilot-combined-v1-eval-smoke-1780254244/summary.json
eval/pilot-combined-v1-eval-smoke-1780254244/outputs.jsonl

Current scale fixed eval, 12 prompts:

Metric	Base Gemma E4B	Scale adapter
Held-out loss	2.3896	0.1744
Exact generation matches	n/a	0 / 12
Avg output chars	230.3333	133.25
Avg Thaana ratio	0.4639	0.7433
Avg Latin ratio	0.3150	0.1038
Empty outputs	0	0

Qualitative notes:

The adapter gives shorter, more Dhivehi-heavy outputs than the base model.
English-to-Dhivehi translation has more training support than the earlier pilot, but still needs native-speaker review.
Dhivehi-to-English translation can be correct on simple examples, but has very little training support.
Chat/rewrite outputs can repeat or preserve prompt wording too closely.
A separate 20-step DPO smoke run was completed in mohamedrayyan/gemma-e4b-dhivehi-dpo-v1, but it did not beat this SFT adapter on the current smoke eval.

Limitations

Synthetic training set with repeated local templates.
No native-speaker rating pass yet.
Translation quality is not stable.
The adapter may overfit to short school/homework-style rewriting and correction patterns.
The model may hallucinate details or produce mixed-script text.
This is a pilot adapter, not a general Dhivehi assistant.

Next Gate

Before running another DPO/ORPO pass:

Run a native-speaker review on samples from scale-combined-v1-sft-20260602.
Add broader local templates and longer source paragraphs before spending more data budget.
Separately curate translation and chat data.
Build at least 1K higher-quality preference pairs from real SFT failure cases.

Downloads last month: 2

Model tree for mohamedrayyan/gemma-e4b-dhivehi-lora-v1

Base model

google/gemma-4-E4B

Finetuned

google/gemma-4-E4B-it

Adapter

(277)

this model

mohamedrayyan
/

gemma-e4b-dhivehi-lora-v1