Fine-tune for Indian address parsing to structured JSON

#31
by abr-6024 - opened

I fine-tuned Qwen2.5-1.5B-Instruct (LoRA, merged) to parse messy Indian addresses into JSON (flat, building, street, landmark, area, city, state, pincode, phone).

On a held-out synthetic test set (500 addresses), full-record exact match went from 1.4% (base) to 98.0%.
Note: results are on synthetic data only; real-world accuracy is not yet measured.

Model: https://huggingface.co/abr-6024/indian-address-parser-1.5B
Dataset + generator: https://huggingface.co/datasets/abr-6024/indian-address-parsing-synthetic

Thanks to the Qwen team for the base model. Feedback and real-address test data welcome.

Update: v3 is now the main version of this model, and please disregard the 98.0% figure above.

That 98.0% was measured on synthetic text only, and it did not hold up on addresses I wrote by hand (about 58% on a stress set). v3 is trained on addresses whose locality, pincode, district and state come from the India Post pincode directory (Tamil Nadu is 44% of the data).

Results for v3:

  • Held-out synthetic test (800 addresses): 94.4% full-record exact match; 93.0% on six districts never seen in training
  • Hand-written Tamil Nadu set: 29/30 (written by me with the generator's conventions, so not an independent benchmark)
  • Hand-written mixed-India set: 80.6% (part of the gap is that v3 uses city = district)
  • Not measured on real customer addresses yet, so expect lower accuracy there

Model: https://huggingface.co/abr-6024/indian-address-parser-1.5B
Dataset + generator: https://huggingface.co/datasets/abr-6024/indian-address-parsing-synthetic

Anonymized real-address test cases, or feedback on the conventions, are welcome.

Sign up or log in to comment