Spaces:
Runtime error
Download README.md from build-small-hackathon/quran-live-stt-cpu: direct link, hf CLI and curl.
- Browser
- Download file 10.1 kB
-
https://huggingface.co/spaces/build-small-hackathon/quran-live-stt-cpu/resolve/main/README.md
- Command line
-
hf download hf://spaces/build-small-hackathon/quran-live-stt-cpu/README.md
-
curl -L -o README.md https://huggingface.co/spaces/build-small-hackathon/quran-live-stt-cpu/resolve/main/README.md
A newer version of the Gradio SDK is available: 6.32.0
title: Quran Stt
emoji: ๐
colorFrom: pink
colorTo: purple
sdk: gradio
sdk_version: 6.16.0
python_version: '3.13'
app_file: app.py
pinned: false
license: mit
short_description: Real-time Quran recitation with FastConformer ASR
tags:
- track:backyard
- track:wood
- sponsor:nvidia
- achievement:offgrid
- achievement:welltuned
- achievement:offbrand
- achievement:sharing
- achievement:fieldnotes
๐๏ธ Quran Speech-to-Text
Real-time Quranic recitation transcription powered by a fine-tuned FastConformer model, achieving state-of-the-art accuracy on Arabic Quranic text with full diacritics.
๐น Watch Demo Video
๐ฑ Social Media Posts:
๐ฌ Model: fastconformer-quran-ar
This application uses a custom FastConformer model based on NVIDIA's architecture, fully fine-tuned on Quranic recitations from the tarteel-ai/everyayah dataset, which contains recitations from dozens of reciters across all 114 Surahs of the Quran.
Model Link: mohammed/fastconformer-quran-ar
๐ Performance Metrics
- Word Error Rate (WER): 0.14%
- This means the model correctly recognizes 999 out of every 1000 words โ with complete diacritization (tashkeel)
- ๐ Ranked #2 on the Quranic ASR Leaderboard
- State-of-the-art accuracy for Quranic recitation transcription
โ๏ธ Training Methodology
The model was trained using a progressive unfreezing approach across five phases on an NVIDIA RTX 4070 Ti (12GB):
| Phase | Description | Duration | WER |
|---|---|---|---|
| Phase 1 | Top 3 encoder layers | 6h 5m | 15.66% |
| Phase 2a | Encoder layers 9-17 | 1h 15m | 14.49% |
| Phase 2b | Upper half encoder | 10h 22m | 0.18% |
| Phase 3a | Full model finetune | 2h 36m | 8.71% |
| Phase 3b | Full model (final) | 4h 39m | 0.14% โ |
Total Training Time: ~25 hours WER Improvement: 99.1% reduction (from 15.66% โ 0.14%)
This progressive approach allows for efficient training while achieving maximum accuracy by gradually unfreezing more layers.
๐ฏ Features
The model supports two operational modes:
- Offline Transcription: Full transcription with maximum accuracy
- Streaming Mode: Real-time transcription with low latency using cache-aware inference โ perfect for live recitation correction applications
๐ Built for Build Small Hackathon
This project is part of the Hugging Face Build Small Hackathon, focusing on practical AI solutions using efficient models.
๐ก Use Cases
- Real-time Quran recitation correction
- Quranic memorization assistance (Hifz)
- Tajweed learning applications
- Accessible Quran study tools
- Quranic audio transcription and indexing
๐ ๏ธ Technical Stack
- Model Architecture: FastConformer (NVIDIA NeMo)
- Framework: NeMo ASR
- Interface: Gradio
- Deployment: Hugging Face Spaces
๐ Technical Implementation: Recitation Detection & Error Identification
This application implements a sophisticated real-time recitation verification system that goes beyond simple transcription. Here's how it works:
1. Quran Database & Text Normalization
Database Initialization (app.py:70-89)
- Loads all 6,236 verses from the Quran using the quran-api
- Maintains both original text (with full diacritics) and normalized text (without diacritics) for flexible matching
Arabic Text Normalization (app.py:48-59)
# Normalization process:
1. Remove all diacritics (tashkeel) - แธฅarakฤt, tanwฤซn, shaddah, etc.
2. Remove superscript alif (alif khanjariyah)
3. Normalize alif variations (ุฅ, ุฃ, ุข, ุง โ ุง)
4. Normalize taa marbouta (ุฉ โ ู)
This normalization enables accurate matching despite variations in pronunciation or reciter style while preserving the original diacritized text for display.
2. Intelligent Anchor Detection
Initial Verse Detection (app.py:109-162)
When a user starts reciting, the system must identify which verse they're reciting. This uses a position-based longest common subsequence algorithm:
- Algorithm:
difflib.SequenceMatcherfinds the longest consecutive word match between transcribed audio and all verses in the database - Dynamic Thresholds based on recitation length:
- Short (3-5 words): Requires 100% match from verse beginning
- Medium (6-8 words): Requires โฅ85% coverage
- Long (9+ words): Requires โฅ6 consecutive words with โฅ75% coverage
- Position Preference: Prioritizes matches that start at the beginning of both the verse and the recitation
- Stop-word Filtering: Ignores common Arabic particles (ุงูุ ู ูุ ู ุงุ ูุงุ ููุ ุนูู) to prevent false positives
Example: If user recites "ุจุณู ุงููู ุงูุฑุญู ู ุงูุฑุญูู ", the system:
- Normalizes to "ุจุณู ุงููู ุงูุฑุญู ู ุงูุฑุญูู " (removes diacritics)
- Searches 6,236 verses for longest match
- Finds perfect 4-word match at start of Al-Fatiha (1:1)
- Sets anchor โ expects verse 1:1
3. Real-Time Token-by-Token Alignment
Word-Level Error Detection (app.py:164-203)
Once anchored, the system performs streaming word-level comparison:
# Alignment process using edit distance opcodes:
1. Split canonical verse into tokens: ["ุจูุณูู
ู", "ูฑูููููู", "ูฑูุฑููุญูู
ููฐูู", "ูฑูุฑููุญููู
ู"]
2. Split transcription into tokens: ["ุจุณู
", "ุงููู", "ุงูุฑุญู
ู", "ุงูุฑุญูู
"]
3. Normalize both for comparison
4. Use SequenceMatcher.get_opcodes() to detect:
- 'equal': Correctly recited word โ green highlighting
- 'replace': Wrong word โ red highlighting with underline
- 'delete': Skipped word โ red highlighting
- 'insert': Extra word added โ ignored in canonical display
Visual Feedback:
- โ
Correct words: Green text (
<span class='correct-word'>) - โ Wrong words: Red bold underlined text (
<span class='error-word'>) - โ Completely wrong verse: Red double-underlined (
<span class='all-error-verse'>)
Streaming Display Constraint (app.py:192):
- During active recitation: Only shows correctly matched words (prevents showing unrecited text)
- After pause/completion: Shows all words including errors for review
4. Verse Completion & Auto-Advancement
Intelligent Verse Boundary Detection (app.py:332-348)
The system determines when a verse is complete using two methods:
Method 1: Token Exhaustion
canonical_tokens = "ุจูุณูู
ู ูฑูููููู ูฑูุฑููุญูู
ููฐูู ูฑูุฑููุญููู
ู".split() # 4 tokens
speech_tokens = transcribed_text.split()
is_completed = (last_matched_index >= len(canonical_tokens))
Method 2: Next-Verse Transition Detection
# If user starts reciting next verse while current verse incomplete:
1. Extract leftover tokens after last match
2. Compare with beginning of next verse (verse N+1)
3. If similarity โฅ75% โ mark current verse complete, auto-advance
Upon Completion:
- Finalizes current verse with ayah number marker
๏ดฟูก๏ดพ - Increments expected verse index
- Carries over leftover tokens to next verse
- If leftover tokens don't match next verse โ marks as error
5. Continuous Streaming Pipeline
Real-Time Audio Processing (app.py:222-415)
Audio Stream โ Accumulation โ ASR Transcription โ Alignment โ Display
โ โ โ โ โ
16kHz WAV 0.5s minimum FastConformer SequenceMatcher HTML
buffer (streaming) (live tracking) update
Key Features:
- Silence Detection: RMS < 0.005 triggers pause (2 chunks = ~0.4s)
- Streaming ASR: Runs transcription every 0.5+ seconds on accumulated audio
- Live Preview: Shows real-time alignment even before pause
- Commit on Pause: Finalizes verse detection only during valid pauses
- State Persistence: Maintains verse context, partial recitation, and history across chunks
6. Error Classification
The system distinguishes three error types:
| Error Type | Detection Method | Visual Display |
|---|---|---|
| Wrong Word | Token mismatch via replace opcode |
Red underlined word |
| Skipped Word | Missing token via delete opcode |
Red underlined word in canonical position |
| Wrong Verse | No anchor found or <75% next-verse match | Entire verse in red double-underline |
Performance Characteristics
- Latency: ~200-500ms from speech to visual feedback (depending on chunk size)
- Memory: Streams audio in chunks; clears after each pause
- Accuracy: Inherits 0.14% WER from base model, enhanced by verse-constrained matching
- Robustness: Handles pronunciation variations through normalization and fuzzy matching
This architecture enables the application to function as an intelligent Quran tutor that not only transcribes but actively guides users through sequential memorization with immediate, precise feedback.
๐ License
MIT License