🔨 ForgeRL — Multi-Agent Software Engineering RL Environment
An OpenEnv-compatible reinforcement learning environment where an LLM meta-agent learns to orchestrate 9 specialized sub-agents to autonomously build working software from natural-language specifications.
Meta PyTorch × HuggingFace OpenEnv Hackathon 2026
Reset the environment, then step through the SDLC workflow one action at a time.
1 · Reset
Difficulty Tier (1=simple CRUD → 5=distributed systems)
2 · Take Action
Orchestration Action
Episode History
Action | Reward | Phase |
|---|---|
Full Episode — Heuristic Policy
Runs a complete SDLC episode using the optimal heuristic workflow:
Intake → Architect → Planner → [QA → Coder × N] → Oversight → Security → Finalize
No LLM key required — runs in simulated mode so sub-agents return deterministic outputs instantly.
Difficulty Tier
ForgeRL Architecture
Stack
| Layer | Component | Role |
|---|---|---|
| Environment | ForgeEnvironment |
OpenEnv reset() / step() / state |
| Sub-Agents | 9 specialists | Intake · Architect · Planner · QA · Coder · Recovery · Security · Oversight · Reviewer |
| Rewards | 11 components | Dense shaping + sparse terminal |
| Curriculum | 5 tiers | Auto-promotes at 70% success, demotes at 25% |
Episode FSM
IDLE
└→ intake → specification → architecture → planning
└→ execution
├→ task_qa → task_code ← task_recovery
└→ security_audit → done
Reward Components
| Component | Type | Value |
|---|---|---|
| Phase transition | Dense | +0.5 |
| Task completion | Dense | +2.0 |
| Recovery success | Dense | +1.0 |
| Oversight bonus | Dense | +0.5 |
| Valid delegation | Dense | +0.1 |
| Step cost | Dense | −0.01 |
| Invalid action | Dense | −1.0 |
| Test pass rate | Terminal | ×10.0 |
| Code quality | Terminal | ×5.0 |
| Full success | Terminal | +20.0 |
| Token scaling | Dense | ×0.1 / 1K tokens |
Training Pipeline (GRPO + Unsloth)
Qwen2.5-Coder-3B → Unsloth 4-bit QLoRA load
→ GRPO rollouts (G=8 per prompt)
→ ForgeRL reward (environment execution)
→ TRL GRPOTrainer update
→ Trained meta-agent
Results (300 GRPO steps on free Colab T4)
| Metric | Baseline | Trained |
|---|---|---|
| Mean reward | 0.14 | 0.82 |
| Task success | 8% | 81% |
| Recovery rate | 0% | 73% |
Links
- 📓 Colab:
training/ForgeRL_Training.ipynb - 🖥 Train locally:
python training/train_forgerl.py --steps 500 - 🎬 Demo:
python demo/run_demo.py --tier 1 - 📦 Install:
pip install -e .[deploy]
Adaptive curriculum statistics — tracks episode success rate per tier and auto-promotes / demotes.