Add model card

README.md

@@ -0,0 +1,28 @@
+---
+license: mit
+language: en
+datasets:
+- EleutherAI/bergson-wikitext-512-chunks
+base_model: gpt2
+---
+
+# gpt-2-wikitext-chunks-epsroot-1e-6-bs64
+
+GPT-2 (124M) fine-tuned on [EleutherAI/bergson-wikitext-512-chunks](https://huggingface.co/datasets/EleutherAI/bergson-wikitext-512-chunks) with the [bergson](https://github.com/EleutherAI/bergson) MAGIC trainer, for use as a base model in training data attribution experiments.
+
+## Training details
+
+- Optimizer: AdamW (beta1=0.95, beta2=0.975, `eps_root=1e-6`, weight_decay=0.01)
+- Batch size: 64, epochs: 4, seed: 42, precision: fp32
+- LR: polynomial schedule, peak 8e-4, warmup 25%, end 8e-5
+- Trained with bergson 0.10.1 (git sha `27eb22b3`)
+
+## Replication
+
+The exact bergson run config is included as [`bergson_config.yaml`](./bergson_config.yaml). Replay it with:
+
+```bash
+bergson bergson_config.yaml
+```
+
+A companion model trained identically but with `eps_root=0` is available at [EleutherAI/gpt-2-wikitext-chunks-epsroot-0-bs64](https://huggingface.co/EleutherAI/gpt-2-wikitext-chunks-epsroot-0-bs64).