---
license: mit
tags:
  - seismology
  - earthquake
  - waveform-prediction
  - transformer
  - pytorch
  - pytorch-lightning
library_name: pytorch
pipeline_tag: other
---

# SeismoGPT (Phase 1)

Pre-trained **Phase 1** weights for **SeismoGPT**: a causal RoPE transformer over seismic waveform tokens (Z, N, E), trained with log-cosh loss and multi-horizon auxiliary heads.

This checkpoint is the deterministic model used for paper rollouts and figures in the companion code repository.

**Paper:** [Data-Driven Forecasting of three-Component Seismograms Using Transformer Architectures](https://arxiv.org/abs/2606.02912v1) — [arXiv:2606.02912v1](https://arxiv.org/abs/2606.02912v1)  
**Code:** [github.com/wesmail/SeismoGPT](https://github.com/wesmail/SeismoGPT)

## Files

| File | Description |
|------|-------------|
| `epoch=12-step=633750.ckpt` | PyTorch Lightning checkpoint (Phase 1, epoch 12) |
| `train_phase1_logcosh.yaml` | Training configuration used for Phase 1 |

## Requirements

- Python 3.10+
- `torch`, `lightning`, `seisbench`, and the [SeismoGPT code repository](https://github.com/wesmail/SeismoGPT)

## Download weights

```bash
pip install huggingface_hub
huggingface-cli download wesmail/SeismoGPT \
  epoch=12-step=633750.ckpt train_phase1_logcosh.yaml \
  --local-dir phase1
```

Or from Python:

```python
from huggingface_hub import hf_hub_download

ckpt = hf_hub_download(
    repo_id="wesmail/SeismoGPT",
    filename="epoch=12-step=633750.ckpt",
    local_dir="phase1",
)
```

Use the same `kernel_size` (16) and `num_tokens` (320) as in training; these are stored in the checkpoint hyperparameters. For evaluation (configurations A/B/C), rollouts, and plotting, see the [code repository](https://github.com/wesmail/SeismoGPT).

## Model summary

| Setting | Value |
|---------|--------|
| Phase | 1 (deterministic, `log_cosh`) |
| Channels | Z, N, E |
| Token size K | 16 samples |
| Tokens per window | 320 |
| `d_model` | 512 |
| Encoder layers | 8 |
| Prediction horizons (train) | 4 (inference uses horizon 1) |