# How do I load a trained checkpoint model?

**URL:** <https://discuss.huggingface.co/t/how-do-i-load-a-trained-checkpoint-model/155932>\
**Category:** 🤗Transformers\
**Created:** [May 19, 2025, 5:58pm UTC](https://discuss.huggingface.co/t/how-do-i-load-a-trained-checkpoint-model/155932 "2025-05-19T17:58:38Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [May 20, 2025, 5:55am UTC](https://discuss.huggingface.co/t/how-do-i-load-a-trained-checkpoint-model/155932/2 "2025-05-20T05:55:10Z")

</div>

I think that method is correct, but there seem to be reports of tensor mismatch issues when training with FSDP.

> [@Key errors when trying to load an accelerate-FSDP model checkpoint](https://discuss.huggingface.co/t/key-errors-when-trying-to-load-an-accelerate-fsdp-model-checkpoint/85829):
>
> Hi, I have been training a pythia-2.8b model, using the DPOTrainer in Huggingface TRL. I am training a LORA adaptor on top of the reference SFT model. In order to speed it up and address memory bottlenecks, I used fsdp and launched my code using accelerate launch --config\_file=fsdp.yaml --num\_processes=4 run\_dpo.py After training, I observed my checkpoints to be of the following form optimizer\_0/ pytorch\_model\_fsdp\_0/ rng\_state\_0.pth rng\_state\_1.pth scheduler.pt trainer\_…

> [@How to load a checkpoint model with SHARDED\_STATE\_DICT?](https://discuss.huggingface.co/t/how-to-load-a-checkpoint-model-with-sharded-state-dict/62448):
>
> How to load a checkpoint model with SHARDED\_STATE\_DICT? I have a checkpoint which is place in a folder pytorch\_model\_0, which contains multiple distcp files.

> **[Serialization](https://huggingface.co/docs/huggingface_hub/main/en/package_reference/serialization)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

---

_[View the full topic](https://discuss.huggingface.co/t/how-do-i-load-a-trained-checkpoint-model/155932)._
