# Having trouble loading a fine-tuned PEFT model (CodeLlama-13b-Instruct-hf base)

**URL:** <https://discuss.huggingface.co/t/having-trouble-loading-a-fine-tuned-peft-model-codellama-13b-instruct-hf-base/52880>\
**Category:** 🤗Transformers\
**Created:** [August 30, 2023, 9:29pm UTC](https://discuss.huggingface.co/t/having-trouble-loading-a-fine-tuned-peft-model-codellama-13b-instruct-hf-base/52880 "2023-08-30T21:29:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![thenatefisher](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/thenatefisher/32/17153_2.png) [@thenatefisher](https://discuss.huggingface.co/u/thenatefisher)\
**Post date:** [August 30, 2023, 9:29pm UTC](https://discuss.huggingface.co/t/having-trouble-loading-a-fine-tuned-peft-model-codellama-13b-instruct-hf-base/52880/1 "2023-08-30T21:29:25Z")

</div>

I fine tuned codellama using PEFT, although I added some custom tokens and also a special token for padding. So instead of the original token vocab size of 32016, the adapter was trained using a slightly larger vocab of 32023. It seemed to work correctly after training. However, when I save it (`trainer.model.save_pretrained(...)` ) and reload it (`AutoPeftModelForCausalLM.from_pretrained(...)` ), I get this error:

> RuntimeError: Error(s) in loading state\_dict for PeftModelForCausalLM: size mismatch for base\_model.model.model.embed\_tokens.modules\_to\_save.default.weight: copying a param with shape torch.Size([32023, 5120]) from checkpoint, the shape in current model is torch.Size([32016, 5120]).

Any way to fix this?

---

<div class="post-metadata">

**Author:** ![Agniva](https://avatars.discourse-cdn.com/v4/letter/a/dfb087/32.png) [@Agniva](https://discuss.huggingface.co/u/Agniva)\
**Post date:** [November 21, 2023, 12:13pm UTC](https://discuss.huggingface.co/t/having-trouble-loading-a-fine-tuned-peft-model-codellama-13b-instruct-hf-base/52880/2 "2023-11-21T12:13:48Z")

</div>

hey @thenatefisher 🙂  
in order to load a model where you have changed token embeddings, lm head, you need to:

1. Add embeddings as a lora layer so it will be finetuned as well
2. Add embeddings as part of `modules_to_save` list in lora config
3. Make sure adapters are saved as checkpoints
4. Load base model again and add adapter to it: notice model\_id is the saved adapter checkpoint

hope this helps!

````auto
# Free memory for loading base model and merging adapter weights
del model
del trainer
torch.cuda.empty_cache()

# load base model and reshape embedding
model = AutoModelForCausalLM.from_pretrained(**model_params)
model.resize_token_embeddings(len(tokenizer))

# Merge Model with Adapter
model = PeftModel.from_pretrained(model=model, model_id="sft_llm/checkpoint-30")```
````

---

<div class="post-metadata">

**Author:** ![Victorano](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/victorano/32/32502_2.png) [@Victorano](https://discuss.huggingface.co/u/Victorano)\
**Post date:** [October 6, 2024, 2:36pm UTC](https://discuss.huggingface.co/t/having-trouble-loading-a-fine-tuned-peft-model-codellama-13b-instruct-hf-base/52880/3 "2024-10-06T14:36:27Z")

</div>

```bash
from trl import setup_chat_format
_, tokenizer = setup_chat_format(base_model, tokenizer)

```

Check `len(tokenizer)` before and after the above code block and see the difference.  
You can then resize the base model token embeddings using the new tokenizer

```bash
base_model.resize_token_embeddings(len(tokenizer))

```