# Add additional trainable layers to StableDiffusion for fine-tuning

**URL:** <https://discuss.huggingface.co/t/add-additional-trainable-layers-to-stablediffusion-for-fine-tuning/57833>\
**Category:** 🧨 Diffusers\
**Created:** [October 8, 2023, 10:40pm UTC](https://discuss.huggingface.co/t/add-additional-trainable-layers-to-stablediffusion-for-fine-tuning/57833 "2023-10-08T22:40:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![aashananth](https://avatars.discourse-cdn.com/v4/letter/a/f4b2a3/32.png) [@aashananth](https://discuss.huggingface.co/u/aashananth)\
**Post date:** [October 8, 2023, 10:40pm UTC](https://discuss.huggingface.co/t/add-additional-trainable-layers-to-stablediffusion-for-fine-tuning/57833/1 "2023-10-08T22:40:00Z")

</div>

Hi, I am trying to modify the default StableDiffusion model architecture to support different types of text input in place of a single text caption. For this I plan to encode these text inputs using CLIP and combine the resultant embeddings using Cross Attention.

1. Where in the diffusers codebase would I need add this Cross Attention Layer/ modify existing layers and how would I be able to train the same?
2. Is it possible to combine the training of this layer while fine-tuning the base Stable Diffusion model (Full Finetuning or with LoRA)?