# Special tokens with inputs\_embeds input

**URL:** <https://discuss.huggingface.co/t/special-tokens-with-inputs-embeds-input/46323>\
**Category:** Beginners\
**Created:** [July 10, 2023, 6:42pm UTC](https://discuss.huggingface.co/t/special-tokens-with-inputs-embeds-input/46323 "2023-07-10T18:42:03Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Paul1020](https://avatars.discourse-cdn.com/v4/letter/p/c37758/32.png) [@Paul1020](https://discuss.huggingface.co/u/Paul1020)\
**Post date:** [July 10, 2023, 6:42pm UTC](https://discuss.huggingface.co/t/special-tokens-with-inputs-embeds-input/46323/1 "2023-07-10T18:42:03Z")

</div>

I want to train models with my own embeddings. I am doing two sequences classification task.  
How I deal with CLS and SEP token with his case ?  
cls\_embedding = torch.randn(1, embedding\_dim) # [CLS] token embedding  
sep\_embedding = torch.randn(1, embedding\_dim) # [SEP] token embedding  
And pad my embeddings ? Or there some other way for that ?