# Transformer vs Sentence-Transformer for text classification

**URL:** <https://discuss.huggingface.co/t/transformer-vs-sentence-transformer-for-text-classification/77047>\
**Category:** Intermediate\
**Created:** [March 12, 2024, 8:58am UTC](https://discuss.huggingface.co/t/transformer-vs-sentence-transformer-for-text-classification/77047 "2024-03-12T08:58:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![MattBlue92](https://avatars.discourse-cdn.com/v4/letter/m/3d9bf3/32.png) [@MattBlue92](https://discuss.huggingface.co/u/MattBlue92)\
**Post date:** [March 12, 2024, 8:58am UTC](https://discuss.huggingface.co/t/transformer-vs-sentence-transformer-for-text-classification/77047/1 "2024-03-12T08:58:00Z")

</div>

Hi to everyone!

I was wondering if I can training a sentence transformer with a triplet-loss (with and without labels data) and then use this model (or its embedding) freezeing all its layers for fine-tuning of a classification head (such as a classic fully connected network) with the same data or an hidden portion data.

In alternative, can I training a classic transformer (BERT cross encoder) with a classification head using a classic classification loss such as cross entropy, but insted of using the embedding of CLS token to feed into classification head I feed into the head an embedding vector created by or max pooling o average pooling from all tokens embedding from the last layer of the transformer?

I ask because I am curious to find out whether the feature space of sentence transformer and a classic transformer is different but still useful for the classification task.

Thank you!
