# Fionetune model always predicts same output class for new data

**URL:** <https://discuss.huggingface.co/t/fionetune-model-always-predicts-same-output-class-for-new-data/12863>\
**Category:** Models\
**Created:** [December 15, 2021, 2:40pm UTC](https://discuss.huggingface.co/t/fionetune-model-always-predicts-same-output-class-for-new-data/12863 "2021-12-15T14:40:01Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![emmakelo](https://avatars.discourse-cdn.com/v4/letter/e/ea666f/32.png) [@emmakelo](https://discuss.huggingface.co/u/emmakelo)\
**Post date:** [December 15, 2021, 2:40pm UTC](https://discuss.huggingface.co/t/fionetune-model-always-predicts-same-output-class-for-new-data/12863/1 "2021-12-15T14:40:01Z")

</div>

I have a problem, trained a model with bert which give around 0.90% on test data and I decide to use it on new data which were not annotated. When running the model, I keep getting the same class output. And I would like to know why ?

Can you help me ?

---

<div class="post-metadata">

**Author:** ![bhavnicksm](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bhavnicksm/32/3547_2.png) [@bhavnicksm](https://discuss.huggingface.co/u/bhavnicksm)\
**Post date:** [December 25, 2021, 2:12pm UTC](https://discuss.huggingface.co/t/fionetune-model-always-predicts-same-output-class-for-new-data/12863/2 "2021-12-25T14:12:01Z")

</div>

Hey! Just wanted to state that I am facing the same problem and still don’t know how this is happening. I was working with RoBERTaForSequenceClassification with a binary classification problem and it gave 50% ACC. after 5 epochs. It did not learn anything at all.

---

<div class="post-metadata">

**Author:** ![satpalsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/satpalsr/32/3553_2.png) [@satpalsr](https://discuss.huggingface.co/u/satpalsr)\
**Post date:** [December 25, 2021, 3:33pm UTC](https://discuss.huggingface.co/t/fionetune-model-always-predicts-same-output-class-for-new-data/12863/3 "2021-12-25T15:33:56Z")

</div>

Hey @emmakelo, it may be a dataset imbalance. Accuracy is generally not a good metric, check the precision and recall values.

---

<div class="post-metadata">

**Author:** ![emmakelo](https://avatars.discourse-cdn.com/v4/letter/e/ea666f/32.png) [@emmakelo](https://discuss.huggingface.co/u/emmakelo)\
**Post date:** [December 30, 2021, 8:53am UTC](https://discuss.huggingface.co/t/fionetune-model-always-predicts-same-output-class-for-new-data/12863/4 "2021-12-30T08:53:05Z")

</div>

Thank you!! I solve the problem. Actually I was not loading correctly the model after training, do not know why ? Now the problem is corrected.

---

<div class="post-metadata">

**Author:** ![dboy10000](https://avatars.discourse-cdn.com/v4/letter/d/fbc32d/32.png) [@dboy10000](https://discuss.huggingface.co/u/dboy10000)\
**Post date:** [August 17, 2022, 10:55am UTC](https://discuss.huggingface.co/t/fionetune-model-always-predicts-same-output-class-for-new-data/12863/5 "2022-08-17T10:55:27Z")

</div>

What was the issue when loading it? I think I’m having the same problem. I get exactly the same output from the model no matter what I give it as input.

```auto
loaded_model = AutoModelForSequenceClassification.from_pretrained("custom_model")
tokenizer = AutoTokenizer.from_pretrained("roberta-base")
predict_input = tokenizer("Hello world", truncation=True, padding=True, return_tensors="pt")
predictions = model(**predict_input)

```

---

<div class="post-metadata">

**Author:** ![emmakelo](https://avatars.discourse-cdn.com/v4/letter/e/ea666f/32.png) [@emmakelo](https://discuss.huggingface.co/u/emmakelo)\
**Post date:** [August 29, 2022, 11:58am UTC](https://discuss.huggingface.co/t/fionetune-model-always-predicts-same-output-class-for-new-data/12863/6 "2022-08-29T11:58:10Z")

</div>

I think you need to set a seed before initialising the model.

---

<div class="post-metadata">

**Author:** ![felixckyeung](https://avatars.discourse-cdn.com/v4/letter/f/3ab097/32.png) [@felixckyeung](https://discuss.huggingface.co/u/felixckyeung)\
**Post date:** [April 4, 2024, 8:19pm UTC](https://discuss.huggingface.co/t/fionetune-model-always-predicts-same-output-class-for-new-data/12863/7 "2024-04-04T20:19:29Z")

</div>

I tried setting a seed and it worked for a bit, and then it went back to outputting the same thing. Any advice?

---

<div class="post-metadata">

**Author:** ![syamaK6](https://avatars.discourse-cdn.com/v4/letter/s/a587f6/32.png) [@syamaK6](https://discuss.huggingface.co/u/syamaK6)\
**Post date:** [June 19, 2024, 11:29am UTC](https://discuss.huggingface.co/t/fionetune-model-always-predicts-same-output-class-for-new-data/12863/8 "2024-06-19T11:29:30Z")

</div>

Hi, I also have the same problem. I am using Protbert model for protein sequence classification. While training it produces different logits for each train sequences. but for model.eval(), it produces the same logits for each of the sequences in the validation dataset? The same issue for the protT5 model from the huggingface.
