# Finetuned MT5 model generating the same first token for any input

**URL:** <https://discuss.huggingface.co/t/finetuned-mt5-model-generating-the-same-first-token-for-any-input/39138>\
**Category:** Intermediate\
**Created:** [May 9, 2023, 12:42am UTC](https://discuss.huggingface.co/t/finetuned-mt5-model-generating-the-same-first-token-for-any-input/39138 "2023-05-09T00:42:56Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![gp06aug](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/gp06aug/32/15965_2.png) [@gp06aug](https://discuss.huggingface.co/u/gp06aug)\
**Post date:** [May 9, 2023, 12:42am UTC](https://discuss.huggingface.co/t/finetuned-mt5-model-generating-the-same-first-token-for-any-input/39138/1 "2023-05-09T00:42:56Z")

</div>

Hi All,  
I finetuned an MT5-base checkpoint for a natural language to BASH translation task and notice the model is generating the same first token for any input. MBART on the same data doesn’t have this problem and generates expected results. Any idea what could be going wrong? Is there any issue with MT5 checkpoints, I earlier had to remove the mixed precision setting for it to work properly.