# Utilizing GPU in masked language modelling in Tensorflow

**URL:** <https://discuss.huggingface.co/t/utilizing-gpu-in-masked-language-modelling-in-tensorflow/32623>\
**Category:** 🤗Transformers\
**Created:** [February 27, 2023, 12:10pm UTC](https://discuss.huggingface.co/t/utilizing-gpu-in-masked-language-modelling-in-tensorflow/32623 "2023-02-27T12:10:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![bhavya-shah](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bhavya-shah/32/47362_2.png) [@bhavya-shah](https://discuss.huggingface.co/u/bhavya-shah)\
**Post date:** [February 27, 2023, 12:10pm UTC](https://discuss.huggingface.co/t/utilizing-gpu-in-masked-language-modelling-in-tensorflow/32623/1 "2023-02-27T12:10:01Z")

</div>

I am doing a masked language modelling task using distilbert model

This is the tokenizer that I trained:  
`tokenizer = t.PreTrainedTokenizerFast(tokenizer_file='path/to/tokenizer.json', unk_token="[UNK]", pad_token="[PAD]", cls_token="[CLS]", sep_token="[SEP]", mask_token="[MASK]")`

I am also using huggingface datasets library to preprocess my data so everything is in a standard way as shown in this tutorial: [Fine-tuning a masked language model - Hugging Face Course](https://huggingface.co/course/chapter7/3?fw=tf) .

_I’m not getting any errors and the model is training well._

But my complaint is that the training is not leveraging the GPU on my machine(I found this by opening the task manager + the training of 1 epochs is taking 3hrs). I even tried by enabling the GPU access from NVIDIA Control Panel but it did not help. I have NVIDIA GTX 1650 on my machine with CUDA 12.0(Found this by using the `nvidia-smi` command in CLI).

Please help me understand my mistake.