# While tokenizing the dataset I'm getting this type of error

**URL:** <https://discuss.huggingface.co/t/while-tokenizing-the-dataset-im-getting-this-type-of-error/24779>\
**Category:** Beginners\
**Created:** [October 21, 2022, 3:25pm UTC](https://discuss.huggingface.co/t/while-tokenizing-the-dataset-im-getting-this-type-of-error/24779 "2022-10-21T15:25:26Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mahwizzzz](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahwizzzz/32/59233_2.png) [@mahwizzzz](https://discuss.huggingface.co/u/mahwizzzz)\
**Post date:** [October 21, 2022, 3:25pm UTC](https://discuss.huggingface.co/t/while-tokenizing-the-dataset-im-getting-this-type-of-error/24779/1 "2022-10-21T15:25:26Z")

</div>

can anyone help how to solve it…

 ![Capture](https://us1.discourse-cdn.com/hellohellohello/original/2X/8/89a8bde0042be92b954a346947473db322356a78.png)

---

<div class="post-metadata">

**Author:** ![mineshj1291](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mineshj1291/32/37149_2.png) [@mineshj1291](https://discuss.huggingface.co/u/mineshj1291)\
**Post date:** [October 21, 2022, 4:59pm UTC](https://discuss.huggingface.co/t/while-tokenizing-the-dataset-im-getting-this-type-of-error/24779/2 "2022-10-21T16:59:25Z")

</div>

Hi,  
I can’t see the error message. Can you paste the trace along with error message.

---

<div class="post-metadata">

**Author:** ![mahwizzzz](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahwizzzz/32/59233_2.png) [@mahwizzzz](https://discuss.huggingface.co/u/mahwizzzz)\
**Post date:** [October 21, 2022, 5:43pm UTC](https://discuss.huggingface.co/t/while-tokenizing-the-dataset-im-getting-this-type-of-error/24779/3 "2022-10-21T17:43:36Z")

</div>

sorry there is no error message but process stops at 95%

I’m using this example [notebooks/language\_modeling\_from\_scratch.ipynb at main · huggingface/notebooks · GitHub](https://github.com/huggingface/notebooks/blob/main/examples/language_modeling_from_scratch.ipynb)

and change the dataset to OSCAR

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\
**Post date:** [October 23, 2022, 1:19pm UTC](https://discuss.huggingface.co/t/while-tokenizing-the-dataset-im-getting-this-type-of-error/24779/4 "2022-10-23T13:19:39Z")

</div>

Hi! This is a (known) bug in the printing logic, not in the processing itself (see [Progress bars have color red and never completed to 100% · Issue #5117 · huggingface/datasets · GitHub](https://github.com/huggingface/datasets/issues/5117)), which we’ve already addressed. You can either install from “main” (`pip install git+https://github.com/huggingface/datasets.git`) or wait for the next release to fetch the fix.
