# Generate raw word embeddings using transformer models like BERT for downstream process

**URL:** <https://discuss.huggingface.co/t/generate-raw-word-embeddings-using-transformer-models-like-bert-for-downstream-process/2958>\
**Category:** Beginners\
**Created:** [December 25, 2020, 1:50am UTC](https://discuss.huggingface.co/t/generate-raw-word-embeddings-using-transformer-models-like-bert-for-downstream-process/2958 "2020-12-25T01:50:08Z")\
**Posts on this page:** 1\
**Showing post:** 9

<div class="post-metadata">

**Author:** ![Sanjeeth](https://avatars.discourse-cdn.com/v4/letter/s/eb9ed0/32.png) [@Sanjeeth](https://discuss.huggingface.co/u/Sanjeeth)\
**Post date:** [October 4, 2021, 1:27pm UTC](https://discuss.huggingface.co/t/generate-raw-word-embeddings-using-transformer-models-like-bert-for-downstream-process/2958/9 "2021-10-04T13:27:19Z")

</div>

Hi Bram Vanroy, I was trying the above mentioned code with respect to BertTokenizer instead of AutoTokenizer but i get a error as mentioned below.

**ValueError: word\_ids() is not available when using Python-based tokenizers**

Can you please let me know what changes to be made in code to get a list indicating the word corresponding to each token. Special tokens added by the tokenizer are mapped to `None` and other tokens are mapped to the index of their corresponding word

---

_[View the full topic](https://discuss.huggingface.co/t/generate-raw-word-embeddings-using-transformer-models-like-bert-for-downstream-process/2958)._
