# ELECTRA for Causal LM

**URL:** <https://discuss.huggingface.co/t/electra-for-causal-lm/5364>\
**Category:** 🤗Transformers\
**Created:** [April 8, 2021, 5:32am UTC](https://discuss.huggingface.co/t/electra-for-causal-lm/5364 "2021-04-08T05:32:38Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![yusukemori](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/yusukemori/32/23302_2.png) [@yusukemori](https://discuss.huggingface.co/u/yusukemori)\
**Post date:** [April 8, 2021, 5:32am UTC](https://discuss.huggingface.co/t/electra-for-causal-lm/5364/1 "2021-04-08T05:32:38Z")

</div>

Hi,

Can we use `ElectraForMaskedLM` for causal language modeling?

It seems `BertForMaskedLM` was split into and `BertLMHeadModel` and `BertForMaskedLM`, one model for causal LM, one for masked LM.  
[https://github.com/huggingface/transformers/pull/4874](https://github.com/huggingface/transformers/pull/4874)

Is ELECTRA’s Masked LM class can be applied for causal LM?  
Or is there a separate class?  
Or is it not possible to use ELECTRA as Causal LM in the first place?

Thank you in advance.