# Smart Batching - speech up Bert finetune

**URL:** <https://discuss.huggingface.co/t/smart-batching-speech-up-bert-finetune/4406>\
**Category:** Intermediate\
**Created:** [March 15, 2021, 9:31am UTC](https://discuss.huggingface.co/t/smart-batching-speech-up-bert-finetune/4406 "2021-03-15T09:31:17Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![BinhMinhs10](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/binhminhs10/32/60_2.png) [@BinhMinhs10](https://discuss.huggingface.co/u/BinhMinhs10)\
**Post date:** [March 15, 2021, 9:31am UTC](https://discuss.huggingface.co/t/smart-batching-speech-up-bert-finetune/4406/1 "2021-03-15T09:31:17Z")

</div>

code: [Smart Batching Tutorial - Speed Up BERT Training · Chris McCormick](https://mccormickml.com/2020/07/29/smart-batching-tutorial/)  
I have tried fine-tune on 1 epoch and the time will decrease & accuracy increase but if it increases by 3-4 epoch then it looks like the model has learned less focus on pad token so smart batch will not accuracy increase.
