# Evaluate question answering with squad dataset

**URL:** <https://discuss.huggingface.co/t/evaluate-question-answering-with-squad-dataset/10586>\
**Category:** Beginners\
**Created:** [October 7, 2021, 9:33am UTC](https://discuss.huggingface.co/t/evaluate-question-answering-with-squad-dataset/10586 "2021-10-07T09:33:59Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![aycha](https://avatars.discourse-cdn.com/v4/letter/a/c4cdca/32.png) [@aycha](https://discuss.huggingface.co/u/aycha)\
**Post date:** [October 7, 2021, 9:33am UTC](https://discuss.huggingface.co/t/evaluate-question-answering-with-squad-dataset/10586/1 "2021-10-07T09:33:59Z")

</div>

Hello everybody

I want to build question answering system by fine tuning bert using squad1.1 or squad2.0  
i would like to ask about evaluating question answering system, i know there is [squad](https://huggingface.co/metrics/squad) and squad\_v2 metrics, how can we use them when fine-tune bert with pytorch?  
thank you

---

<div class="post-metadata">

**Author:** ![nbroad](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nbroad/32/26465_2.png) [@nbroad](https://discuss.huggingface.co/u/nbroad)\
**Post date:** [October 7, 2021, 5:26pm UTC](https://discuss.huggingface.co/t/evaluate-question-answering-with-squad-dataset/10586/2 "2021-10-07T17:26:22Z")

</div>

This example should hopefully answer your question.

> <https://github.com/huggingface/transformers/blob/master/examples/pytorch/question-answering/run_qa.py>

If the purpose is to have a good question answering model, you could also use one of the many pretrained models on the hugging face model hub. [Models - Hugging Face](https://huggingface.co/models?sort=downloads&search=squad)

---

<div class="post-metadata">

**Author:** ![aycha](https://avatars.discourse-cdn.com/v4/letter/a/c4cdca/32.png) [@aycha](https://discuss.huggingface.co/u/aycha)\
**Post date:** [October 10, 2021, 7:29am UTC](https://discuss.huggingface.co/t/evaluate-question-answering-with-squad-dataset/10586/3 "2021-10-10T07:29:15Z")

</div>

thanks for your response