# How to sample from the validation set when using Trainer?

**URL:** <https://discuss.huggingface.co/t/how-to-sample-from-the-validation-set-when-using-trainer/25240>\
**Category:** 🤗Transformers\
**Created:** [October 31, 2022, 12:39am UTC](https://discuss.huggingface.co/t/how-to-sample-from-the-validation-set-when-using-trainer/25240 "2022-10-31T00:39:24Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![ruan-xf](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@ruan-xf](https://discuss.huggingface.co/u/ruan-xf)\
**Post date:** [November 21, 2025, 9:30am UTC](https://discuss.huggingface.co/t/how-to-sample-from-the-validation-set-when-using-trainer/25240/6 "2025-11-21T09:30:31Z")

</div>

Time has passed, but for people coming here:

that’s what I want for my large eval dataset, and I totally got you, it means fast and frequent evaluation but still make use of the existing eval dataset. it’s possible with TrainerCallback and indefinite IterableDataset. I’ve replied here:

> [@Evaluate subset of data during training](https://discuss.huggingface.co/t/evaluate-subset-of-data-during-training/10952/7):
>
> I implement a “sliding” eval dataset by using TrainerCallback and indefinite IterableDataset, here it’s: class DatasetSetting: def \_\_init\_\_(self, do\_encode: bool, per\_eval\_size: int, process\_fn: Callable=None): user\_enc, item\_enc = None, None if do\_encode: pkl\_path = Path('data/user\_item\_encoder.pkl') if not pkl\_path.exists(): fit\_encoder() enc: OrdinalEncoder = joblib.load(pkl\_path) user\_enc, item\_enc = (MyEncoder(cate) for c…

---

_[View the full topic](https://discuss.huggingface.co/t/how-to-sample-from-the-validation-set-when-using-trainer/25240)._
