# File upload slower and crashing

**URL:** <https://discuss.huggingface.co/t/file-upload-slower-and-crashing/62189>\
**Category:** 🤗Datasets\
**Created:** [November 14, 2023, 10:21am UTC](https://discuss.huggingface.co/t/file-upload-slower-and-crashing/62189 "2023-11-14T10:21:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![aleroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aleroy/32/21184_2.png) [@aleroy](https://discuss.huggingface.co/u/aleroy)\
**Post date:** [November 14, 2023, 10:21am UTC](https://discuss.huggingface.co/t/file-upload-slower-and-crashing/62189/1 "2023-11-14T10:21:29Z")

</div>

I have been using the UI to upload files to my dataset composed of 1080p images and optical flow data. It has been working fine for large chunks of files (a few thousand each time). However, for the last week or so, I cannot send the same amount of files (only a few hundreds at a time), otherwise, none of the files starts hashing and uploading.

When the upload does start, it is considerably slow and can take up to 5min for a few Mb file (which usually take a few seconds).

My initial thought the problem was coming from the bandwidth or my computer. But both work perfectly fine with other applications.

EDIT: I can see the following error being shown :

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/2/8/284840acdef1be97070064e1ba07d2aae0c7ae0d.jpeg)

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/64813_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [November 14, 2023, 1:05pm UTC](https://discuss.huggingface.co/t/file-upload-slower-and-crashing/62189/2 "2023-11-14T13:05:28Z")

</div>

We indeed recommend to do \<100 operations at a time to avoid 500 errors, see [Repository limitations and recommendations](https://huggingface.co/docs/hub/repositories-recommendations)

If you have thousands of files to upload you might consider using the Hub python client: [Upload files to the Hub](https://huggingface.co/docs/huggingface_hub/guides/upload)

---

<div class="post-metadata">

**Author:** ![aleroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aleroy/32/21184_2.png) [@aleroy](https://discuss.huggingface.co/u/aleroy)\
**Post date:** [November 14, 2023, 1:32pm UTC](https://discuss.huggingface.co/t/file-upload-slower-and-crashing/62189/3 "2023-11-14T13:32:23Z")

</div>

Thanks.  
Indeed it looks like using Git CLI could help me solve the problem.
