# Autoscaling on inference endpoints not initializing from 0 replicas

**URL:** <https://discuss.huggingface.co/t/autoscaling-on-inference-endpoints-not-initializing-from-0-replicas/76144>\
**Category:** Inference Endpoints on the Hub\
**Created:** [March 6, 2024, 7:30am UTC](https://discuss.huggingface.co/t/autoscaling-on-inference-endpoints-not-initializing-from-0-replicas/76144 "2024-03-06T07:30:37Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![spacecon](https://avatars.discourse-cdn.com/v4/letter/s/5fc32e/32.png) [@spacecon](https://discuss.huggingface.co/u/spacecon)\
**Post date:** [March 6, 2024, 7:30am UTC](https://discuss.huggingface.co/t/autoscaling-on-inference-endpoints-not-initializing-from-0-replicas/76144/1 "2024-03-06T07:30:37Z")

</div>

Hello!

Have an inference endpoint with Replica scaling to 0 upon 15min of no requests. When it scales to 0, an API call and the “Test your endpoint” section in the frontend results in an “500 Internal Server Error” and the Replica does not scale up to 1. I can see the request in the analytics tab as well.

The endpoint only scales up to 1 if I press this button in the frontend.  
 ![here](https://us1.discourse-cdn.com/hellohellohello/original/3X/a/0/a08d4bf2dc1342fec27b04d59959dfba9e2b3472.png)

In this case, the endpoint starts as normal and can take requests from the API like normal.  
It might be important to note that I am using a custom handler.py .

I would like the inference endpoint to scale to 1 replica upon an API request.  
Any help is highly appreciated!

---

<div class="post-metadata">

**Author:** ![Korfhage](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@Korfhage](https://discuss.huggingface.co/u/Korfhage)\
**Post date:** [June 19, 2024, 7:53am UTC](https://discuss.huggingface.co/t/autoscaling-on-inference-endpoints-not-initializing-from-0-replicas/76144/2 "2024-06-19T07:53:46Z")

</div>

Same problem here, any help would be welcome!

---

<div class="post-metadata">

**Author:** ![duckduckgrayduck](https://avatars.discourse-cdn.com/v4/letter/d/e9c0ed/32.png) [@duckduckgrayduck](https://discuss.huggingface.co/u/duckduckgrayduck)\
**Post date:** [June 27, 2024, 8:56am UTC](https://discuss.huggingface.co/t/autoscaling-on-inference-endpoints-not-initializing-from-0-replicas/76144/3 "2024-06-27T08:56:35Z")

</div>

I resolved this by sending an initial probing request, which I expected to get a 503 in response, and then sleep for 60 seconds while it initializes. I then send another request to confirm that it has woken up before sending my actual requests. See my example here: [https://github.com/duckduckgrayduck/documentcloud-gumshoe-2-addon/blob/aa3806c3b46b7cf7c9118871e921f14102d75ef5/main.py#L82](https://github.com/duckduckgrayduck/documentcloud-gumshoe-2-addon/blob/aa3806c3b46b7cf7c9118871e921f14102d75ef5/main.py#L82)