# Cannot Setup Mixtral Models and Other Models on Inference Endpoints

**URL:** <https://discuss.huggingface.co/t/cannot-setup-mixtral-models-and-other-models-on-inference-endpoints/66125>\
**Category:** Inference Endpoints on the Hub\
**Created:** [December 17, 2023, 8:16pm UTC](https://discuss.huggingface.co/t/cannot-setup-mixtral-models-and-other-models-on-inference-endpoints/66125 "2023-12-17T20:16:19Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Blevlabs](https://avatars.discourse-cdn.com/v4/letter/b/7ab992/32.png) [@Blevlabs](https://discuss.huggingface.co/u/Blevlabs)\
**Post date:** [December 17, 2023, 8:16pm UTC](https://discuss.huggingface.co/t/cannot-setup-mixtral-models-and-other-models-on-inference-endpoints/66125/1 "2023-12-17T20:16:19Z")

</div>

Here is a link to the error logs and the discussion thread related to my issue and many others who have the same problem:

> **[ehartford/dolphin-2.5-mixtral-8x7b · HuggingFace Inference Endpoints Issue...](https://huggingface.co/ehartford/dolphin-2.5-mixtral-8x7b/discussions/4#657c6f400a3297531f117d86)**
>
> Hello, I saw this topic was related to one of the closed tickets, and that the user did not provide adequate information. I wanted to share my attempt to host it on the inference endpoint as I have...

Even with other models, I am encountering the same issue on my account. Any ideas for possible resolutions? The Inference Endpoints as a service does not function for me right now.

---

<div class="post-metadata">

**Author:** ![Agniva](https://avatars.discourse-cdn.com/v4/letter/a/dfb087/32.png) [@Agniva](https://discuss.huggingface.co/u/Agniva)\
**Post date:** [December 22, 2023, 2:14pm UTC](https://discuss.huggingface.co/t/cannot-setup-mixtral-models-and-other-models-on-inference-endpoints/66125/2 "2023-12-22T14:14:26Z")

</div>

Having the same problem. I keep getting sharding errors like these issues:

1. [cognitivecomputations/dolphin-2.5-mixtral-8x7b · HuggingFace Inference Endpoints Issue (Detailed Information)](https://huggingface.co/cognitivecomputations/dolphin-2.5-mixtral-8x7b/discussions/4#657c6f400a3297531f117d86)
2. [tiiuae/falcon-7b-instruct · Trying to deploy this using Inference Endpoints](https://huggingface.co/tiiuae/falcon-7b-instruct/discussions/96)