# Tool calling gets stuck in an infinite loop **URL:** \ **Category:** Inference Endpoints on the Hub\ **Created:** [April 10, 2025, 12:55pm UTC](https://discuss.huggingface.co/t/tool-calling-gets-stuck-in-an-infinite-loop/149792 "2025-04-10T12:55:37Z")\ **Posts on this page:** 1\ **Showing post:** 2
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [April 11, 2025, 4:12am UTC](https://discuss.huggingface.co/t/tool-calling-gets-stuck-in-an-infinite-loop/149792/2 "2025-04-11T04:12:29Z")
There is probably something wrong with the function calling in all models. It may be a bug in TGI. > [@Function calling not working with inference clients on (seemingly) any model](https://discuss.huggingface.co/t/function-calling-not-working-with-inference-clients-on-seemingly-any-model/138581): > > I’m attempting to use function-calling capabilities with text inference models, but none of the ways I’ve attempted to do with the inference clients work. I use OpenAI’s function calling regularly, but Hugging Face capabilities seem fragile and unpredictable, but I suspect I’m just “doing it wrong.” I’d appreciate any insights into the following cases I’m seeing: JavaScript inference client fails If I just include the first message (e.g. “What’s the weather in X place?”), the models will prope… > > > \### Description > > When using the inference client with function calling, models s…eem to never resolve their calls. > > As we know, typically, with the OpenAI pattern, the simplest function/tool call is a series of messages of various roles (system, user, assistant, tool) organized like this: > > system → user ("what's the weather?") → assistant (tool\_calls) → tool (result: "4ºC") → assistant (content: "it's 4ºC") > > The HF docs seem to indicate this is the same pattern, although the messages have some minor differences (e.g. description: null, which never happens with OpenAI). When using the Python inference client, these tool\_calls never resolve even after functions are called and their return values are included and seemingly properly referenced. Instead, they look like this: > > system → user ("what's the weather?") → assistant (tool\_calls) → tool (result: "4ºC") → assistant (tool\_calls) … > > Instead of returning a text completion, the HF inference client returns the same "assistant" message specifying a required tool\_calls. In OpenAI, they resolve to a typical "assistant" message with token content if the function calls have been satisfied and no further calls are required. > > Models used that exhibit this behavior: > \- NousResearch/Hermes-3-Llama-3.1-8B > \- Qwen/Qwen2.5-72B-Instruct > \- meta-llama/Meta-Llama-3-8B-Instruct > > \--- > > It's worth noting that Mistral models also error out, specifying that a 9-character alphanumeric string is required for the \`tool\_call\_id\`. Now, the models themselves don't provide such IDs, so we need to supply them ourselves. But even when doing so, the same error occurs, that 9-char identifiers are missing. (e.g. mistralai/Mistral-7B-Instruct-v0.3) > > The JavaScript client also fails with the above errors, and also a third: "An error occurred while fetching the blob". > > \### System Info > > \- macOS 15.2 > \- Python 3.13.1 > \- huggingface\_hub 0.28.1 > > \### Information > > \- \[\] Docker > \- \[x\] The CLI directly > > \### Tasks > > \- \[x\] An officially supported command > \- \[\] My own modifications > > \### Reproduction > > Gist of sample error code is here: https://gist.github.com/awmartin/c64c84fbbdc3a9f0c2ce6e5ae0dab3dc > > 1. Provide API token > 2. python inference-tool-calls.py > > A message results that's unexpected. I expected this to be a typical message with a string content, something like, "It's 4 degrees today." Instead, it just repeats the assistant message with the original tool\_call message: > > \[ChatCompletionOutputComplete(finish\_reason='stop', index=0, message=ChatCompletionOutputMessage(role='assistant', content=None, tool\_calls=\[ChatCompletionOutputToolCall(function=ChatCompletionOutputFunctionDefinition(arguments={'unit': 'Celsius', 'location': 'Philadelphia, PA, US'}, name='get\_current\_temperature', description=None), id='0', type='function')\]), logprobs=None)\] > > \### Expected behavior > > I expected a message that resolved to something similar to "It's 4 degrees Celsius today" rather than the tool\_call message repeated. --- _[View the full topic](https://discuss.huggingface.co/t/tool-calling-gets-stuck-in-an-infinite-loop/149792)._