Hi, I think there’s a bug with Inference Providers billing. For the last 2 days, every request is being charged a flat $0.01, regardless of model or actual usage. Screenshots of the last 2 days attached showing the pattern. Anyone having same issue?
Think I’m also getting this bug.
I just started using inference providers
13 requests with Qwen2.5-Coder-32B-Instruct cost $0.09
and 15 requests with Qwen2.5-Coder-7B-Instruct cost $0.10
The requests are very simple
I recorded the input/output tokens for all the requests
e.g. for the last 32B API call I have 313 input tokens, 143 output tokens, calculated cost:
(313/1e6)*0.06 + (143/1e6)*0.20 = $0.0000188 + $0.0000286 = $0.0000474
and all of the requests are quite similar, so 15 requests shouldn’t be costing $0.10
In particular the 7B model was running on the same dataset and same prompts, and should cost even less than the 32B model
A little strange - the prices seem to have corrected themselves on the billing page after a couple hours.
But then I tried making more API calls, and the new batch of API calls costs $0.01 per request.
And it seems like even if the billing page corrects itself after a while, the limits based on your budget are based on the $0.01 per request.
Yea, that seems to be the case. It’s a bit frustrated though cause there’s no indication on the chart to explain the cost will correct itself, maybe a bit of UX improvements needed there.
Anw, here’s reply from support, I ll keep monitoring though:
“When you send a request, a placeholder price is charged instantly so usage shows up right away in your billing settings. Later, we’ll update the total charges with the real per-request cost from the provider, and we’ll also ensure unsuccessful calls aren’t billed.
The best metric to gauge your real-time inference billing is the number of remaining credits you have.”
Maybe or must be a bug…? @hysts @meganariley @michellehbn
Thanks for reporting this. I’ll check with the team internally.
Thank you!
Just checked and this seems to be working okay now. You still get charged $0.01 per query, but it self-corrects to the actual price in under a minute, much faster than before.
The problem before is that if you wanted to run say 10,000 cheap queries you would have to have $100 in credits, even if they would cost much less. But this seems to be an okay solution, as long as your queries aren’t going too fast you won’t hit the limit.
Thanks!