# Question about PAST KEY VALUES in ONNX format decoder

**URL:** <https://discuss.huggingface.co/t/question-about-past-key-values-in-onnx-format-decoder/146817>\
**Category:** Models\
**Created:** [March 21, 2025, 3:30pm UTC](https://discuss.huggingface.co/t/question-about-past-key-values-in-onnx-format-decoder/146817 "2025-03-21T15:30:03Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![itsmax](https://avatars.discourse-cdn.com/v4/letter/i/9fc29f/32.png) [@itsmax](https://discuss.huggingface.co/u/itsmax)\
**Post date:** [March 21, 2025, 3:30pm UTC](https://discuss.huggingface.co/t/question-about-past-key-values-in-onnx-format-decoder/146817/1 "2025-03-21T15:30:03Z")

</div>

I am currently using ONNX musicgen to infer musics. However, I encountered a problem, I just want to ask how to generate inputs when the inputs like this  
=== Model: decoder\_model\_merged.onnx ===

Inputs:  
Name: encoder\_attention\_mask Shape: [‘batch\_size’, ‘encoder\_sequence\_length’] DType: int64  
Name: input\_ids Shape: [‘total\_batch\_size\_x\_num\_codebooks’, ‘decoder\_sequence\_length’] DType: int64  
Name: encoder\_hidden\_states Shape: [‘total\_batch\_size’, ‘encoder\_sequence\_length’, 768] DType: float32  
Name: past\_key\_values.0.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.0.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.0.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.0.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.1.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.1.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.1.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.1.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.2.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.2.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.2.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.2.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.3.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.3.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.3.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.3.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.4.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.4.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.4.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.4.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.5.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.5.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.5.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.5.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.6.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.6.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.6.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.6.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.7.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.7.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.7.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.7.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.8.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.8.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.8.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.8.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.9.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.9.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.9.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.9.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.10.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.10.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.10.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.10.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.11.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.11.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.11.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.11.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.12.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.12.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.12.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.12.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.13.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.13.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.13.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.13.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.14.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.14.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.14.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.14.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.15.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.15.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.15.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.15.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.16.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.16.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.16.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.16.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.17.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.17.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.17.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.17.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.18.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.18.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.18.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.18.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.19.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.19.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.19.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.19.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.20.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.20.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.20.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.20.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.21.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.21.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.21.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.21.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.22.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.22.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.22.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.22.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.23.decoder.key Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.23.decoder.value Shape: [‘total\_batch\_size’, 16, ‘past\_decoder\_sequence\_length’, 64] DType: float32  
Name: past\_key\_values.23.encoder.key Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: past\_key\_values.23.encoder.value Shape: [‘total\_batch\_size’, 16, ‘encoder\_sequence\_length\_out’, 64] DType: float32  
Name: use\_cache\_branch Shape: [1] DType: bool

> Blockquote

also when come to the second stop of generation, I encounter this problem  
2025-03-21 22:59:12.9606373 [E:onnxruntime:, sequential\_executor.cc:572 onnxruntime::ExecuteKernel] Non-zero status code returned while running Reshape node. Name:‘/decoder/model/decoder/layers.0/encoder\_attn/Reshape\_4’ Status Message: D:\a\_work\1\s\onnxruntime\core/providers/cpu/tensor/reshape\_helper.h:30 onnxruntime::ReshapeHelper::ReshapeHelper i \< input\_shape.NumDimensions() was false. The dimension with value zero exceeds the dimension size of the input tensor.

2025-03-21 22:59:12.9689039 [E:onnxruntime:, sequential\_executor.cc:572 onnxruntime::ExecuteKernel] Non-zero status code returned while running If node. Name:‘optimum::if’ Status Message: Non-zero status code returned while running Reshape node. Name:‘/decoder/model/decoder/layers.0/encoder\_attn/Reshape\_4’ Status Message: D:\a\_work\1\s\onnxruntime\core/providers/cpu/tensor/reshape\_helper.h:30 onnxruntime::ReshapeHelper::ReshapeHelper i \< input\_shape.NumDimensions() was false. The dimension with value zero exceeds the dimension size of the input tensor.

Traceback (most recent call last):  
File “d:\allcode\on-device-music-generator\text\_code\musicgen\_generate.py”, line 444, in   
musicgen\_onnx.test\_onnx()  
File “d:\allcode\on-device-music-generator\text\_code\musicgen\_generate.py”, line 132, in test\_onnx  
logits, past\_key\_values = self.run\_cached\_step(decoder\_inputs, past\_key\_values)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “d:\allcode\on-device-music-generator\text\_code\musicgen\_generate.py”, line 251, in run\_cached\_step  
outputs = self.sessions[‘decoder\_merged’].run(None, decoder\_inputs)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
File “C:\Users\31819\anaconda3\envs\all-stack-ai\Lib\site-packages\onnxruntime\capi\onnxruntime\_inference\_collection.py”, line 270, in run  
return self.\_sess.run(output\_names, input\_feed, run\_options)  
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
onnxruntime.capi.onnxruntime\_pybind11\_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME\_EXCEPTION : Non-zero status code returned while running If node. Name:‘optimum::if’ Status Message: Non-zero status code returned while running Reshape node. Name:‘/decoder/model/decoder/layers.0/encoder\_attn/Reshape\_4’ Status Message: D:\a\_work\1\s\onnxruntime\core/providers/cpu/tensor/reshape\_helper.h:30 onnxruntime::ReshapeHelper::ReshapeHelper i \< input\_shape.NumDimensions() was false. The dimension with value zero exceeds the dimension size of the input tensor.

> Blockquote

does anyone know how to solve it