# Getting issue on query\_engine for llm\_completion\_callback

**URL:** <https://discuss.huggingface.co/t/getting-issue-on-query-engine-for-llm-completion-callback/82780>\
**Category:** Beginners\
**Created:** [April 20, 2024, 11:54am UTC](https://discuss.huggingface.co/t/getting-issue-on-query-engine-for-llm-completion-callback/82780 "2024-04-20T11:54:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![marsalan06](https://avatars.discourse-cdn.com/v4/letter/m/ecd19e/32.png) [@marsalan06](https://discuss.huggingface.co/u/marsalan06)\
**Post date:** [April 20, 2024, 11:54am UTC](https://discuss.huggingface.co/t/getting-issue-on-query-engine-for-llm-completion-callback/82780/1 "2024-04-20T11:54:42Z")

</div>

I am implementing a Rag using Hugging Face Model, i have 0.10 version of LLama Index , facing error on query engine

import torch

llm = HuggingFaceLLM(  
context\_window=4096,  
max\_new\_tokens=256,  
generate\_kwargs={“temperature”:0.5,“do\_sample”:False},  
system\_prompt=system\_prompt,  
query\_wrapper\_prompt=query\_wrapper\_prompt,  
tokenizer\_name=‘meta-llama/Llama-2-7b-chat-hf’,  
model\_name=‘meta-llama/Llama-2-7b-chat-hf’,  
device\_map=‘auto’,  
model\_kwargs={‘torch\_dtype’:torch.float16},  
)

embed\_model=LangchainEmbedding(HuggingFaceEmbeddings(model\_name=“sentence-transformers/all-mpnet-base-v2”))  
from llama\_index.core import Settings

from llama\_index.core.node\_parser import SentenceSplitter

Settings.llm = llm

Settings.embed\_model = embed\_model

Settings.node\_parser = SentenceSplitter(chunk\_size=1024, chunk\_overlap=20)  
index = VectorStoreIndex.from\_documents(documents, show\_progress=True)

query\_engine = index.as\_query\_engine()

##########Error In Execution##################  
response = query\_engine.query(“what is a wide ball”)  
##########################################

* * *

ValueError Traceback (most recent call last)  
 in \<cell line: 2\>()  
1 query\_engine = index.as\_query\_engine()  
----\> 2 response = query\_engine.query(“what is a wide ball”)  
3 # response

15 frames  
/usr/local/lib/python3.10/dist-packages/llama\_index/legacy/llms/base.py in wrapper\_logic(\_self)  
161 callback\_manager = getattr(\_self, “callback\_manager”, None)  
162 if not isinstance(callback\_manager, CallbackManager):  
 → 163 raise ValueError(  
164 "Cannot use llm\_completion\_callback on an instance "  
165 “without a callback\_manager attribute.”

ValueError: Cannot use llm\_completion\_callback on an instance without a callback\_manager attribute.

Desperately looking for a solution

---

<div class="post-metadata">

**Author:** ![marsalan06](https://avatars.discourse-cdn.com/v4/letter/m/ecd19e/32.png) [@marsalan06](https://discuss.huggingface.co/u/marsalan06)\
**Post date:** [April 26, 2024, 10:09am UTC](https://discuss.huggingface.co/t/getting-issue-on-query-engine-for-llm-completion-callback/82780/2 "2024-04-26T10:09:32Z")

</div>

Resolution

pip install llama-index-llms-huggingface  
from llama\_index.core import VectorStoreIndex, SimpleDirectoryReader, ServiceContext

from llama\_index.llms.huggingface import HuggingFaceLLM

from llama\_index.core import PromptTemplate

from llama\_index.core import Settings

import torch  
from transformers import BitsAndBytesConfig  
quantization\_config = BitsAndBytesConfig(  
load\_in\_8bit=True,  
# bnb\_8bit\_compute\_dtype=torch.float16,  
# bnb\_8bit\_quant\_type=“nf8”,  
# bnb\_8bit\_use\_double\_quant=True,  
)

import torch

llm = HuggingFaceLLM(  
context\_window=4096,  
max\_new\_tokens=256,  
generate\_kwargs={“temperature”:0.5,“do\_sample”:False},  
system\_prompt=system\_prompt,  
query\_wrapper\_prompt=query\_wrapper\_prompt,  
tokenizer\_name=‘meta-llama/Llama-2-7b-chat-hf’,  
model\_name=‘meta-llama/Llama-2-7b-chat-hf’,  
model\_kwargs={“torch\_dtype”: torch.float16, “quantization\_config”: quantization\_config},  
device\_map=‘auto’  
)

from llama\_index.core import Settings  
Settings.llm = llm  
Settings.embed\_model = embed\_model  
Settings.node\_parser = SentenceSplitter(chunk\_size=1024, chunk\_overlap=20)

index = VectorStoreIndex.from\_documents(documents, show\_progress=True)

query\_engine=index.as\_query\_engine(llm=llm, streaming=True)  
response = query\_engine.query(“Your query here”)

print(response)