gguf support

#8
by Shiza001 - opened

Hi. Firstly, congratulations for this incredible work. It's because you that I can use minimax H3 without depending bad quants like Q2. Anyway, I'm using Qwen3-vl-8b-instruct as text encoder, the fp8 version from boogu works nice, so I tried unsloth's gguf version (Q8_0) due it's size and quality, but it returned this error:
[INFO] gguf qtypes: Q8_0 (254), F32 (145)
[WARNING] Dequantizing token_embd.weight to prevent runtime OOM.
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] [ClipProj] mmh3-8b-ClipProj-v3.1-mlp.safetensors | tap 24 | 4096 -> 5120 | residual only | cos_test 0.8207
[ERROR] !!! Exception during processing !!! 'Qwen3_8B' object has no attribute 'preprocess_embed'
[ERROR] Traceback (most recent call last):
File "/content/drive/MyDrive/ComfyUI/execution.py", line 545, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/drive/MyDrive/ComfyUI/execution.py", line 344, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/content/drive/MyDrive/ComfyUI/execution.py", line 318, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "/content/drive/MyDrive/ComfyUI/execution.py", line 306, in process_inputs
result = f(**inputs)
File "/content/drive/MyDrive/ComfyUI/comfy_api/internal/init.py", line 149, in wrapped_func
return method(locked_class, **inputs)
File "/content/drive/MyDrive/ComfyUI/comfy_api/latest/_io.py", line 1990, in EXECUTE_NORMALIZED
to_return = cls.execute(*args, **kwargs)
File "/content/drive/MyDrive/ComfyUI/comfy_extras/nodes_minimax_h3.py", line 142, in execute
cond = clip.encode_from_tokens_scheduled(tokens)
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-ClipProj/clipproj_nodes.py", line 561, in encode_from_tokens_scheduled
cond, tags = self._encode(tokens)
~~~~~~~~~~~~^^^^^^^^
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-ClipProj/clipproj_nodes.py", line 427, in _encode
install_video_blocks(sm)
~~~~~~~~~~~~~~~~~~~~^^^^
File "/content/drive/MyDrive/ComfyUI/custom_nodes/ComfyUI-ClipProj/clipproj_nodes.py", line 202, in install_video_blocks
original = tr.preprocess_embed
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/dist-packages/torch/nn/modules/module.py", line 1968, in getattr
raise AttributeError(
f"'{type(self).name}' object has no attribute '{name}'"
)
AttributeError: 'Qwen3_8B' object has no attribute 'preprocess_embed'

Shiza001 changed discussion title from No gguf support? to gguf support

In my workflow, the clip config was:
Clip loader (gguf): Qwen3-vl-8b-Instruct.gguf > type: Boogu > Apply ClipProj: mmh3-8b-ClipProj-v3.1-mlp.safetensors
×
Note that I tried this config because the all-in-one node didn't recognized the gguf text encoder...

Sign up or log in to comment