Talkie 1930 in transformers format
There's 2 versions of Talkie 1930 that were converted to transformers format, already with config.json, ninja template and .safetensors files
Is it possible to convert one of those to gguf?
https://huggingface.co/lewtun/talkie-1930-13b-it-hf
https://huggingface.co/Abstract4700/talkie-1930-13b-it-hf-transformers
wonderful, someone requested not that long ago. since they seem to be identical, I will queue https://huggingface.co/lewtun/talkie-1930-13b-it-hf
hopefully it's supported
You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#talkie-1930-13b-it-hf-GGUF for quants to appear.
TalkieForCausalLM not compatible =(
please remind me in a week or something, it might be merged into main llama cpp and work =)
you can also try to talk to the author about helping adding compatibility for their model in llamacpp
Someone published a q5 gguf of talkie 1930
https://huggingface.co/sol-wy/talkie-1930-13b-it-q5
said it's not compatible with stock llama.cpp or stock LM Studio because Talkie uses a custom architecture.
but they provide custom patched llama.cpp version to run it
https://github.com/solwyc/talkie-1930-13b-it-q5
no information about those changes in main llama cpp yet
And here's the patch that was applied to llama.cpp to make it work with llama
https://github.com/solwyc/talkie-1930-13b-it-q5/tree/main/patches
we sadly dont support the custom forks, we need it to be merged into main llamacpp