cstr/embeddinggemma-300m-GGUF

embeddinggemma-300m GGUF

GGUF format of google/embeddinggemma-300m for use with CrispEmbed and Ollama-compatible runtimes.

Google EmbeddingGemma 300M. Lightweight multilingual embedding model based on Gemma 3, optimized for search, retrieval, and semantic similarity across 100+ languages.

Model details

• Architecture: Gemma 3 transformer (300M params), bidirectional attention
• Pooling: Mean pooling + Dense projection (768→3072→768) + L2 normalize
• Embedding dimension: 768
• Languages: 100+ languages
• Context length: 2,048 tokens
• License: Gemma

Files

File	Quantization	Size	Parity (cos vs HF)
embeddinggemma-300m.gguf	F32	~1.2 GB	1.0000
embeddinggemma-300m-q8_0.gguf	Q8_0	~327 MB	0.9998
embeddinggemma-300m-q5_k.gguf	Q5_K	~289 MB	0.9954
embeddinggemma-300m-q4_k.gguf	Q4_K	~277 MB	0.9834
embeddinggemma-300m-qat-q8_0-dense.gguf	Q8_0 (QAT)	~347 MB	0.984 vs vanilla HF

Q4_K shows mild degradation (~1.7%) which is typical for 4-bit quantization on embedding models. Use Q8_0 or Q5_K if highest fidelity is needed.

embeddinggemma-300m-qat-q8_0-dense.gguf (community / gemma-embedding arch)

This is the official llama.cpp QAT export (ggml-org/embeddinggemma-300m-qat-q8_0-GGUF, general.architecture = gemma-embedding) with the SentenceTransformers Dense head (2_Dense/3_Dense from google/embeddinggemma-300m) baked in as dense.0/1.weight. The upstream gemma-embedding GGUF omits the Dense modules (llama.cpp applies them from an external --sentence-transformers-dense-modules file); without them the output is the raw backbone mean-pool, orthogonal to real EmbeddingGemma. With Dense baked, CrispEmbed reproduces the HF SentenceTransformer output at cos 0.984 (the QAT checkpoint's weights differ slightly from vanilla google/embeddinggemma-300m, so this is measured vs vanilla — not a defect). Reproduce with models/add-st-dense-to-gguf.py.

Quick Start

# With CrispEmbed
crispembed -m embeddinggemma-300m.gguf "Hello world"

See CrispEmbed for full documentation.

Notes

These GGUFs use the Ollama-compatible format with CrispEmbed extension keys:

• gemma3.is_bidirectional = 1 — bidirectional (no causal mask)
• gemma3.pooling_type = 1 — mean pooling
• gemma3.rope.freq_base_local = 10000.0 — sliding-window RoPE theta
• Dense projection weights stored in F32 for correctness across all quant levels