k-m-irfan/Omni-Embed-Mini-0.9B-onnx

Omni-Embed Mini 0.9B — ONNX

ONNX export of the Omni-Embed Mini 0.9B multimodal embedding model, built to run in a browser on WebGPU. One model embeds text, images, video, speech, general audio and document pages into a single 1024-dimensional space.

This is a six-graph bundle driven from JavaScript, not a transformers.js architecture: the splice, the pooling and two of the vision tower's three inputs are computed by the caller. See manifest.json for the preconditions a caller cannot read off the graphs.

Component	Graph	Notes
backbone	backbone/model.onnx	takes inputs_embeds; pooling is applied outside the graph
embedding table	backbone/embed_tokens.onnx	read row-wise off disk rather than held in a session
vision	vision_encoder/model.onnx	resolution-independent; one image per call
audio	whisper_encoder/, dasheng_encoder/	mel front end is computed caller-side
projectors	projectors/*.onnx	image, video, and one per audio encoder

Every component was parity-checked against its PyTorch reference before export; */conversion_metadata.json and */parity_report.json carry the measured numbers.

Preprocessing is part of the model

A still image is squared to 224 with PIL bilinear, then the processor takes it to 256 with a Pillow-compatible bicubic. At 64 merged visual tokens this is not a detail: substituting a browser canvas resize moves the final embedding by cos 0.734. Any client that does not reproduce that chain is a different model, and its vectors must not be mixed into an index built with this one.

Demo

Built for a multimodal retrieval demo whose document index is computed offline in PyTorch while queries are embedded in-browser — the two halves share a space only because of the preprocessing note above.