Instructions to use srv-sngh/LocateAnything-3B-mlx-nvfp4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use srv-sngh/LocateAnything-3B-mlx-nvfp4 with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("srv-sngh/LocateAnything-3B-mlx-nvfp4")
config = load_config("srv-sngh/LocateAnything-3B-mlx-nvfp4")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use srv-sngh/LocateAnything-3B-mlx-nvfp4 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "srv-sngh/LocateAnything-3B-mlx-nvfp4"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "srv-sngh/LocateAnything-3B-mlx-nvfp4"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use srv-sngh/LocateAnything-3B-mlx-nvfp4 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "srv-sngh/LocateAnything-3B-mlx-nvfp4"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default srv-sngh/LocateAnything-3B-mlx-nvfp4

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use srv-sngh/LocateAnything-3B-mlx-nvfp4 with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "srv-sngh/LocateAnything-3B-mlx-nvfp4"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "srv-sngh/LocateAnything-3B-mlx-nvfp4" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

LocateAnything-3B — MLX nvfp4 (for Krill)

A mixed-precision nvfp4 MLX build of nvidia/LocateAnything-3B, the NVIDIA visual-grounding VLM that locates anything in an image as bounding boxes (<box><x1><y1><x2><y2></box>, coordinates normalized to 0–1000). Runs natively on Apple Silicon via Krill — no Python, no PyTorch.

Why this build

Complete VLM, quantized end-to-end for MLX. The MoonViT (Kimi-VL) native-resolution vision tower is included and kept at full precision; only the Qwen2.5-3B text decoder is quantized, so grounding accuracy is preserved.
Grounding parity with bf16. On a synthetic check, boxes are identical to the bf16 source (e.g. a red rectangle at true 125/125/375/375 → 125/125/376/377).
Small and fast. ~3.1 GB (vs 7.8 GB bf16); ~48 tok/s decode, ~1.1 s time-to-first-box on an M-series Mac — fits any 16 GB machine with room to spare.
Native Krill runtime. Custom Swift+MLX MoonViT tower + connector, logit-parity-verified against the NVIDIA reference.

Quantization

Mixed precision produced by krill quantize --mode nvfp4:

Base: nvfp4 (4-bit, group 16) on the Qwen2.5-3B text Linears + embeddings.
Protected: 8-bit affine (group 64) on every o_proj and down_proj (quality).
MoonViT vision tower + mlp1 connector: fp (not quantized).

Usage (Krill)

# by alias (ships with Krill):
krill run locateanything-3b "Locate the red car." --image street.jpg

# or directly from this repo:
krill run srv-sngh/LocateAnything-3B-mlx-nvfp4 "Detect all people." --image crowd.jpg

Output is grounding text with boxes, e.g. <ref>the red car</ref><box><612><448><770><690></box>. Coordinates are 0–1000; scale by the image width/height to get pixels.

License & attribution

This is a derivative work of nvidia/LocateAnything-3B and is distributed under the same NVIDIA License (see LICENSE). Use is limited to non-commercial research or evaluation purposes only (NVIDIA and its affiliates excepted). The base model builds on Qwen2.5 (Qwen Research License) and MoonViT (MIT). Please cite and follow NVIDIA's original terms.

About this build

Quantized and tested by Sourav Singh for Krill, a Mac-native LLM runtime and coding agent in one Swift + MLX binary. More Apple Silicon builds: huggingface.co/srv-sngh. Notes on building with local models and frontier AI: souravailabs.ai · GitHub · LinkedIn · YouTube