MoraxCheng/ohmyshot-paddleocr-vl-1.6-mlx

PaddleOCR-VL-1.6 for MLX (OhMyShot model pack)

PaddlePaddle's PaddleOCR-VL-1.6, a 0.9B vision-language model that reads text, tables, formulas, and charts in a picture, on MLX with 8-bit weights, and PP-DocLayoutV3, which finds the tables and charts, as Core ML, for OhMyShot, a screenshot app for macOS, which downloads them as a model pack: the text it recognizes then holds the tables and charts in a picture a row a line, cells apart by tabs, and its Recognized Text window shows them as tables. The weights are PaddlePaddle's, quantized; see LICENSE.txt for the license and what was changed. While it reads, the reading process peaks near 2.6 GB of memory.

File	What it is
config.json	The model's configuration, as mlx-vlm writes it (paddleocr_vl; quantization: 8 bits, groups of 64).
model.safetensors	The weights, as mlx-vlm names them (visual.*, language_model.*); a module with .scales and .biases beside its .weight is quantized.
vocabulary.json	The tokens by id, a JSON array of strings: ▁ is a space, <0xHH> a byte of UTF-8 text.
prompts.json	Each task's prompt around the picture's tokens, as ids: {"table": {"before": […], "after": […]}, …}; the picture's tokens (id 100295) go between, one for each 2 × 2 patches of 14 pixels.
layout.mlpackage	PP-DocLayoutV3: input pixel_values, 1 × 3 × 800 × 800 (value / 255); outputs logits, 1 × 300 × 25, and boxes, 1 × 300 × 4. Class 21 is a table, 3 a chart.
layout.json	The layout model's input, the classes read, and the threshold (0.5).
LICENSE.txt	Apache License 2.0, with what was changed.

A table reads as OTSL (<fcel> a cell, <ecel> an empty one, <lcel>/<ucel>/<xcel> merged, <nl> a row's end), a chart as rows with cells apart by |. Pictures are scaled with PIL's bicubic filter to whole 28-pixel blocks between 112,896 and 1,003,520 pixels; decoding is greedy and ends at token 2.

Made with Tools/model-packs/vlm_pack.py from these upstream revisions:

• https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6/tree/c5630abae1d940eafe0697512a0325494b02ab42
• https://huggingface.co/PaddlePaddle/PP-DocLayoutV3_safetensors/tree/e4f489dc536556fc1be02b973ba06756e0d4a2ac

Checked with OhMyShot's Swift reader (Packages/OhMyShotVLM), whose tokens match mlx-vlm's on the pictures compared: text in the OhMyShot benchmark's 40 pictures read with "OCR:", content character error rate 3.97%; 10 screen-style tables read with "Table Recognition:", 261 of 261 cells right, 10 of 10 in the right shape.