oQ4e-quantized MLX build (created by xunlinkx with the omlx pipeline) of Mondk/Spark-X2.5-4B-Abliterated-Uncensored for first-class, all-local serving on Apple Silicon via omlx (the MLX OpenAI-compatible server) and mlx-lm. Attribution chain: XHToken/Spark-X2.5-4B (official base, Apache-2.0) → Mondk’s abliterated/uncensored adaptation → xunlinkx’s fp16 MLX conversion + oQ4e quantization (this repo). This is not a new model — it is a quantization of the existing checkpoint, produced with omlx’s quantizeoqstreaming(…, oqlevel=4, enhanced=True)` pipeline: — Why we made it: run a full 4B Spark-X2.5-class model locally in ~2.5 GB of memory at usable decode rates (≈44 tok/s on an M3 Pro), with the attention gates and projection heads kept at higher precision so reasoning quality survives 4-bit compression. The -fp16 suffix records that the quantizer consumed full-precision float16 source weights (best-fidelity input), as opposed to bf16 or already-quantized sources. — Quantization: method oQ4e (omlx enhanced streaming imatrix calibration, 128×512 blocks), mixed-bit per-layer layout embedded in config.json (quantization / quantizationconfig`) so the exact layout is restored…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: xunlinkx
Теги: mlx, spark2_5, omlx, oq4e, 4-bit, quantization, apple-silicon, conversational
Лайков: 3 | Загрузок: 625
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
